[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:15:45.875969  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:36349
I20260709 13:15:45.877561  5410 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:15:45.878659  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:45.895824  5416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:45.895859  5419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:45.895896  5417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:45.897323  5410 server_base.cc:1061] running on GCE node
I20260709 13:15:47.101601  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:47.101897  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:47.102056  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602947102038 us; error 0 us; skew 500 ppm
I20260709 13:15:47.102996  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:47.108457  5410 webserver.cc:533] Webserver started at http://127.5.72.190:37727/ using document root <none> and password file <none>
I20260709 13:15:47.109328  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:47.109535  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:47.109949  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:47.115957  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "63dc443bd46a434a8a91c1d2c99c5acd"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.124109  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:15:47.129426  5425 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.130724  5410 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:15:47.131043  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "63dc443bd46a434a8a91c1d2c99c5acd"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.131343  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:47.185526  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:47.187114  5410 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:15:47.187518  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:47.259024  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:36349
I20260709 13:15:47.259173  5476 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:36349 every 8 connection(s)
I20260709 13:15:47.267006  5477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:47.288889  5477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd: Bootstrap starting.
I20260709 13:15:47.295135  5477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:47.297072  5477 log.cc:826] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd: Log is configured to *not* fsync() on all Append() calls
I20260709 13:15:47.301681  5477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd: No bootstrap required, opened a new log
I20260709 13:15:47.322039  5477 raft_consensus.cc:359] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" member_type: VOTER }
I20260709 13:15:47.322589  5477 raft_consensus.cc:385] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:47.322887  5477 raft_consensus.cc:740] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63dc443bd46a434a8a91c1d2c99c5acd, State: Initialized, Role: FOLLOWER
I20260709 13:15:47.324079  5477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" member_type: VOTER }
I20260709 13:15:47.324592  5477 raft_consensus.cc:399] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:47.324898  5477 raft_consensus.cc:493] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:47.325248  5477 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:47.329388  5477 raft_consensus.cc:515] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" member_type: VOTER }
I20260709 13:15:47.330163  5477 leader_election.cc:304] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63dc443bd46a434a8a91c1d2c99c5acd; no voters: 
I20260709 13:15:47.331923  5477 leader_election.cc:290] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:47.332289  5480 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:47.334488  5480 raft_consensus.cc:697] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 1 LEADER]: Becoming Leader. State: Replica: 63dc443bd46a434a8a91c1d2c99c5acd, State: Running, Role: LEADER
I20260709 13:15:47.335482  5480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" member_type: VOTER }
I20260709 13:15:47.336519  5477 sys_catalog.cc:565] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:47.346591  5482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63dc443bd46a434a8a91c1d2c99c5acd. Latest consensus state: current_term: 1 leader_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" member_type: VOTER } }
I20260709 13:15:47.347388  5482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:47.346586  5481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63dc443bd46a434a8a91c1d2c99c5acd" member_type: VOTER } }
I20260709 13:15:47.349225  5481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:47.350853  5489 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:47.362178  5489 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:47.364812  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:47.380637  5489 catalog_manager.cc:1383] Generated new cluster ID: b4efc6fbaf5d4b07a56d03b074cc4121
I20260709 13:15:47.381008  5489 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:47.416715  5489 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:47.418947  5489 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:47.433840  5489 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd: Generated new TSK 0
I20260709 13:15:47.435066  5489 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:47.495888  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:47.504643  5499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:47.505739  5500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:47.510800  5502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:47.511504  5410 server_base.cc:1061] running on GCE node
I20260709 13:15:47.512494  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:47.512787  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:47.513062  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602947513042 us; error 0 us; skew 500 ppm
I20260709 13:15:47.514017  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:47.518239  5410 webserver.cc:533] Webserver started at http://127.5.72.129:44287/ using document root <none> and password file <none>
I20260709 13:15:47.518987  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:47.519253  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:47.519603  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:47.521116  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "313fbe57577947fe985adef5aeb1ec03"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.528195  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.004s
I20260709 13:15:47.533365  5507 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.534264  5410 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:15:47.534559  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "313fbe57577947fe985adef5aeb1ec03"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.534955  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:47.563710  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:47.565176  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:47.568141  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:47.572587  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:47.572870  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.573168  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:47.573377  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.679024  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:38669
I20260709 13:15:47.679101  5569 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:38669 every 8 connection(s)
I20260709 13:15:47.686913  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:47.702684  5574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:47.704497  5575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:47.709846  5410 server_base.cc:1061] running on GCE node
W20260709 13:15:47.710582  5577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:47.711813  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:47.712153  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:47.712415  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602947712391 us; error 0 us; skew 500 ppm
I20260709 13:15:47.713263  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:47.716596  5410 webserver.cc:533] Webserver started at http://127.5.72.130:43911/ using document root <none> and password file <none>
I20260709 13:15:47.717166  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:47.717381  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:47.717665  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:47.718989  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "59aa9b814c304015a0b4696951184f80"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.725663  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:15:47.725805  5570 heartbeater.cc:344] Connected to a master server at 127.5.72.190:36349
I20260709 13:15:47.726292  5570 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:47.727336  5570 heartbeater.cc:507] Master 127.5.72.190:36349 requested a full tablet report, sending...
I20260709 13:15:47.730759  5442 ts_manager.cc:194] Registered new tserver with Master: 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669)
I20260709 13:15:47.731309  5582 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.732229  5410 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:15:47.732517  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "59aa9b814c304015a0b4696951184f80"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.732820  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:47.733922  5442 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60536
I20260709 13:15:47.752198  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:47.753490  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:47.755690  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:47.758574  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:47.758880  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.759141  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:47.759325  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.852629  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:35145
I20260709 13:15:47.852725  5644 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:35145 every 8 connection(s)
I20260709 13:15:47.858927  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:47.867710  5648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:47.870669  5649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:47.874938  5645 heartbeater.cc:344] Connected to a master server at 127.5.72.190:36349
I20260709 13:15:47.875370  5645 heartbeater.cc:461] Registering TS with master...
W20260709 13:15:47.876179  5651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:47.876390  5645 heartbeater.cc:507] Master 127.5.72.190:36349 requested a full tablet report, sending...
I20260709 13:15:47.876818  5410 server_base.cc:1061] running on GCE node
I20260709 13:15:47.877774  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:47.878026  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:47.878222  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602947878202 us; error 0 us; skew 500 ppm
I20260709 13:15:47.878876  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:47.878901  5442 ts_manager.cc:194] Registered new tserver with Master: 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:47.880679  5442 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60538
I20260709 13:15:47.881778  5410 webserver.cc:533] Webserver started at http://127.5.72.131:43921/ using document root <none> and password file <none>
I20260709 13:15:47.882349  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:47.882635  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:47.882963  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:47.884078  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "c968d75da1aa478db16e3de8072583b6"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.888679  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:15:47.892573  5656 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.893407  5410 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:15:47.893743  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "c968d75da1aa478db16e3de8072583b6"
format_stamp: "Formatted at 2026-07-09 13:15:47 on dist-test-slave-n5kz"
I20260709 13:15:47.894080  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:47.918365  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:47.919940  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:47.922400  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:47.925974  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:47.926229  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:47.926528  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:47.926762  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:15:48.011216  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:44261
I20260709 13:15:48.011282  5718 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:44261 every 8 connection(s)
I20260709 13:15:48.027995  5719 heartbeater.cc:344] Connected to a master server at 127.5.72.190:36349
I20260709 13:15:48.028373  5719 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:48.029146  5719 heartbeater.cc:507] Master 127.5.72.190:36349 requested a full tablet report, sending...
I20260709 13:15:48.031633  5442 ts_manager.cc:194] Registered new tserver with Master: c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:48.031780  5410 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01518517s
I20260709 13:15:48.033428  5442 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60552
I20260709 13:15:48.091534  5442 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60554:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:48.096369  5442 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:48.209621  5684 tablet_service.cc:1511] Processing CreateTablet for tablet 4386c4eaca784bd6bb100d33b51ca769 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:48.211869  5683 tablet_service.cc:1511] Processing CreateTablet for tablet cf9561610c3945f6b15485ba6a8116d0 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:48.213685  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf9561610c3945f6b15485ba6a8116d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.217409  5684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4386c4eaca784bd6bb100d33b51ca769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.217660  5682 tablet_service.cc:1511] Processing CreateTablet for tablet 84232878e64440a08a417561c944da47 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:48.217990  5681 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7e0a08ded54091a1c12b9fa253b38a (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:48.221858  5682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84232878e64440a08a417561c944da47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.223495  5681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7e0a08ded54091a1c12b9fa253b38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.275357  5679 tablet_service.cc:1511] Processing CreateTablet for tablet e5534c1666ec4b819cd64abef9c4126d (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:48.282681  5533 tablet_service.cc:1511] Processing CreateTablet for tablet 84232878e64440a08a417561c944da47 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:48.284164  5533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84232878e64440a08a417561c944da47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.290669  5679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5534c1666ec4b819cd64abef9c4126d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.287632  5535 tablet_service.cc:1511] Processing CreateTablet for tablet 4386c4eaca784bd6bb100d33b51ca769 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:48.288107  5534 tablet_service.cc:1511] Processing CreateTablet for tablet cf9561610c3945f6b15485ba6a8116d0 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:48.292219  5610 tablet_service.cc:1511] Processing CreateTablet for tablet 4386c4eaca784bd6bb100d33b51ca769 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:48.284595  5531 tablet_service.cc:1511] Processing CreateTablet for tablet e5534c1666ec4b819cd64abef9c4126d (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:48.298136  5530 tablet_service.cc:1511] Processing CreateTablet for tablet 2793b38d6e2c4175bbfce95233f44810 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:48.293363  5532 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7e0a08ded54091a1c12b9fa253b38a (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:48.302093  5534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf9561610c3945f6b15485ba6a8116d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.300083  5609 tablet_service.cc:1511] Processing CreateTablet for tablet cf9561610c3945f6b15485ba6a8116d0 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:48.307461  5535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4386c4eaca784bd6bb100d33b51ca769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.306073  5608 tablet_service.cc:1511] Processing CreateTablet for tablet 84232878e64440a08a417561c944da47 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:48.311365  5532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7e0a08ded54091a1c12b9fa253b38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.313503  5607 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7e0a08ded54091a1c12b9fa253b38a (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:48.320396  5605 tablet_service.cc:1511] Processing CreateTablet for tablet 2793b38d6e2c4175bbfce95233f44810 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:48.323266  5606 tablet_service.cc:1511] Processing CreateTablet for tablet e5534c1666ec4b819cd64abef9c4126d (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:48.327590  5530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2793b38d6e2c4175bbfce95233f44810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.322978  5610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4386c4eaca784bd6bb100d33b51ca769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.333120  5680 tablet_service.cc:1511] Processing CreateTablet for tablet 2793b38d6e2c4175bbfce95233f44810 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:48.334729  5608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84232878e64440a08a417561c944da47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.337764  5609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf9561610c3945f6b15485ba6a8116d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.338567  5680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2793b38d6e2c4175bbfce95233f44810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.339315  5531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5534c1666ec4b819cd64abef9c4126d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.350328  5606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5534c1666ec4b819cd64abef9c4126d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.348994  5529 tablet_service.cc:1511] Processing CreateTablet for tablet 223d4ab346c24f6c8cce3ba665ceac07 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:48.357172  5529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223d4ab346c24f6c8cce3ba665ceac07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.357277  5607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7e0a08ded54091a1c12b9fa253b38a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:48.359251  5490 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:48.364106  5605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2793b38d6e2c4175bbfce95233f44810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.382018  5737 tablet_bootstrap.cc:492] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.382077  5604 tablet_service.cc:1511] Processing CreateTablet for tablet 223d4ab346c24f6c8cce3ba665ceac07 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:48.383970  5604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223d4ab346c24f6c8cce3ba665ceac07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.446403  5684 tablet_service.cc:1511] Processing CreateTablet for tablet 223d4ab346c24f6c8cce3ba665ceac07 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:48.447798  5684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 223d4ab346c24f6c8cce3ba665ceac07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.453651  5528 tablet_service.cc:1511] Processing CreateTablet for tablet 97a3c3a7823c4eebbebb1762fae68bd2 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:48.457296  5528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a3c3a7823c4eebbebb1762fae68bd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.472779  5603 tablet_service.cc:1511] Processing CreateTablet for tablet 97a3c3a7823c4eebbebb1762fae68bd2 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:48.475204  5603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a3c3a7823c4eebbebb1762fae68bd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.474442  5682 tablet_service.cc:1511] Processing CreateTablet for tablet 97a3c3a7823c4eebbebb1762fae68bd2 (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:48.477873  5682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a3c3a7823c4eebbebb1762fae68bd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.503713  5737 tablet_bootstrap.cc:654] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.506201  5527 tablet_service.cc:1511] Processing CreateTablet for tablet 09e603edad6c4f86819273f5646dd75c (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:48.511693  5527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e603edad6c4f86819273f5646dd75c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.540850  5739 tablet_bootstrap.cc:492] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:48.540786  5602 tablet_service.cc:1511] Processing CreateTablet for tablet 09e603edad6c4f86819273f5646dd75c (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:48.543690  5602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e603edad6c4f86819273f5646dd75c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.557062  5738 tablet_bootstrap.cc:492] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:48.575865  5682 tablet_service.cc:1511] Processing CreateTablet for tablet 09e603edad6c4f86819273f5646dd75c (DEFAULT_TABLE table=test-workload [id=378fc217e3a742829a261f367551b315]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:48.577077  5682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e603edad6c4f86819273f5646dd75c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.579674  5737 tablet_bootstrap.cc:492] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.580144  5737 ts_tablet_manager.cc:1403] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.221s	user 0.021s	sys 0.004s
I20260709 13:15:48.593202  5737 raft_consensus.cc:359] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.595381  5737 raft_consensus.cc:385] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.595721  5737 raft_consensus.cc:740] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.596508  5737 consensus_queue.cc:260] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.597469  5739 tablet_bootstrap.cc:654] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.601492  5738 tablet_bootstrap.cc:654] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.606705  5737 ts_tablet_manager.cc:1434] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.010s
I20260709 13:15:48.607550  5719 heartbeater.cc:499] Master 127.5.72.190:36349 was elected leader, sending a full tablet report...
I20260709 13:15:48.607847  5737 tablet_bootstrap.cc:492] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.614212  5737 tablet_bootstrap.cc:654] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.617851  5738 tablet_bootstrap.cc:492] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:48.618413  5738 ts_tablet_manager.cc:1403] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.092s	user 0.008s	sys 0.015s
I20260709 13:15:48.621378  5738 raft_consensus.cc:359] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.622184  5738 raft_consensus.cc:385] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.622529  5738 raft_consensus.cc:740] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.623327  5738 consensus_queue.cc:260] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.627871  5739 tablet_bootstrap.cc:492] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:48.628163  5737 tablet_bootstrap.cc:492] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.628352  5739 ts_tablet_manager.cc:1403] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.090s	user 0.012s	sys 0.000s
I20260709 13:15:48.628592  5737 ts_tablet_manager.cc:1403] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.007s
I20260709 13:15:48.631376  5739 raft_consensus.cc:359] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.631778  5737 raft_consensus.cc:359] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.632467  5737 raft_consensus.cc:385] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.632786  5737 raft_consensus.cc:740] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.632133  5739 raft_consensus.cc:385] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.633438  5739 raft_consensus.cc:740] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.633534  5737 consensus_queue.cc:260] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.634338  5739 consensus_queue.cc:260] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.636224  5737 ts_tablet_manager.cc:1434] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:15:48.637151  5737 tablet_bootstrap.cc:492] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.639865  5570 heartbeater.cc:499] Master 127.5.72.190:36349 was elected leader, sending a full tablet report...
I20260709 13:15:48.640384  5738 ts_tablet_manager.cc:1434] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:15:48.641284  5738 tablet_bootstrap.cc:492] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:48.643990  5737 tablet_bootstrap.cc:654] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.647787  5738 tablet_bootstrap.cc:654] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.656032  5737 tablet_bootstrap.cc:492] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.656519  5737 ts_tablet_manager.cc:1403] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:15:48.659302  5737 raft_consensus.cc:359] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.660009  5737 raft_consensus.cc:385] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.660300  5737 raft_consensus.cc:740] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.661031  5737 consensus_queue.cc:260] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.663759  5737 ts_tablet_manager.cc:1434] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:15:48.664878  5737 tablet_bootstrap.cc:492] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.671845  5737 tablet_bootstrap.cc:654] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:48.687412  5571 tablet.cc:2406] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:15:48.692049  5741 raft_consensus.cc:493] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.692711  5741 raft_consensus.cc:515] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.696537  5741 leader_election.cc:290] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:48.710381  5739 ts_tablet_manager.cc:1434] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.000s
I20260709 13:15:48.711318  5645 heartbeater.cc:499] Master 127.5.72.190:36349 was elected leader, sending a full tablet report...
I20260709 13:15:48.728291  5739 tablet_bootstrap.cc:492] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:48.728940  5738 tablet_bootstrap.cc:492] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:48.729421  5738 ts_tablet_manager.cc:1403] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.026s
I20260709 13:15:48.730046  5737 tablet_bootstrap.cc:492] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.730494  5737 ts_tablet_manager.cc:1403] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.003s
I20260709 13:15:48.733856  5737 raft_consensus.cc:359] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.734987  5737 raft_consensus.cc:385] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.735304  5737 raft_consensus.cc:740] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.736891  5737 consensus_queue.cc:260] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.739462  5737 ts_tablet_manager.cc:1434] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:15:48.740406  5737 tablet_bootstrap.cc:492] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.747404  5737 tablet_bootstrap.cc:654] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.747406  5739 tablet_bootstrap.cc:654] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.743531  5738 raft_consensus.cc:359] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.749244  5738 raft_consensus.cc:385] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.749568  5738 raft_consensus.cc:740] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.752465  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
W20260709 13:15:48.754751  5659 leader_election.cc:343] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.754752  5738 consensus_queue.cc:260] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.759207  5738 ts_tablet_manager.cc:1434] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:15:48.765738  5738 tablet_bootstrap.cc:492] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:48.766130  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:48.766796  5737 tablet_bootstrap.cc:492] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.767259  5737 ts_tablet_manager.cc:1403] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
W20260709 13:15:48.770363  5657 leader_election.cc:343] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.770910  5657 leader_election.cc:304] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c968d75da1aa478db16e3de8072583b6; no voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80
I20260709 13:15:48.771775  5741 raft_consensus.cc:2749] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.773386  5738 tablet_bootstrap.cc:654] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.770323  5737 raft_consensus.cc:359] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.778512  5737 raft_consensus.cc:385] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.778963  5737 raft_consensus.cc:740] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.779908  5737 consensus_queue.cc:260] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.782833  5737 ts_tablet_manager.cc:1434] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 13:15:48.783320  5745 raft_consensus.cc:493] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.784117  5737 tablet_bootstrap.cc:492] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.783784  5745 raft_consensus.cc:515] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.787844  5745 leader_election.cc:290] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:48.801941  5737 tablet_bootstrap.cc:654] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.804229  5738 tablet_bootstrap.cc:492] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:48.804733  5738 ts_tablet_manager.cc:1403] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.039s	user 0.004s	sys 0.010s
I20260709 13:15:48.807946  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:48.808050  5738 raft_consensus.cc:359] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.808821  5738 raft_consensus.cc:385] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.809131  5738 raft_consensus.cc:740] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
W20260709 13:15:48.810019  5508 leader_election.cc:343] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.809916  5738 consensus_queue.cc:260] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.813886  5738 ts_tablet_manager.cc:1434] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:48.815510  5738 tablet_bootstrap.cc:492] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:48.812269  5739 tablet_bootstrap.cc:492] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:48.816128  5739 ts_tablet_manager.cc:1403] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.017s
I20260709 13:15:48.819099  5739 raft_consensus.cc:359] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.826742  5739 raft_consensus.cc:385] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.827207  5739 raft_consensus.cc:740] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.827088  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:48.828281  5739 consensus_queue.cc:260] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
W20260709 13:15:48.828821  5508 leader_election.cc:343] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.829293  5508 leader_election.cc:304] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03; no voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6
I20260709 13:15:48.830060  5745 raft_consensus.cc:2749] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:48.830550  5737 tablet_bootstrap.cc:492] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.831022  5737 ts_tablet_manager.cc:1403] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.007s
I20260709 13:15:48.831269  5739 ts_tablet_manager.cc:1434] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:15:48.832233  5739 tablet_bootstrap.cc:492] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:48.826387  5738 tablet_bootstrap.cc:654] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.834295  5737 raft_consensus.cc:359] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.835100  5737 raft_consensus.cc:385] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.835412  5737 raft_consensus.cc:740] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.836269  5737 consensus_queue.cc:260] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.838832  5737 ts_tablet_manager.cc:1434] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:48.839816  5737 tablet_bootstrap.cc:492] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.842510  5739 tablet_bootstrap.cc:654] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.846500  5737 tablet_bootstrap.cc:654] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.853134  5738 tablet_bootstrap.cc:492] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:48.853713  5738 ts_tablet_manager.cc:1403] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.004s
I20260709 13:15:48.856585  5738 raft_consensus.cc:359] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.857937  5737 tablet_bootstrap.cc:492] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.858729  5739 tablet_bootstrap.cc:492] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:48.858899  5738 raft_consensus.cc:385] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.859174  5737 ts_tablet_manager.cc:1403] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:15:48.859294  5738 raft_consensus.cc:740] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.859395  5739 ts_tablet_manager.cc:1403] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.004s
I20260709 13:15:48.860100  5738 consensus_queue.cc:260] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.862267  5738 ts_tablet_manager.cc:1434] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.000s
I20260709 13:15:48.862674  5741 raft_consensus.cc:493] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.863230  5738 tablet_bootstrap.cc:492] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:48.863122  5741 raft_consensus.cc:515] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.865228  5741 leader_election.cc:290] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:48.867900  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:48.867983  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:48.869246  5545 raft_consensus.cc:2468] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 0.
W20260709 13:15:48.869521  5657 leader_election.cc:343] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.870733  5737 raft_consensus.cc:359] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.871434  5737 raft_consensus.cc:385] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.871584  5659 leader_election.cc:304] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 59aa9b814c304015a0b4696951184f80
I20260709 13:15:48.871737  5737 raft_consensus.cc:740] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.867573  5739 raft_consensus.cc:359] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.872395  5741 raft_consensus.cc:2804] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.872619  5739 raft_consensus.cc:385] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.872769  5741 raft_consensus.cc:493] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.872956  5739 raft_consensus.cc:740] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.873176  5741 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.871786  5738 tablet_bootstrap.cc:654] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.874432  5737 consensus_queue.cc:260] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.873715  5739 consensus_queue.cc:260] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.878182  5739 ts_tablet_manager.cc:1434] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:15:48.879156  5739 tablet_bootstrap.cc:492] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:48.881692  5741 raft_consensus.cc:515] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.885829  5739 tablet_bootstrap.cc:654] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.888710  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
W20260709 13:15:48.890393  5657 leader_election.cc:343] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:48.890290  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:48.891136  5545 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.892202  5737 ts_tablet_manager.cc:1434] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.004s
I20260709 13:15:48.893397  5737 tablet_bootstrap.cc:492] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.899103  5545 raft_consensus.cc:2468] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 1.
I20260709 13:15:48.900733  5737 tablet_bootstrap.cc:654] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.900543  5659 leader_election.cc:304] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 59aa9b814c304015a0b4696951184f80
I20260709 13:15:48.901482  5741 leader_election.cc:290] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:48.901528  5762 raft_consensus.cc:2804] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.904143  5762 raft_consensus.cc:697] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:48.905077  5762 consensus_queue.cc:237] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.910595  5738 tablet_bootstrap.cc:492] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:48.911255  5738 ts_tablet_manager.cc:1403] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.007s
I20260709 13:15:48.911315  5737 tablet_bootstrap.cc:492] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.911944  5737 ts_tablet_manager.cc:1403] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 13:15:48.915454  5737 raft_consensus.cc:359] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.915337  5738 raft_consensus.cc:359] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.916401  5737 raft_consensus.cc:385] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.916471  5738 raft_consensus.cc:385] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.916855  5737 raft_consensus.cc:740] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.916882  5738 raft_consensus.cc:740] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.918316  5739 tablet_bootstrap.cc:492] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:48.917872  5737 consensus_queue.cc:260] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.917848  5738 consensus_queue.cc:260] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.920619  5737 ts_tablet_manager.cc:1434] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:48.920938  5738 ts_tablet_manager.cc:1434] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:15:48.921547  5739 ts_tablet_manager.cc:1403] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.011s
I20260709 13:15:48.921612  5737 tablet_bootstrap.cc:492] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6: Bootstrap starting.
I20260709 13:15:48.922012  5738 tablet_bootstrap.cc:492] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:48.928689  5737 tablet_bootstrap.cc:654] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.929567  5738 tablet_bootstrap.cc:654] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.931265  5762 raft_consensus.cc:493] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.931763  5762 raft_consensus.cc:515] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.935267  5762 leader_election.cc:290] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:48.937196  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:48.937832  5620 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 0.
I20260709 13:15:48.941387  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:48.942029  5545 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 0.
I20260709 13:15:48.936355  5739 raft_consensus.cc:359] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.943403  5657 leader_election.cc:304] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:48.943464  5739 raft_consensus.cc:385] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.944680  5762 raft_consensus.cc:2804] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.944695  5739 raft_consensus.cc:740] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:48.945180  5762 raft_consensus.cc:493] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.945564  5762 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.948550  5442 catalog_manager.cc:5697] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:48.955304  5762 raft_consensus.cc:515] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.950380  5739 consensus_queue.cc:260] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.965013  5762 leader_election.cc:290] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:48.967096  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:48.968261  5545 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.968493  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:48.969123  5620 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.975586  5739 ts_tablet_manager.cc:1434] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:15:48.979135  5545 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 1.
I20260709 13:15:48.979346  5738 tablet_bootstrap.cc:492] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:48.980445  5659 leader_election.cc:304] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:48.981968  5738 ts_tablet_manager.cc:1403] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.015s
I20260709 13:15:48.984813  5738 raft_consensus.cc:359] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.985502  5738 raft_consensus.cc:385] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:48.985651  5762 raft_consensus.cc:2804] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:48.980890  5739 tablet_bootstrap.cc:492] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:48.986459  5762 raft_consensus.cc:697] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:48.987412  5762 consensus_queue.cc:237] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.990581  5737 tablet_bootstrap.cc:492] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6: No bootstrap required, opened a new log
I20260709 13:15:48.991107  5737 ts_tablet_manager.cc:1403] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.022s
I20260709 13:15:48.992183  5745 raft_consensus.cc:493] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.993077  5739 tablet_bootstrap.cc:654] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:48.992648  5745 raft_consensus.cc:515] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:48.994824  5745 leader_election.cc:290] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:48.985116  5620 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 1.
I20260709 13:15:48.995859  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:48.997030  5694 raft_consensus.cc:2468] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 0.
I20260709 13:15:48.998230  5508 leader_election.cc:304] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:48.995823  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:48.999007  5745 raft_consensus.cc:2804] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:48.999367  5745 raft_consensus.cc:493] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.999717  5745 raft_consensus.cc:3060] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:48.999635  5737 raft_consensus.cc:359] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.000344  5737 raft_consensus.cc:385] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:15:49.000278  5508 leader_election.cc:343] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.000842  5737 raft_consensus.cc:740] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.001973  5737 consensus_queue.cc:260] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.005733  5737 ts_tablet_manager.cc:1434] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:15:48.985821  5738 raft_consensus.cc:740] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.007922  5738 consensus_queue.cc:260] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.019556  5745 raft_consensus.cc:515] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.024287  5738 ts_tablet_manager.cc:1434] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.010s
I20260709 13:15:49.026253  5738 tablet_bootstrap.cc:492] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:49.036478  5738 tablet_bootstrap.cc:654] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.035662  5442 catalog_manager.cc:5697] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.049687  5745 leader_election.cc:290] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.050321  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
W20260709 13:15:49.052268  5508 leader_election.cc:343] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.053288  5739 tablet_bootstrap.cc:492] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:49.053632  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.053936  5739 ts_tablet_manager.cc:1403] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.077s	user 0.012s	sys 0.004s
I20260709 13:15:49.054047  5694 raft_consensus.cc:3060] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.058131  5739 raft_consensus.cc:359] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.060611  5694 raft_consensus.cc:2468] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 1.
I20260709 13:15:49.060756  5739 raft_consensus.cc:385] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.061141  5739 raft_consensus.cc:740] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.061769  5508 leader_election.cc:304] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.061947  5738 tablet_bootstrap.cc:492] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:49.062037  5739 consensus_queue.cc:260] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.062512  5738 ts_tablet_manager.cc:1403] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.013s
I20260709 13:15:49.062510  5745 raft_consensus.cc:2804] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.063328  5745 raft_consensus.cc:697] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.064594  5745 consensus_queue.cc:237] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.065584  5738 raft_consensus.cc:359] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.066263  5738 raft_consensus.cc:385] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.066569  5738 raft_consensus.cc:740] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.067301  5738 consensus_queue.cc:260] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.068349  5739 ts_tablet_manager.cc:1434] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 13:15:49.069324  5739 tablet_bootstrap.cc:492] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:49.069504  5738 ts_tablet_manager.cc:1434] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:15:49.070449  5738 tablet_bootstrap.cc:492] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:49.077765  5738 tablet_bootstrap.cc:654] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.089283  5738 tablet_bootstrap.cc:492] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:49.089756  5738 ts_tablet_manager.cc:1403] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260709 13:15:49.092617  5739 tablet_bootstrap.cc:654] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.092813  5738 raft_consensus.cc:359] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.093472  5738 raft_consensus.cc:385] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.093719  5738 raft_consensus.cc:740] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.094358  5738 consensus_queue.cc:260] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.094213  5442 catalog_manager.cc:5697] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.096529  5738 ts_tablet_manager.cc:1434] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:15:49.097412  5738 tablet_bootstrap.cc:492] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03: Bootstrap starting.
I20260709 13:15:49.102864  5738 tablet_bootstrap.cc:654] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.104780  5739 tablet_bootstrap.cc:492] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:49.105310  5739 ts_tablet_manager.cc:1403] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.004s
I20260709 13:15:49.108318  5739 raft_consensus.cc:359] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.109069  5739 raft_consensus.cc:385] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.109448  5739 raft_consensus.cc:740] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.110219  5739 consensus_queue.cc:260] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.112906  5739 ts_tablet_manager.cc:1434] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:49.114374  5746 raft_consensus.cc:493] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.114923  5746 raft_consensus.cc:515] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.115449  5738 tablet_bootstrap.cc:492] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03: No bootstrap required, opened a new log
I20260709 13:15:49.116011  5738 ts_tablet_manager.cc:1403] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 13:15:49.117895  5746 leader_election.cc:290] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.119223  5738 raft_consensus.cc:359] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.113916  5739 tablet_bootstrap.cc:492] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:49.119861  5738 raft_consensus.cc:385] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.120159  5738 raft_consensus.cc:740] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.120769  5738 consensus_queue.cc:260] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.122967  5738 ts_tablet_manager.cc:1434] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:49.127130  5739 tablet_bootstrap.cc:654] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.136560  5762 raft_consensus.cc:493] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.137065  5762 raft_consensus.cc:515] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.139271  5762 leader_election.cc:290] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:49.140372  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:49.140323  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:49.141321  5545 raft_consensus.cc:2468] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 0.
W20260709 13:15:49.141898  5657 leader_election.cc:343] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.143154  5659 leader_election.cc:304] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.144032  5762 raft_consensus.cc:2804] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.144474  5762 raft_consensus.cc:493] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.144881  5762 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.151450  5762 raft_consensus.cc:515] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.153455  5762 leader_election.cc:290] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:49.154525  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
W20260709 13:15:49.155916  5657 leader_election.cc:343] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:49.154551  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.157032  5545 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.159173  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:49.159971  5694 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 0.
I20260709 13:15:49.160209  5739 tablet_bootstrap.cc:492] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:49.160730  5739 ts_tablet_manager.cc:1403] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.047s	user 0.023s	sys 0.013s
I20260709 13:15:49.161712  5583 leader_election.cc:304] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.163004  5746 raft_consensus.cc:2804] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.163318  5746 raft_consensus.cc:493] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.163627  5746 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.163805  5739 raft_consensus.cc:359] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.164315  5545 raft_consensus.cc:2468] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 1.
I20260709 13:15:49.164575  5739 raft_consensus.cc:385] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.164915  5739 raft_consensus.cc:740] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.165536  5659 leader_election.cc:304] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.165685  5762 raft_consensus.cc:493] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.166190  5762 raft_consensus.cc:515] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.166831  5765 raft_consensus.cc:2804] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.167323  5765 raft_consensus.cc:697] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:49.165550  5739 consensus_queue.cc:260] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.168164  5765 consensus_queue.cc:237] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.169271  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:49.170078  5619 raft_consensus.cc:2468] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 0.
I20260709 13:15:49.170169  5746 raft_consensus.cc:515] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.171734  5657 leader_election.cc:304] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.172179  5746 leader_election.cc:290] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.172873  5741 raft_consensus.cc:2804] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.173310  5741 raft_consensus.cc:493] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.173653  5741 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.176285  5544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:49.176978  5544 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 0.
I20260709 13:15:49.168174  5762 leader_election.cc:290] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:49.178776  5544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.179359  5544 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.180716  5745 raft_consensus.cc:493] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.181146  5745 raft_consensus.cc:515] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.183233  5745 leader_election.cc:290] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.183352  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.183895  5694 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.188460  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:49.188913  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:49.189289  5619 raft_consensus.cc:2468] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 0.
I20260709 13:15:49.191032  5508 leader_election.cc:304] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: 
I20260709 13:15:49.191985  5745 raft_consensus.cc:2804] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.192348  5745 raft_consensus.cc:493] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.192682  5745 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.197252  5741 raft_consensus.cc:515] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.198493  5693 raft_consensus.cc:2393] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 313fbe57577947fe985adef5aeb1ec03 in current term 1: Already voted for candidate c968d75da1aa478db16e3de8072583b6 in this term.
I20260709 13:15:49.200320  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.203248  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:49.206049  5544 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 1.
I20260709 13:15:49.206290  5694 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 1.
I20260709 13:15:49.207363  5583 leader_election.cc:304] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.208460  5745 raft_consensus.cc:515] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.209807  5543 raft_consensus.cc:2393] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c968d75da1aa478db16e3de8072583b6 in current term 1: Already voted for candidate 313fbe57577947fe985adef5aeb1ec03 in this term.
I20260709 13:15:49.211818  5746 raft_consensus.cc:2804] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.212442  5746 raft_consensus.cc:697] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.212345  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.213223  5619 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.212836  5741 leader_election.cc:290] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:49.213523  5746 consensus_queue.cc:237] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.216187  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.218425  5657 leader_election.cc:304] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c968d75da1aa478db16e3de8072583b6; no voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.220554  5745 leader_election.cc:290] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.222965  5739 ts_tablet_manager.cc:1434] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.062s	user 0.040s	sys 0.021s
I20260709 13:15:49.223536  5741 raft_consensus.cc:2749] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.224138  5739 tablet_bootstrap.cc:492] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:49.222944  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.224834  5694 raft_consensus.cc:2393] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 313fbe57577947fe985adef5aeb1ec03 in current term 1: Already voted for candidate c968d75da1aa478db16e3de8072583b6 in this term.
I20260709 13:15:49.224485  5442 catalog_manager.cc:5697] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.226441  5619 raft_consensus.cc:2468] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 1.
I20260709 13:15:49.227731  5508 leader_election.cc:304] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: c968d75da1aa478db16e3de8072583b6
I20260709 13:15:49.229883  5745 raft_consensus.cc:2804] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.231057  5745 raft_consensus.cc:697] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.231741  5739 tablet_bootstrap.cc:654] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.231890  5745 consensus_queue.cc:237] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.243491  5442 catalog_manager.cc:5697] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.249895  5739 tablet_bootstrap.cc:492] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:49.250378  5739 ts_tablet_manager.cc:1403] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.003s
I20260709 13:15:49.253060  5739 raft_consensus.cc:359] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.253741  5739 raft_consensus.cc:385] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.254180  5739 raft_consensus.cc:740] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.253734  5440 catalog_manager.cc:5697] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.255319  5739 consensus_queue.cc:260] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.257431  5739 ts_tablet_manager.cc:1434] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:15:49.258088  5739 tablet_bootstrap.cc:492] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80: Bootstrap starting.
I20260709 13:15:49.262284  5739 tablet_bootstrap.cc:654] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:49.266597  5765 raft_consensus.cc:493] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.267277  5765 raft_consensus.cc:515] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.269984  5765 leader_election.cc:290] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:49.271006  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:49.271636  5619 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 0.
I20260709 13:15:49.273023  5544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:49.273638  5544 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 0.
I20260709 13:15:49.274521  5657 leader_election.cc:304] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.275355  5765 raft_consensus.cc:2804] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.275686  5765 raft_consensus.cc:493] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.275967  5765 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.276508  5739 tablet_bootstrap.cc:492] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80: No bootstrap required, opened a new log
I20260709 13:15:49.276933  5739 ts_tablet_manager.cc:1403] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:15:49.279312  5739 raft_consensus.cc:359] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.279805  5739 raft_consensus.cc:385] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:49.280025  5739 raft_consensus.cc:740] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Initialized, Role: FOLLOWER
I20260709 13:15:49.280625  5739 consensus_queue.cc:260] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.281788  5765 raft_consensus.cc:515] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.282305  5739 ts_tablet_manager.cc:1434] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:15:49.283738  5765 leader_election.cc:290] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:49.284353  5544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.284898  5544 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.286222  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.286799  5619 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.290208  5784 raft_consensus.cc:493] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.290678  5784 raft_consensus.cc:515] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.292848  5784 leader_election.cc:290] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.293516  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:49.294133  5543 raft_consensus.cc:2468] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 0.
I20260709 13:15:49.295274  5585 leader_election.cc:304] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: 
I20260709 13:15:49.295411  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:49.296051  5784 raft_consensus.cc:2804] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.295982  5694 raft_consensus.cc:2468] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 0.
I20260709 13:15:49.296523  5784 raft_consensus.cc:493] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.296900  5784 raft_consensus.cc:3060] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.298321  5544 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 1.
I20260709 13:15:49.299266  5659 leader_election.cc:304] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.299922  5765 raft_consensus.cc:2804] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.300336  5765 raft_consensus.cc:697] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:49.301113  5765 consensus_queue.cc:237] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.302215  5619 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 1.
I20260709 13:15:49.303856  5745 raft_consensus.cc:493] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.304476  5745 raft_consensus.cc:515] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.306327  5784 raft_consensus.cc:515] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.307085  5745 leader_election.cc:290] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.308305  5784 leader_election.cc:290] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.309589  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80" is_pre_election: true
I20260709 13:15:49.310055  5619 raft_consensus.cc:2468] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 0.
I20260709 13:15:49.311200  5508 leader_election.cc:304] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: 
I20260709 13:15:49.312052  5745 raft_consensus.cc:2804] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:49.312283  5745 raft_consensus.cc:493] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.312487  5745 raft_consensus.cc:3060] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.315063  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:49.315447  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.315693  5694 raft_consensus.cc:2468] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 0.
I20260709 13:15:49.315984  5693 raft_consensus.cc:3060] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.317381  5441 catalog_manager.cc:5697] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.321135  5745 raft_consensus.cc:515] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.322834  5693 raft_consensus.cc:2468] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 1.
I20260709 13:15:49.323879  5544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.324167  5619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.326339  5544 raft_consensus.cc:3060] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.326653  5619 raft_consensus.cc:3060] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.324704  5583 leader_election.cc:304] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.328728  5784 raft_consensus.cc:2804] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.329190  5784 raft_consensus.cc:697] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.329944  5784 consensus_queue.cc:237] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.332772  5544 raft_consensus.cc:2468] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 1.
I20260709 13:15:49.333294  5619 raft_consensus.cc:2468] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 1.
I20260709 13:15:49.325451  5745 leader_election.cc:290] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.334080  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.334372  5508 leader_election.cc:304] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: 
I20260709 13:15:49.334836  5693 raft_consensus.cc:3060] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.335371  5745 raft_consensus.cc:2804] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:49.335988  5745 raft_consensus.cc:697] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.336995  5745 consensus_queue.cc:237] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.342659  5693 raft_consensus.cc:2468] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 1.
I20260709 13:15:49.348644  5441 catalog_manager.cc:5697] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.351847  5442 catalog_manager.cc:5697] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.388262  5619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:47872
I20260709 13:15:49.388406  5765 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.388952  5619 raft_consensus.cc:606] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Received request to transfer leadership to TS 313fbe57577947fe985adef5aeb1ec03
I20260709 13:15:49.419947  5741 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.423514  5619 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.431963  5693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84232878e64440a08a417561c944da47"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
mode: GRACEFUL
new_leader_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:54130
I20260709 13:15:49.432783  5693 raft_consensus.cc:606] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Received request to transfer leadership to TS 313fbe57577947fe985adef5aeb1ec03
I20260709 13:15:49.440726  5693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
mode: GRACEFUL
new_leader_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:54130
I20260709 13:15:49.440552  5765 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:49.441533  5693 raft_consensus.cc:606] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Received request to transfer leadership to TS 313fbe57577947fe985adef5aeb1ec03
I20260709 13:15:49.448593  5765 raft_consensus.cc:993] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6: : Instructing follower 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:49.449867  5765 raft_consensus.cc:1081] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Signalling peer 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:49.451723  5762 raft_consensus.cc:993] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: : Instructing follower 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:49.452149  5762 raft_consensus.cc:1081] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Signalling peer 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:49.456357  5741 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.457871  5543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:34720
I20260709 13:15:49.458441  5543 raft_consensus.cc:493] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.458823  5543 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.461952  5784 raft_consensus.cc:493] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.462419  5784 raft_consensus.cc:515] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.463507  5544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84232878e64440a08a417561c944da47"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:34720
I20260709 13:15:49.464035  5544 raft_consensus.cc:493] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.464389  5544 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.464421  5784 leader_election.cc:290] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.466053  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:49.468464  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:49.469287  5693 raft_consensus.cc:2393] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 59aa9b814c304015a0b4696951184f80 in current term 1: Already voted for candidate 313fbe57577947fe985adef5aeb1ec03 in this term.
I20260709 13:15:49.470275  5583 leader_election.cc:304] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59aa9b814c304015a0b4696951184f80; no voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6
I20260709 13:15:49.471112  5544 raft_consensus.cc:515] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.471449  5784 raft_consensus.cc:3060] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.476787  5618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.479163  5543 raft_consensus.cc:515] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.480823  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.481689  5693 raft_consensus.cc:3055] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.480927  5544 leader_election.cc:290] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 2 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.482183  5693 raft_consensus.cc:740] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:49.481743  5543 leader_election.cc:290] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 2 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.485569  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.486156  5694 raft_consensus.cc:3055] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.486753  5694 raft_consensus.cc:740] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:49.488485  5618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.483153  5693 consensus_queue.cc:260] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.490456  5694 consensus_queue.cc:260] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.492280  5693 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.494127  5784 raft_consensus.cc:2749] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:49.494984  5694 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.505149  5746 raft_consensus.cc:493] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c968d75da1aa478db16e3de8072583b6)
I20260709 13:15:49.505719  5746 raft_consensus.cc:515] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.509622  5746 leader_election.cc:290] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.510370  5544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "313fbe57577947fe985adef5aeb1ec03" is_pre_election: true
I20260709 13:15:49.498678  5543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:34738
I20260709 13:15:49.511102  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c968d75da1aa478db16e3de8072583b6" is_pre_election: true
I20260709 13:15:49.511854  5543 raft_consensus.cc:606] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.514173  5585 leader_election.cc:304] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59aa9b814c304015a0b4696951184f80; no voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6
I20260709 13:15:49.514971  5746 raft_consensus.cc:2749] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:49.518692  5692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:54130
I20260709 13:15:49.519277  5692 raft_consensus.cc:606] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.519899  5693 raft_consensus.cc:2468] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 2.
I20260709 13:15:49.520908  5508 leader_election.cc:304] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.521584  5745 raft_consensus.cc:2804] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.522054  5745 raft_consensus.cc:697] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.522094  5694 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 2.
I20260709 13:15:49.523289  5508 leader_election.cc:304] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.523206  5745 consensus_queue.cc:237] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.526177  5745 raft_consensus.cc:2804] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.526599  5745 raft_consensus.cc:697] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.527051  5544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:34738
I20260709 13:15:49.527667  5544 raft_consensus.cc:606] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.527778  5745 consensus_queue.cc:237] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.531208  5442 catalog_manager.cc:5697] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 1 to 2, leader changed from c968d75da1aa478db16e3de8072583b6 (127.5.72.131) to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.534109  5694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:54130
I20260709 13:15:49.534720  5694 raft_consensus.cc:606] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:49.540836  5620 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:47872
I20260709 13:15:49.541426  5620 raft_consensus.cc:606] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Received request to transfer leadership to TS c968d75da1aa478db16e3de8072583b6
I20260709 13:15:49.543358  5442 catalog_manager.cc:5697] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 1 to 2, leader changed from c968d75da1aa478db16e3de8072583b6 (127.5.72.131) to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.598240  5745 consensus_queue.cc:1048] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.604588  5774 raft_consensus.cc:993] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.605351  5745 raft_consensus.cc:1081] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.609783  5619 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47860
I20260709 13:15:49.610412  5619 raft_consensus.cc:493] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.610929  5619 raft_consensus.cc:3060] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.614184  5799 consensus_queue.cc:1048] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:49.617816  5745 consensus_queue.cc:1048] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.628762  5745 consensus_queue.cc:1048] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.628539  5619 raft_consensus.cc:515] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.632521  5619 leader_election.cc:290] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.632683  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.633323  5543 raft_consensus.cc:3055] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.633661  5543 raft_consensus.cc:740] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.634483  5543 consensus_queue.cc:260] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.635885  5543 raft_consensus.cc:3060] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.636570  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "223d4ab346c24f6c8cce3ba665ceac07" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.643992  5543 raft_consensus.cc:2468] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:49.645355  5585 leader_election.cc:304] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: c968d75da1aa478db16e3de8072583b6
I20260709 13:15:49.646358  5746 raft_consensus.cc:2804] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.646937  5746 raft_consensus.cc:697] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.647980  5746 consensus_queue.cc:237] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.655795  5442 catalog_manager.cc:5697] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 1 to 2, leader changed from 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.662652  5619 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:49.671515  5765 consensus_queue.cc:1048] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.673580  5745 consensus_queue.cc:1048] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.674780  5801 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.700011  5745 raft_consensus.cc:993] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.700467  5745 raft_consensus.cc:1081] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.702220  5619 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47860
I20260709 13:15:49.702852  5619 raft_consensus.cc:493] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.703320  5619 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.711037  5762 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.716943  5619 raft_consensus.cc:515] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.719503  5619 leader_election.cc:290] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.720165  5545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.721434  5545 raft_consensus.cc:3055] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.722278  5545 raft_consensus.cc:740] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.723116  5545 consensus_queue.cc:260] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.723680  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.724351  5693 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.724614  5545 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.732919  5545 raft_consensus.cc:2468] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:49.733476  5801 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.731030  5693 raft_consensus.cc:2468] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:49.738511  5583 leader_election.cc:304] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.739323  5801 raft_consensus.cc:2804] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.739779  5801 raft_consensus.cc:697] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.740679  5801 consensus_queue.cc:237] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.744277  5765 raft_consensus.cc:993] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.744788  5765 raft_consensus.cc:1081] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.745149  5746 consensus_queue.cc:1048] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.751412  5746 raft_consensus.cc:993] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: : Instructing follower 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:49.751865  5801 raft_consensus.cc:1081] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Signalling peer 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:49.755726  5620 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47856
I20260709 13:15:49.756313  5620 raft_consensus.cc:493] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.756677  5620 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.757323  5542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:34730
I20260709 13:15:49.757913  5542 raft_consensus.cc:493] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.758378  5542 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.763267  5620 raft_consensus.cc:515] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.766876  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.767498  5694 raft_consensus.cc:3055] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.767809  5694 raft_consensus.cc:740] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:49.768651  5694 consensus_queue.cc:260] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.769770  5694 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.776885  5694 raft_consensus.cc:2468] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:49.778028  5583 leader_election.cc:304] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.778847  5620 leader_election.cc:290] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.778899  5784 raft_consensus.cc:2804] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.779587  5784 raft_consensus.cc:697] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.780831  5784 consensus_queue.cc:237] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.767969  5541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.786527  5541 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.790855  5765 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:49.796082  5542 raft_consensus.cc:515] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.797186  5543 raft_consensus.cc:1240] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Rejecting Update request from peer 59aa9b814c304015a0b4696951184f80 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:49.797900  5774 consensus_queue.cc:1048] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.800738  5801 consensus_queue.cc:1048] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.801841  5746 consensus_queue.cc:1059] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.802873  5746 raft_consensus.cc:3055] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.803225  5746 raft_consensus.cc:740] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.803872  5542 leader_election.cc:290] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 2 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.804004  5746 consensus_queue.cc:260] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.805246  5746 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.808966  5617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.810830  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.811368  5693 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.822358  5774 consensus_queue.cc:1048] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:49.823597  5693 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 2.
I20260709 13:15:49.824681  5508 leader_election.cc:304] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.830201  5541 raft_consensus.cc:2468] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:49.830440  5774 raft_consensus.cc:2804] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.831121  5774 raft_consensus.cc:697] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:49.831941  5774 consensus_queue.cc:237] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.846690  5801 raft_consensus.cc:993] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80: : Instructing follower c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:49.856127  5821 raft_consensus.cc:1081] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Signalling peer c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:49.858839  5692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54114
I20260709 13:15:49.859517  5692 raft_consensus.cc:493] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.860016  5692 raft_consensus.cc:3060] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.869405  5440 catalog_manager.cc:5697] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 1 to 2, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.874291  5692 raft_consensus.cc:515] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.877547  5692 leader_election.cc:290] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 2 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:49.880141  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.880736  5543 raft_consensus.cc:3060] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.885128  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09e603edad6c4f86819273f5646dd75c" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:49.885761  5620 raft_consensus.cc:3055] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.886087  5620 raft_consensus.cc:740] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.887027  5620 consensus_queue.cc:260] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.888228  5543 raft_consensus.cc:2468] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 2.
I20260709 13:15:49.872632  5617 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 2.
I20260709 13:15:49.889370  5659 leader_election.cc:304] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:49.890720  5765 raft_consensus.cc:2804] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.891467  5765 raft_consensus.cc:697] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 2 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:49.892465  5765 consensus_queue.cc:237] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.900283  5620 raft_consensus.cc:3060] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:49.897312  5585 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:38669, user_credentials={real_user=slave}} blocked reactor thread for 61562.5us
I20260709 13:15:49.912505  5741 raft_consensus.cc:993] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.912961  5741 raft_consensus.cc:1081] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:49.915580  5441 catalog_manager.cc:5697] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 1 to 2, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.917613  5620 raft_consensus.cc:2468] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 2.
I20260709 13:15:49.919952  5620 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47856
I20260709 13:15:49.920471  5620 raft_consensus.cc:493] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.920852  5620 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.927903  5620 raft_consensus.cc:515] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.927685  5440 catalog_manager.cc:5697] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 1 to 2, leader changed from c968d75da1aa478db16e3de8072583b6 (127.5.72.131) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.930547  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:49.929584  5440 catalog_manager.cc:5697] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 1 to 2, leader changed from 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.931190  5543 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.936456  5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:49.937160  5691 raft_consensus.cc:3055] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.937475  5691 raft_consensus.cc:740] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:49.938266  5543 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:49.938305  5691 consensus_queue.cc:260] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.939555  5691 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.939532  5585 leader_election.cc:304] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: 
I20260709 13:15:49.940702  5820 raft_consensus.cc:2804] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.941220  5820 raft_consensus.cc:697] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:49.942094  5820 consensus_queue.cc:237] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:49.946566  5691 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:49.946691  5620 leader_election.cc:290] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:49.956822  5439 catalog_manager.cc:5697] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 1 to 2, leader changed from c968d75da1aa478db16e3de8072583b6 (127.5.72.131) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:49.988417  5691 raft_consensus.cc:1275] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:49.989770  5774 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.000942  5620 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:50.001163  5694 raft_consensus.cc:1275] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.002926  5774 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.011734  5619 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:50.015249  5620 raft_consensus.cc:1275] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.016862  5823 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:50.018635  5619 raft_consensus.cc:1275] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.020005  5823 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.059137  5691 raft_consensus.cc:3060] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:50.066270  5691 raft_consensus.cc:1275] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.067729  5820 consensus_queue.cc:1048] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.076036  5543 raft_consensus.cc:1275] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.077383  5746 consensus_queue.cc:1048] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.169662  5543 raft_consensus.cc:1275] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.171244  5746 consensus_queue.cc:1048] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.182583  5691 raft_consensus.cc:1275] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.185024  5820 consensus_queue.cc:1048] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.252739  5691 raft_consensus.cc:1275] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.255712  5824 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:50.270757  5620 raft_consensus.cc:1275] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.274024  5823 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.279932  5543 raft_consensus.cc:1275] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.281441  5820 consensus_queue.cc:1048] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.292936  5691 raft_consensus.cc:1275] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.294358  5784 consensus_queue.cc:1048] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.392446  5691 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.393987  5784 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.402196  5543 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.405277  5784 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.446091  5620 raft_consensus.cc:1275] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.447402  5765 consensus_queue.cc:1048] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.460285  5543 raft_consensus.cc:1275] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.462162  5765 consensus_queue.cc:1048] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:52.570721  5410 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:15:52.571521  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:52.586545  5620 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54618
I20260709 13:15:52.587157  5620 raft_consensus.cc:606] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Received request to transfer leadership to TS c968d75da1aa478db16e3de8072583b6
I20260709 13:15:52.597240  5543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84232878e64440a08a417561c944da47"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
mode: GRACEFUL
new_leader_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:42598
I20260709 13:15:52.597749  5543 raft_consensus.cc:606] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Received request to transfer leadership to TS c968d75da1aa478db16e3de8072583b6
I20260709 13:15:52.598944  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:52.599375  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:52.616957  5821 raft_consensus.cc:993] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: : Instructing follower c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:52.617420  5832 raft_consensus.cc:1081] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Signalling peer c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:52.618858  5691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54114
I20260709 13:15:52.619359  5691 raft_consensus.cc:493] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:52.619658  5691 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:52.625303  5691 raft_consensus.cc:515] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:52.627079  5691 leader_election.cc:290] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 3 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:52.627739  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:52.628132  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:52.628293  5543 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:52.628675  5620 raft_consensus.cc:3055] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:52.628939  5620 raft_consensus.cc:740] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:52.629655  5620 consensus_queue.cc:260] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:52.630539  5620 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:52.633364  5543 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 3.
I20260709 13:15:52.634589  5659 leader_election.cc:304] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:52.635382  5620 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 3.
I20260709 13:15:52.635391  5827 raft_consensus.cc:2804] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:52.636857  5827 raft_consensus.cc:697] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:52.637586  5827 consensus_queue.cc:237] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:52.644651  5439 catalog_manager.cc:5697] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 2 to 3, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 3 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:53.098527  5839 raft_consensus.cc:993] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: : Instructing follower c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:53.098977  5828 raft_consensus.cc:1081] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Signalling peer c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:53.100260  5691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84232878e64440a08a417561c944da47"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54104
I20260709 13:15:53.100735  5691 raft_consensus.cc:493] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.101046  5691 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.105561  5691 raft_consensus.cc:515] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.107201  5691 leader_election.cc:290] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 3 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:53.107990  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:53.108207  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:53.108569  5543 raft_consensus.cc:3055] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:53.108721  5620 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.108922  5543 raft_consensus.cc:740] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:53.109732  5543 consensus_queue.cc:260] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.110922  5543 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.115223  5620 raft_consensus.cc:2468] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 3.
I20260709 13:15:53.116291  5657 leader_election.cc:304] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:53.116955  5827 raft_consensus.cc:2804] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:53.117085  5543 raft_consensus.cc:2468] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 3.
I20260709 13:15:53.117455  5827 raft_consensus.cc:697] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:53.118340  5827 consensus_queue.cc:237] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.127470  5439 catalog_manager.cc:5697] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 2 to 3, leader changed from 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129) to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 3 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:53.208396  5620 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:53.209668  5827 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:53.218518  5543 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:53.221175  5827 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:53.533154  5543 raft_consensus.cc:1275] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:53.534421  5827 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:53.543661  5620 raft_consensus.cc:1275] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:53.545182  5848 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:53.600098  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:53.610420  5543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:34738
I20260709 13:15:53.610973  5543 raft_consensus.cc:606] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:53.615587  5543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:34738
I20260709 13:15:53.616171  5543 raft_consensus.cc:606] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:53.620711  5691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84232878e64440a08a417561c944da47"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:54130
I20260709 13:15:53.621241  5691 raft_consensus.cc:606] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:53.625907  5543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:34738
I20260709 13:15:53.626443  5543 raft_consensus.cc:606] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:53.634270  5691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
mode: GRACEFUL
new_leader_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:54130
I20260709 13:15:53.634892  5691 raft_consensus.cc:606] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Received request to transfer leadership to TS 59aa9b814c304015a0b4696951184f80
I20260709 13:15:53.717345  5827 raft_consensus.cc:993] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.717770  5848 raft_consensus.cc:1081] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.719283  5620 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47856
I20260709 13:15:53.719784  5620 raft_consensus.cc:493] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.720120  5620 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.724699  5620 raft_consensus.cc:515] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.726411  5620 leader_election.cc:290] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 4 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:53.727252  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:53.727555  5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:53.727857  5543 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.728197  5691 raft_consensus.cc:3055] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:53.728528  5691 raft_consensus.cc:740] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:53.729328  5691 consensus_queue.cc:260] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.730515  5691 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.734776  5543 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 4.
I20260709 13:15:53.736063  5585 leader_election.cc:304] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: 
I20260709 13:15:53.736841  5846 raft_consensus.cc:2804] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:53.736934  5691 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 4.
I20260709 13:15:53.737401  5846 raft_consensus.cc:697] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:53.738235  5846 consensus_queue.cc:237] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.744419  5439 catalog_manager.cc:5697] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 3 to 4, leader changed from c968d75da1aa478db16e3de8072583b6 (127.5.72.131) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:53.840348  5828 raft_consensus.cc:993] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.840785  5853 raft_consensus.cc:1081] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.842207  5620 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47860
I20260709 13:15:53.842587  5620 raft_consensus.cc:493] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.842825  5620 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.846462  5620 raft_consensus.cc:515] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.847849  5620 leader_election.cc:290] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 3 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:53.848726  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:53.848934  5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:53.849248  5543 raft_consensus.cc:3055] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:53.849470  5691 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.849617  5543 raft_consensus.cc:740] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:53.850385  5543 consensus_queue.cc:260] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.851390  5543 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.855309  5691 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 3.
I20260709 13:15:53.856353  5543 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 3.
I20260709 13:15:53.856359  5583 leader_election.cc:304] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:53.857223  5846 raft_consensus.cc:2804] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:53.857681  5846 raft_consensus.cc:697] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:53.858388  5846 consensus_queue.cc:237] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.864434  5439 catalog_manager.cc:5697] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 2 to 3, leader changed from 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:53.868880  5828 raft_consensus.cc:993] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.869360  5853 raft_consensus.cc:1081] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.870980  5620 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47860
I20260709 13:15:53.871414  5620 raft_consensus.cc:493] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.871724  5620 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.875603  5620 raft_consensus.cc:515] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.876994  5620 leader_election.cc:290] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 3 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:53.877786  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:53.878015  5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:53.878305  5543 raft_consensus.cc:3055] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:53.878540  5691 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.878685  5543 raft_consensus.cc:740] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:53.879316  5543 consensus_queue.cc:260] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.880119  5543 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.883270  5691 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 3.
I20260709 13:15:53.883824  5543 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 3.
I20260709 13:15:53.884353  5583 leader_election.cc:304] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:53.885015  5846 raft_consensus.cc:2804] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:53.885340  5846 raft_consensus.cc:697] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:53.886034  5846 consensus_queue.cc:237] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.892637  5439 catalog_manager.cc:5697] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 2 to 3, leader changed from 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:53.971091  5833 raft_consensus.cc:993] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.971154  5828 raft_consensus.cc:993] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03: : Instructing follower 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.971747  5828 raft_consensus.cc:1081] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.971712  5848 raft_consensus.cc:1081] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Signalling peer 59aa9b814c304015a0b4696951184f80 to start an election
I20260709 13:15:53.973376  5620 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47860
I20260709 13:15:53.973435  5618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84232878e64440a08a417561c944da47"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
 from {username='slave'} at 127.0.0.1:47856
I20260709 13:15:53.973874  5620 raft_consensus.cc:493] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.973996  5618 raft_consensus.cc:493] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.974335  5620 raft_consensus.cc:3060] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.974413  5618 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.980752  5618 raft_consensus.cc:515] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.981050  5620 raft_consensus.cc:515] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.982509  5620 leader_election.cc:290] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:53.982641  5618 leader_election.cc:290] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 4 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:53.984056  5541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:53.984200  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:53.984119  5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf9561610c3945f6b15485ba6a8116d0" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:53.984715  5541 raft_consensus.cc:3055] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:53.984807  5694 raft_consensus.cc:3055] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:53.984828  5543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84232878e64440a08a417561c944da47" candidate_uuid: "59aa9b814c304015a0b4696951184f80" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:53.985095  5691 raft_consensus.cc:3060] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.985116  5541 raft_consensus.cc:740] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:53.985255  5694 raft_consensus.cc:740] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:53.985544  5543 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.986173  5694 consensus_queue.cc:260] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.987150  5694 raft_consensus.cc:3060] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.986173  5541 consensus_queue.cc:260] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.991271  5691 raft_consensus.cc:2468] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:53.991271  5543 raft_consensus.cc:2468] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 4.
I20260709 13:15:53.992023  5541 raft_consensus.cc:3060] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.992623  5583 leader_election.cc:304] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:53.993476  5846 raft_consensus.cc:2804] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:53.993505  5585 leader_election.cc:304] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, 59aa9b814c304015a0b4696951184f80; no voters: 
I20260709 13:15:53.993598  5694 raft_consensus.cc:2468] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 4.
I20260709 13:15:53.994503  5832 raft_consensus.cc:2804] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:53.994262  5846 raft_consensus.cc:697] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:53.995689  5846 consensus_queue.cc:237] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.996495  5832 raft_consensus.cc:697] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 4 LEADER]: Becoming Leader. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:53.997478  5832 consensus_queue.cc:237] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:53.998760  5541 raft_consensus.cc:2468] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59aa9b814c304015a0b4696951184f80 in term 2.
I20260709 13:15:54.004129  5439 catalog_manager.cc:5697] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 1 to 2, leader changed from 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:54.014806  5439 catalog_manager.cc:5697] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 reported cstate change: term changed from 3 to 4, leader changed from c968d75da1aa478db16e3de8072583b6 (127.5.72.131) to 59aa9b814c304015a0b4696951184f80 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "59aa9b814c304015a0b4696951184f80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:54.174561  5694 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:54.175791  5832 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:54.183355  5541 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:54.185540  5846 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:54.263679  5541 raft_consensus.cc:1275] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:54.265048  5846 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:54.273288  5694 raft_consensus.cc:1275] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:54.274850  5856 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:54.392136  5694 raft_consensus.cc:1275] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:54.393323  5856 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:54.403118  5541 raft_consensus.cc:1275] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:54.404465  5846 consensus_queue.cc:1048] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:54.414994  5694 raft_consensus.cc:1275] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:54.416333  5846 consensus_queue.cc:1048] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:54.426064  5541 raft_consensus.cc:1275] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:54.427033  5691 raft_consensus.cc:1275] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:54.428334  5846 consensus_queue.cc:1048] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:54.429361  5856 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:54.451251  5541 raft_consensus.cc:1275] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Refusing update from remote peer 59aa9b814c304015a0b4696951184f80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:54.452653  5846 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:56.644942  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:56.651877  5618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:54618
I20260709 13:15:56.652635  5618 raft_consensus.cc:606] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 LEADER]: Received request to transfer leadership to TS 313fbe57577947fe985adef5aeb1ec03
I20260709 13:15:56.655208  5618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54618
I20260709 13:15:56.655616  5618 raft_consensus.cc:606] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Received request to transfer leadership to TS c968d75da1aa478db16e3de8072583b6
I20260709 13:15:56.657869  5618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54618
I20260709 13:15:56.658259  5618 raft_consensus.cc:606] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Received request to transfer leadership to TS c968d75da1aa478db16e3de8072583b6
I20260709 13:15:56.660387  5618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:54618
I20260709 13:15:56.660769  5618 raft_consensus.cc:606] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Received request to transfer leadership to TS 313fbe57577947fe985adef5aeb1ec03
I20260709 13:15:56.662894  5618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d"
dest_uuid: "59aa9b814c304015a0b4696951184f80"
mode: GRACEFUL
new_leader_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:54618
I20260709 13:15:56.663266  5618 raft_consensus.cc:606] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Received request to transfer leadership to TS 313fbe57577947fe985adef5aeb1ec03
I20260709 13:15:56.664217  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:56.664655  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:56.972949  5869 raft_consensus.cc:993] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80: : Instructing follower c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:56.973548  5856 raft_consensus.cc:1081] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Signalling peer c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:56.975440  5691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54114
I20260709 13:15:56.976089  5691 raft_consensus.cc:493] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:56.976562  5691 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:56.983656  5691 raft_consensus.cc:515] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:56.985356  5691 leader_election.cc:290] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 4 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:56.986373  5618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:56.987272  5618 raft_consensus.cc:3055] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:56.987646  5618 raft_consensus.cc:740] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:56.986411  5541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7e0a08ded54091a1c12b9fa253b38a" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:56.988312  5541 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:56.990373  5618 consensus_queue.cc:260] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:56.991817  5869 raft_consensus.cc:993] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: : Instructing follower 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:56.991914  5618 raft_consensus.cc:3060] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:56.993332  5856 raft_consensus.cc:1081] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 LEADER]: Signalling peer 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:56.998023  5618 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 4.
I20260709 13:15:56.999539  5657 leader_election.cc:304] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59aa9b814c304015a0b4696951184f80, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:57.000590  5848 raft_consensus.cc:2804] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:57.001394  5541 raft_consensus.cc:2468] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 4.
I20260709 13:15:57.001670  5543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:34730
I20260709 13:15:57.002277  5543 raft_consensus.cc:493] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:57.002604  5543 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:57.004801  5870 raft_consensus.cc:993] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80: : Instructing follower 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:57.005973  5870 raft_consensus.cc:1081] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Signalling peer 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:57.009768  5542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:34730
I20260709 13:15:57.010480  5542 raft_consensus.cc:493] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:57.010957  5542 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:57.011238  5848 raft_consensus.cc:697] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 4 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:57.011965  5848 consensus_queue.cc:237] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.017028  5543 raft_consensus.cc:515] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.020030  5869 raft_consensus.cc:993] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: : Instructing follower c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:57.020763  5543 leader_election.cc:290] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 5 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:57.021078  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:57.021740  5694 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:57.020536  5869 raft_consensus.cc:1081] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Signalling peer c968d75da1aa478db16e3de8072583b6 to start an election
I20260709 13:15:57.026463  5691 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769"
dest_uuid: "c968d75da1aa478db16e3de8072583b6"
 from {username='slave'} at 127.0.0.1:54114
I20260709 13:15:57.027011  5691 raft_consensus.cc:493] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:57.027338  5691 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:57.028919  5694 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 5.
I20260709 13:15:57.029150  5440 catalog_manager.cc:5697] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 3 to 4, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 4 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:57.031275  5508 leader_election.cc:304] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:57.021392  5542 raft_consensus.cc:515] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.034107  5542 leader_election.cc:290] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 3 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:57.037315  5874 raft_consensus.cc:2804] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:57.037424  5618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2793b38d6e2c4175bbfce95233f44810" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:57.038048  5618 raft_consensus.cc:3055] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:57.038445  5618 raft_consensus.cc:740] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:57.039254  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:57.039805  5694 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:57.039678  5618 consensus_queue.cc:260] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.041492  5618 raft_consensus.cc:3060] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:57.039291  5691 raft_consensus.cc:515] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.045300  5542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
I20260709 13:15:57.045485  5874 raft_consensus.cc:697] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 5 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:57.045904  5542 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:57.046816  5694 raft_consensus.cc:2468] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 3.
I20260709 13:15:57.046437  5874 consensus_queue.cc:237] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.047904  5508 leader_election.cc:304] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:57.050232  5618 raft_consensus.cc:2468] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 5.
I20260709 13:15:57.050388  5876 raft_consensus.cc:2804] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:57.050432  5620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4386c4eaca784bd6bb100d33b51ca769" candidate_uuid: "c968d75da1aa478db16e3de8072583b6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:57.051257  5620 raft_consensus.cc:3055] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:57.051553  5620 raft_consensus.cc:740] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:57.052268  5620 consensus_queue.cc:260] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.053123  5542 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 4.
I20260709 13:15:57.052970  5618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a3c3a7823c4eebbebb1762fae68bd2" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:57.053316  5620 raft_consensus.cc:3060] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:57.053776  5618 raft_consensus.cc:3055] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:57.054263  5618 raft_consensus.cc:740] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:57.054533  5659 leader_election.cc:304] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:57.056106  5871 raft_consensus.cc:2804] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:57.056650  5871 raft_consensus.cc:697] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 4 LEADER]: Becoming Leader. State: Replica: c968d75da1aa478db16e3de8072583b6, State: Running, Role: LEADER
I20260709 13:15:57.057066  5876 raft_consensus.cc:697] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 3 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:57.057798  5691 leader_election.cc:290] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [CANDIDATE]: Term 4 election: Requested vote from peers 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669), 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145)
I20260709 13:15:57.057657  5871 consensus_queue.cc:237] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.058032  5876 consensus_queue.cc:237] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.060748  5620 raft_consensus.cc:2468] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c968d75da1aa478db16e3de8072583b6 in term 4.
I20260709 13:15:57.065532  5618 consensus_queue.cc:260] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.067924  5618 raft_consensus.cc:3060] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:57.074064  5439 catalog_manager.cc:5697] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 4 to 5, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 5 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.074890  5618 raft_consensus.cc:2468] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 3.
I20260709 13:15:57.083416  5440 catalog_manager.cc:5697] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 reported cstate change: term changed from 3 to 4, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to c968d75da1aa478db16e3de8072583b6 (127.5.72.131). New cstate: current_term: 4 leader_uuid: "c968d75da1aa478db16e3de8072583b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:57.086128  5439 catalog_manager.cc:5697] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 2 to 3, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.095690  5870 raft_consensus.cc:993] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80: : Instructing follower 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:57.096241  5872 raft_consensus.cc:1081] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Signalling peer 313fbe57577947fe985adef5aeb1ec03 to start an election
I20260709 13:15:57.097640  5542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d"
dest_uuid: "313fbe57577947fe985adef5aeb1ec03"
 from {username='slave'} at 127.0.0.1:34730
I20260709 13:15:57.098100  5542 raft_consensus.cc:493] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:57.098403  5542 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:57.102959  5542 raft_consensus.cc:515] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.104770  5542 leader_election.cc:290] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 3 election: Requested vote from peers 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145), c968d75da1aa478db16e3de8072583b6 (127.5.72.131:44261)
I20260709 13:15:57.105795  5691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c968d75da1aa478db16e3de8072583b6"
I20260709 13:15:57.105818  5618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5534c1666ec4b819cd64abef9c4126d" candidate_uuid: "313fbe57577947fe985adef5aeb1ec03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "59aa9b814c304015a0b4696951184f80"
I20260709 13:15:57.106319  5691 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:57.106446  5618 raft_consensus.cc:3055] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:57.106875  5618 raft_consensus.cc:740] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 59aa9b814c304015a0b4696951184f80, State: Running, Role: LEADER
I20260709 13:15:57.107563  5618 consensus_queue.cc:260] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.108455  5618 raft_consensus.cc:3060] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:57.110460  5691 raft_consensus.cc:2468] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 3.
I20260709 13:15:57.111477  5508 leader_election.cc:304] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313fbe57577947fe985adef5aeb1ec03, c968d75da1aa478db16e3de8072583b6; no voters: 
I20260709 13:15:57.112149  5876 raft_consensus.cc:2804] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:57.112617  5876 raft_consensus.cc:697] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 3 LEADER]: Becoming Leader. State: Replica: 313fbe57577947fe985adef5aeb1ec03, State: Running, Role: LEADER
I20260709 13:15:57.113538  5618 raft_consensus.cc:2468] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313fbe57577947fe985adef5aeb1ec03 in term 3.
I20260709 13:15:57.113556  5876 consensus_queue.cc:237] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } }
I20260709 13:15:57.120051  5439 catalog_manager.cc:5697] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 reported cstate change: term changed from 2 to 3, leader changed from 59aa9b814c304015a0b4696951184f80 (127.5.72.130) to 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "313fbe57577947fe985adef5aeb1ec03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.443697  5618 raft_consensus.cc:1275] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:57.444949  5876 consensus_queue.cc:1048] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:57.455266  5691 raft_consensus.cc:1275] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:57.456593  5874 consensus_queue.cc:1048] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:57.495723  5542 raft_consensus.cc:1275] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:57.497236  5871 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:57.506556  5618 raft_consensus.cc:1275] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:57.507970  5848 consensus_queue.cc:1048] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:57.544556  5618 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 5 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:57.544942  5542 raft_consensus.cc:1275] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:57.545867  5876 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:57.546304  5871 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "313fbe57577947fe985adef5aeb1ec03" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 38669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:57.556821  5620 raft_consensus.cc:1275] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Refusing update from remote peer c968d75da1aa478db16e3de8072583b6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:57.556821  5691 raft_consensus.cc:1275] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 5 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:57.558473  5848 consensus_queue.cc:1048] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:57.560105  5878 consensus_queue.cc:1048] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:57.622985  5620 raft_consensus.cc:1275] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:57.624217  5878 consensus_queue.cc:1048] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa9b814c304015a0b4696951184f80" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:57.632767  5691 raft_consensus.cc:1275] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Refusing update from remote peer 313fbe57577947fe985adef5aeb1ec03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:57.634865  5876 consensus_queue.cc:1048] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c968d75da1aa478db16e3de8072583b6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:57.665601  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:57.671036  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
I20260709 13:15:57.708581  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:57.710237  5410 tablet_replica.cc:333] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.711519  5410 raft_consensus.cc:2243] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.712085  5410 raft_consensus.cc:2272] T 09e603edad6c4f86819273f5646dd75c P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.714437  5410 tablet_replica.cc:333] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.715004  5410 raft_consensus.cc:2243] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.715426  5410 raft_consensus.cc:2272] T 4386c4eaca784bd6bb100d33b51ca769 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.717433  5410 tablet_replica.cc:333] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.717973  5410 raft_consensus.cc:2243] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.718397  5410 raft_consensus.cc:2272] T cf9561610c3945f6b15485ba6a8116d0 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.720082  5410 tablet_replica.cc:333] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.720552  5410 raft_consensus.cc:2243] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.721251  5410 raft_consensus.cc:2272] T 97a3c3a7823c4eebbebb1762fae68bd2 P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.723014  5410 tablet_replica.cc:333] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.723480  5410 raft_consensus.cc:2243] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.724167  5410 raft_consensus.cc:2272] T e5534c1666ec4b819cd64abef9c4126d P 313fbe57577947fe985adef5aeb1ec03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.725889  5410 tablet_replica.cc:333] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.726349  5410 raft_consensus.cc:2243] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.726805  5410 raft_consensus.cc:2272] T 223d4ab346c24f6c8cce3ba665ceac07 P 313fbe57577947fe985adef5aeb1ec03 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.728261  5410 tablet_replica.cc:333] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.728714  5410 raft_consensus.cc:2243] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.729101  5410 raft_consensus.cc:2272] T 4d7e0a08ded54091a1c12b9fa253b38a P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.731160  5410 tablet_replica.cc:333] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.731606  5410 raft_consensus.cc:2243] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.732270  5410 raft_consensus.cc:2272] T 2793b38d6e2c4175bbfce95233f44810 P 313fbe57577947fe985adef5aeb1ec03 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.734237  5410 tablet_replica.cc:333] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03: stopping tablet replica
I20260709 13:15:57.734723  5410 raft_consensus.cc:2243] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.735105  5410 raft_consensus.cc:2272] T 84232878e64440a08a417561c944da47 P 313fbe57577947fe985adef5aeb1ec03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.829416  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
I20260709 13:15:57.847374  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
W20260709 13:15:57.867296  5657 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:57.873210  5657 consensus_peers.cc:597] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 -> Peer 59aa9b814c304015a0b4696951184f80 (127.5.72.130:35145): Couldn't send request to peer 59aa9b814c304015a0b4696951184f80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.874907  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:15:57.875520  5585 consensus_peers.cc:597] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 -> Peer 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669): Couldn't send request to peer 313fbe57577947fe985adef5aeb1ec03. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:38669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.875649  5410 tablet_replica.cc:333] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.876614  5410 raft_consensus.cc:2243] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.877148  5410 raft_consensus.cc:2272] T 09e603edad6c4f86819273f5646dd75c P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.878932  5410 tablet_replica.cc:333] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.879413  5410 raft_consensus.cc:2243] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.879760  5410 raft_consensus.cc:2272] T e5534c1666ec4b819cd64abef9c4126d P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.881708  5410 tablet_replica.cc:333] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.882174  5410 raft_consensus.cc:2243] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.882529  5410 raft_consensus.cc:2272] T 97a3c3a7823c4eebbebb1762fae68bd2 P 59aa9b814c304015a0b4696951184f80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.885181  5410 tablet_replica.cc:333] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.885628  5410 raft_consensus.cc:2243] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.886300  5410 raft_consensus.cc:2272] T cf9561610c3945f6b15485ba6a8116d0 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.887928  5410 tablet_replica.cc:333] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.888356  5410 raft_consensus.cc:2243] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.888989  5410 raft_consensus.cc:2272] T 223d4ab346c24f6c8cce3ba665ceac07 P 59aa9b814c304015a0b4696951184f80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.890504  5410 tablet_replica.cc:333] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.890982  5410 raft_consensus.cc:2243] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.891321  5410 raft_consensus.cc:2272] T 4d7e0a08ded54091a1c12b9fa253b38a P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.893219  5410 tablet_replica.cc:333] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.893640  5410 raft_consensus.cc:2243] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.894286  5410 raft_consensus.cc:2272] T 84232878e64440a08a417561c944da47 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.896018  5410 tablet_replica.cc:333] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.896462  5410 raft_consensus.cc:2243] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.896839  5410 raft_consensus.cc:2272] T 2793b38d6e2c4175bbfce95233f44810 P 59aa9b814c304015a0b4696951184f80 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.898756  5410 tablet_replica.cc:333] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80: stopping tablet replica
I20260709 13:15:57.899163  5410 raft_consensus.cc:2243] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.899488  5410 raft_consensus.cc:2272] T 4386c4eaca784bd6bb100d33b51ca769 P 59aa9b814c304015a0b4696951184f80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.926936  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
I20260709 13:15:57.944711  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
W20260709 13:15:57.948938  5659 consensus_peers.cc:597] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 -> Peer 313fbe57577947fe985adef5aeb1ec03 (127.5.72.129:38669): Couldn't send request to peer 313fbe57577947fe985adef5aeb1ec03. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:38669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.971079  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:57.971741  5410 tablet_replica.cc:333] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.972319  5410 raft_consensus.cc:2243] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.973074  5410 raft_consensus.cc:2272] T 09e603edad6c4f86819273f5646dd75c P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.974787  5410 tablet_replica.cc:333] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.975277  5410 raft_consensus.cc:2243] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.975695  5410 raft_consensus.cc:2272] T e5534c1666ec4b819cd64abef9c4126d P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.977896  5410 tablet_replica.cc:333] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.978372  5410 raft_consensus.cc:2243] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.979101  5410 raft_consensus.cc:2272] T 4386c4eaca784bd6bb100d33b51ca769 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.980856  5410 tablet_replica.cc:333] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.981323  5410 raft_consensus.cc:2243] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.981738  5410 raft_consensus.cc:2272] T 2793b38d6e2c4175bbfce95233f44810 P c968d75da1aa478db16e3de8072583b6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.983506  5410 tablet_replica.cc:333] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.983985  5410 raft_consensus.cc:2243] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.984370  5410 raft_consensus.cc:2272] T 84232878e64440a08a417561c944da47 P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.985865  5410 tablet_replica.cc:333] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.986300  5410 raft_consensus.cc:2243] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.986802  5410 raft_consensus.cc:2272] T 223d4ab346c24f6c8cce3ba665ceac07 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.988894  5410 tablet_replica.cc:333] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.989566  5410 raft_consensus.cc:2243] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.990235  5410 raft_consensus.cc:2272] T 4d7e0a08ded54091a1c12b9fa253b38a P c968d75da1aa478db16e3de8072583b6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.992296  5410 tablet_replica.cc:333] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.992691  5410 raft_consensus.cc:2243] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.993008  5410 raft_consensus.cc:2272] T 97a3c3a7823c4eebbebb1762fae68bd2 P c968d75da1aa478db16e3de8072583b6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.994987  5410 tablet_replica.cc:333] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6: stopping tablet replica
I20260709 13:15:57.995424  5410 raft_consensus.cc:2243] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.995814  5410 raft_consensus.cc:2272] T cf9561610c3945f6b15485ba6a8116d0 P c968d75da1aa478db16e3de8072583b6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.015856  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
I20260709 13:15:58.033716  5410 master.cc:562] Master@127.5.72.190:36349 shutting down...
I20260709 13:15:58.255043  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.255659  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.256004  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 63dc443bd46a434a8a91c1d2c99c5acd: stopping tablet replica
I20260709 13:15:58.276268  5410 master.cc:584] Master@127.5.72.190:36349 shutdown complete.
313fbe57577947fe985adef5aeb1ec03  4
59aa9b814c304015a0b4696951184f80  4
c968d75da1aa478db16e3de8072583b6  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12444 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:15:58.317778  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:42289
I20260709 13:15:58.319422  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:58.331060  5904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.331561  5905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.332459  5907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:58.333995  5410 server_base.cc:1061] running on GCE node
I20260709 13:15:58.334942  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:58.335134  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:58.335340  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602958335308 us; error 0 us; skew 500 ppm
I20260709 13:15:58.335873  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:58.338351  5410 webserver.cc:533] Webserver started at http://127.5.72.190:36535/ using document root <none> and password file <none>
I20260709 13:15:58.338886  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:58.339084  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:58.339389  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:58.340492  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "55249a2acd5b4f5abc0fc7ecb58824b8"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.344846  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:15:58.348047  5912 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.348721  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:15:58.349001  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "55249a2acd5b4f5abc0fc7ecb58824b8"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.349270  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:58.364532  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:58.365733  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:58.402463  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:42289
I20260709 13:15:58.402550  5963 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:42289 every 8 connection(s)
I20260709 13:15:58.408895  5964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:58.426319  5964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8: Bootstrap starting.
I20260709 13:15:58.433876  5964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.441205  5964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8: No bootstrap required, opened a new log
I20260709 13:15:58.443286  5964 raft_consensus.cc:359] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" member_type: VOTER }
I20260709 13:15:58.443651  5964 raft_consensus.cc:385] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.443851  5964 raft_consensus.cc:740] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55249a2acd5b4f5abc0fc7ecb58824b8, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.444324  5964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" member_type: VOTER }
I20260709 13:15:58.444695  5964 raft_consensus.cc:399] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:58.444885  5964 raft_consensus.cc:493] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:58.445091  5964 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.449051  5964 raft_consensus.cc:515] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" member_type: VOTER }
I20260709 13:15:58.449530  5964 leader_election.cc:304] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55249a2acd5b4f5abc0fc7ecb58824b8; no voters: 
I20260709 13:15:58.450730  5964 leader_election.cc:290] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:58.451061  5967 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:58.452457  5967 raft_consensus.cc:697] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 1 LEADER]: Becoming Leader. State: Replica: 55249a2acd5b4f5abc0fc7ecb58824b8, State: Running, Role: LEADER
I20260709 13:15:58.453145  5967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" member_type: VOTER }
I20260709 13:15:58.453691  5964 sys_catalog.cc:565] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:58.457830  5968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" member_type: VOTER } }
I20260709 13:15:58.458066  5969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55249a2acd5b4f5abc0fc7ecb58824b8. Latest consensus state: current_term: 1 leader_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55249a2acd5b4f5abc0fc7ecb58824b8" member_type: VOTER } }
I20260709 13:15:58.458493  5968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:58.458576  5969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:58.463732  5974 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:58.469132  5974 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:58.470222  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:58.477427  5974 catalog_manager.cc:1383] Generated new cluster ID: fcb83acb13224a68bf59b210be83448e
I20260709 13:15:58.477701  5974 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:58.503691  5974 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:58.504938  5974 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:58.523238  5974 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8: Generated new TSK 0
I20260709 13:15:58.523849  5974 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:58.536566  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:58.542090  5985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.543462  5986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.544855  5988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:58.545678  5410 server_base.cc:1061] running on GCE node
I20260709 13:15:58.546555  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:58.546813  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:58.547001  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602958546967 us; error 0 us; skew 500 ppm
I20260709 13:15:58.547528  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:58.549697  5410 webserver.cc:533] Webserver started at http://127.5.72.129:37553/ using document root <none> and password file <none>
I20260709 13:15:58.550186  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:58.550359  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:58.550706  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:58.551748  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.556074  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:15:58.559152  5993 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.559875  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20260709 13:15:58.560154  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.560401  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:58.581686  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:58.583002  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:58.584429  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:58.586656  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:58.586879  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.587122  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:58.587311  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.626408  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:33797
I20260709 13:15:58.626498  6055 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:33797 every 8 connection(s)
I20260709 13:15:58.631347  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:58.638101  6060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.640019  6061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.641434  6063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:58.642304  5410 server_base.cc:1061] running on GCE node
I20260709 13:15:58.643186  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:58.643364  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:58.643486  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602958643474 us; error 0 us; skew 500 ppm
I20260709 13:15:58.643906  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:58.646600  5410 webserver.cc:533] Webserver started at http://127.5.72.130:34779/ using document root <none> and password file <none>
I20260709 13:15:58.647285  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:58.647508  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:58.647806  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:58.648048  6056 heartbeater.cc:344] Connected to a master server at 127.5.72.190:42289
I20260709 13:15:58.648453  6056 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:58.649329  6056 heartbeater.cc:507] Master 127.5.72.190:42289 requested a full tablet report, sending...
I20260709 13:15:58.649312  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "848c6bc37a7d4316aa29e395d99215e3"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.651701  5929 ts_manager.cc:194] Registered new tserver with Master: b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797)
I20260709 13:15:58.653448  5929 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57936
I20260709 13:15:58.654597  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:15:58.657892  6068 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.658565  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:15:58.658919  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "848c6bc37a7d4316aa29e395d99215e3"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.659219  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:58.676067  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:58.677299  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:58.678853  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:58.680929  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:58.681150  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.681394  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:58.681572  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.720253  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:36487
I20260709 13:15:58.720440  6130 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:36487 every 8 connection(s)
I20260709 13:15:58.829840  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:58.838999  6134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.839520  6135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:58.842924  6137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:58.843472  5410 server_base.cc:1061] running on GCE node
I20260709 13:15:58.845192  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:58.845439  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:58.845631  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602958845613 us; error 0 us; skew 500 ppm
I20260709 13:15:58.846267  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:58.865067  5410 webserver.cc:533] Webserver started at http://127.5.72.131:33201/ using document root <none> and password file <none>
I20260709 13:15:58.865810  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:58.866055  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:58.866376  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:58.867836  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.868465  6131 heartbeater.cc:344] Connected to a master server at 127.5.72.190:42289
I20260709 13:15:58.868829  6131 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:58.869508  6131 heartbeater.cc:507] Master 127.5.72.190:42289 requested a full tablet report, sending...
I20260709 13:15:58.871560  5929 ts_manager.cc:194] Registered new tserver with Master: 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:15:58.872781  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:15:58.873219  5929 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57948
I20260709 13:15:58.876156  6142 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.876906  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:15:58.877211  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
format_stamp: "Formatted at 2026-07-09 13:15:58 on dist-test-slave-n5kz"
I20260709 13:15:58.877496  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:58.889041  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:58.890659  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:58.892650  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:58.895687  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:58.895957  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.896231  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:58.896512  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:58.935202  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:42733
I20260709 13:15:58.935292  6204 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:42733 every 8 connection(s)
I20260709 13:15:58.947954  6205 heartbeater.cc:344] Connected to a master server at 127.5.72.190:42289
I20260709 13:15:58.948323  6205 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:58.949060  6205 heartbeater.cc:507] Master 127.5.72.190:42289 requested a full tablet report, sending...
I20260709 13:15:58.950867  5929 ts_manager.cc:194] Registered new tserver with Master: 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:58.951162  5410 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013094924s
I20260709 13:15:58.952170  5929 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57954
I20260709 13:15:58.975587  5929 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57970:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:58.978431  5929 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:59.051733  6096 tablet_service.cc:1511] Processing CreateTablet for tablet cae5f9d1e9824af4a395601857899eea (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:59.053141  6096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae5f9d1e9824af4a395601857899eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.059024  6170 tablet_service.cc:1511] Processing CreateTablet for tablet cae5f9d1e9824af4a395601857899eea (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:59.060416  6170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae5f9d1e9824af4a395601857899eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.060242  6095 tablet_service.cc:1511] Processing CreateTablet for tablet 0aefd4e81722495ea9fc155e3b924c72 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:59.061623  6095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aefd4e81722495ea9fc155e3b924c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.061391  6169 tablet_service.cc:1511] Processing CreateTablet for tablet 0aefd4e81722495ea9fc155e3b924c72 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:59.062736  6169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aefd4e81722495ea9fc155e3b924c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.060017  6168 tablet_service.cc:1511] Processing CreateTablet for tablet 531ec449e905416c95437a5081a86db4 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:59.067660  6168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531ec449e905416c95437a5081a86db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.072991  6094 tablet_service.cc:1511] Processing CreateTablet for tablet 531ec449e905416c95437a5081a86db4 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:59.074369  6094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531ec449e905416c95437a5081a86db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.075006  6021 tablet_service.cc:1511] Processing CreateTablet for tablet cae5f9d1e9824af4a395601857899eea (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:59.075709  6020 tablet_service.cc:1511] Processing CreateTablet for tablet 0aefd4e81722495ea9fc155e3b924c72 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:59.076294  6021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae5f9d1e9824af4a395601857899eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.076967  6020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aefd4e81722495ea9fc155e3b924c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.078341  6019 tablet_service.cc:1511] Processing CreateTablet for tablet 531ec449e905416c95437a5081a86db4 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:59.079653  6019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531ec449e905416c95437a5081a86db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.083740  6018 tablet_service.cc:1511] Processing CreateTablet for tablet 34a2bcee8dc14683a3de1e27aac3e496 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:59.085053  6018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a2bcee8dc14683a3de1e27aac3e496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.107856  6221 tablet_bootstrap.cc:492] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.111835  6222 tablet_bootstrap.cc:492] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.116106  6221 tablet_bootstrap.cc:654] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.121812  6095 tablet_service.cc:1511] Processing CreateTablet for tablet 34a2bcee8dc14683a3de1e27aac3e496 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:59.123603  6095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a2bcee8dc14683a3de1e27aac3e496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.138190  6020 tablet_service.cc:1511] Processing CreateTablet for tablet 0583fd471a2f48afa3d1a211dd84ff1d (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:59.139437  6222 tablet_bootstrap.cc:654] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.139580  6020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0583fd471a2f48afa3d1a211dd84ff1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.141846  6220 tablet_bootstrap.cc:492] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.144958  6221 tablet_bootstrap.cc:492] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.145532  6221 ts_tablet_manager.cc:1403] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.000s
I20260709 13:15:59.140297  6169 tablet_service.cc:1511] Processing CreateTablet for tablet 34a2bcee8dc14683a3de1e27aac3e496 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:59.148875  6169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a2bcee8dc14683a3de1e27aac3e496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.153391  6095 tablet_service.cc:1511] Processing CreateTablet for tablet 0583fd471a2f48afa3d1a211dd84ff1d (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:59.154811  6095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0583fd471a2f48afa3d1a211dd84ff1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.155848  6168 tablet_service.cc:1511] Processing CreateTablet for tablet 0583fd471a2f48afa3d1a211dd84ff1d (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:59.158044  6168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0583fd471a2f48afa3d1a211dd84ff1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.163241  6220 tablet_bootstrap.cc:654] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.164248  6222 tablet_bootstrap.cc:492] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.164781  6222 ts_tablet_manager.cc:1403] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.007s
I20260709 13:15:59.165161  6020 tablet_service.cc:1511] Processing CreateTablet for tablet 85d394d62b444ec9b807d39ea09362cb (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:59.168409  6222 raft_consensus.cc:359] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.169977  6094 tablet_service.cc:1511] Processing CreateTablet for tablet 85d394d62b444ec9b807d39ea09362cb (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:59.183175  6169 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcf0341ab4e4003ba68968236997a97 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:59.166462  6020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d394d62b444ec9b807d39ea09362cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.184487  6169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcf0341ab4e4003ba68968236997a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.185452  6021 tablet_service.cc:1511] Processing CreateTablet for tablet b5057b87f1464fc0ac3c6943975bcdd5 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:59.171586  6221 raft_consensus.cc:359] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.178498  6019 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcf0341ab4e4003ba68968236997a97 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:59.173952  6170 tablet_service.cc:1511] Processing CreateTablet for tablet 85d394d62b444ec9b807d39ea09362cb (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:59.186699  6021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5057b87f1464fc0ac3c6943975bcdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.187302  6019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcf0341ab4e4003ba68968236997a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.187600  6170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d394d62b444ec9b807d39ea09362cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.183616  6094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d394d62b444ec9b807d39ea09362cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.198714  6221 raft_consensus.cc:385] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.198454  6096 tablet_service.cc:1511] Processing CreateTablet for tablet b5057b87f1464fc0ac3c6943975bcdd5 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:59.199214  6221 raft_consensus.cc:740] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.199963  6096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5057b87f1464fc0ac3c6943975bcdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.200003  6221 consensus_queue.cc:260] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.181282  6095 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcf0341ab4e4003ba68968236997a97 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:59.207670  6095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcf0341ab4e4003ba68968236997a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.209259  6220 tablet_bootstrap.cc:492] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.209836  6220 ts_tablet_manager.cc:1403] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.068s	user 0.030s	sys 0.019s
I20260709 13:15:59.212880  6220 raft_consensus.cc:359] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.213683  6220 raft_consensus.cc:385] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.214097  6220 raft_consensus.cc:740] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.218822  6222 raft_consensus.cc:385] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.219723  6222 raft_consensus.cc:740] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.221272  6222 consensus_queue.cc:260] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.222429  6220 consensus_queue.cc:260] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.226118  6167 tablet_service.cc:1511] Processing CreateTablet for tablet b5057b87f1464fc0ac3c6943975bcdd5 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:59.233286  6167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5057b87f1464fc0ac3c6943975bcdd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.241250  6018 tablet_service.cc:1511] Processing CreateTablet for tablet 46b44862ac6c4b79a974ee6f3513430b (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:59.248762  6221 ts_tablet_manager.cc:1434] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.102s	user 0.016s	sys 0.006s
I20260709 13:15:59.248903  6220 ts_tablet_manager.cc:1434] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.005s
I20260709 13:15:59.251159  6018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b44862ac6c4b79a974ee6f3513430b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.252528  6131 heartbeater.cc:499] Master 127.5.72.190:42289 was elected leader, sending a full tablet report...
I20260709 13:15:59.272053  6205 heartbeater.cc:499] Master 127.5.72.190:42289 was elected leader, sending a full tablet report...
I20260709 13:15:59.270524  6056 heartbeater.cc:499] Master 127.5.72.190:42289 was elected leader, sending a full tablet report...
I20260709 13:15:59.270579  6220 tablet_bootstrap.cc:492] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.277067  6096 tablet_service.cc:1511] Processing CreateTablet for tablet 46b44862ac6c4b79a974ee6f3513430b (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:59.278662  6096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b44862ac6c4b79a974ee6f3513430b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.281244  6220 tablet_bootstrap.cc:654] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.280483  6227 raft_consensus.cc:493] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.281656  6221 tablet_bootstrap.cc:492] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.281764  6227 raft_consensus.cc:515] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.282483  6166 tablet_service.cc:1511] Processing CreateTablet for tablet 46b44862ac6c4b79a974ee6f3513430b (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:59.283882  6166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b44862ac6c4b79a974ee6f3513430b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.295567  6021 tablet_service.cc:1511] Processing CreateTablet for tablet d0e92aa120df4961922986aa3cb4e5a8 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:59.286298  6222 ts_tablet_manager.cc:1434] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.121s	user 0.020s	sys 0.008s
I20260709 13:15:59.303479  6222 tablet_bootstrap.cc:492] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.313284  6021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e92aa120df4961922986aa3cb4e5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.313990  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" is_pre_election: true
I20260709 13:15:59.303623  6221 tablet_bootstrap.cc:654] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:59.315804  6071 leader_election.cc:343] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.304304  6227 leader_election.cc:290] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.310284  6094 tablet_service.cc:1511] Processing CreateTablet for tablet d0e92aa120df4961922986aa3cb4e5a8 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:59.324225  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:15:59.325577  6180 raft_consensus.cc:2468] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 0.
I20260709 13:15:59.326112  6222 tablet_bootstrap.cc:654] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.326753  6069 leader_election.cc:304] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:15:59.327405  6094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e92aa120df4961922986aa3cb4e5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.327986  6227 raft_consensus.cc:2804] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.328392  6227 raft_consensus.cc:493] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.328810  6227 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.332858  6169 tablet_service.cc:1511] Processing CreateTablet for tablet d0e92aa120df4961922986aa3cb4e5a8 (DEFAULT_TABLE table=test-workload [id=c2441e42c5d14497a539ecb1d01e28c1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:59.337618  6220 tablet_bootstrap.cc:492] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.337955  6221 tablet_bootstrap.cc:492] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.338446  6220 ts_tablet_manager.cc:1403] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.068s	user 0.009s	sys 0.007s
I20260709 13:15:59.338783  6169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e92aa120df4961922986aa3cb4e5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:59.339558  6227 raft_consensus.cc:515] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.341812  6220 raft_consensus.cc:359] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.338469  6221 ts_tablet_manager.cc:1403] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.000s
I20260709 13:15:59.342478  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:15:59.344682  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:15:59.345273  6180 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.345319  6221 raft_consensus.cc:359] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.346174  6221 raft_consensus.cc:385] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.346343  6227 leader_election.cc:290] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 1 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.346493  6221 raft_consensus.cc:740] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.347375  6221 consensus_queue.cc:260] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.348311  6031 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.349488  6221 ts_tablet_manager.cc:1434] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:15:59.350713  6220 raft_consensus.cc:385] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.351019  6220 raft_consensus.cc:740] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.352128  6180 raft_consensus.cc:2468] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 1.
I20260709 13:15:59.350467  6221 tablet_bootstrap.cc:492] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.353214  6069 leader_election.cc:304] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:15:59.351886  6220 consensus_queue.cc:260] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.359735  6227 raft_consensus.cc:2804] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.360095  6031 raft_consensus.cc:2468] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 1.
I20260709 13:15:59.360999  6222 tablet_bootstrap.cc:492] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.361703  6222 ts_tablet_manager.cc:1403] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.003s
I20260709 13:15:59.365895  6222 raft_consensus.cc:359] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.366999  6222 raft_consensus.cc:385] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.367101  6226 raft_consensus.cc:493] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.367540  6222 raft_consensus.cc:740] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.367849  6226 raft_consensus.cc:515] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.368754  6222 consensus_queue.cc:260] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.373759  6221 tablet_bootstrap.cc:654] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.379091  6222 ts_tablet_manager.cc:1434] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:15:59.382020  6222 tablet_bootstrap.cc:492] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.387876  6226 leader_election.cc:290] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.397933  6221 tablet_bootstrap.cc:492] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.398792  6221 ts_tablet_manager.cc:1403] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.003s
I20260709 13:15:59.408198  6220 ts_tablet_manager.cc:1434] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.069s	user 0.020s	sys 0.031s
I20260709 13:15:59.409370  6220 tablet_bootstrap.cc:492] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.415992  6220 tablet_bootstrap.cc:654] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.408383  6221 raft_consensus.cc:359] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.419370  6221 raft_consensus.cc:385] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.419785  6221 raft_consensus.cc:740] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.420598  6221 consensus_queue.cc:260] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.421947  6222 tablet_bootstrap.cc:654] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.428253  6220 tablet_bootstrap.cc:492] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.428978  6220 ts_tablet_manager.cc:1403] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.020s	user 0.002s	sys 0.012s
I20260709 13:15:59.422924  6221 ts_tablet_manager.cc:1434] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:15:59.424072  6227 raft_consensus.cc:697] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 LEADER]: Becoming Leader. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:15:59.430104  6227 consensus_queue.cc:237] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.432473  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a2bcee8dc14683a3de1e27aac3e496" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:15:59.432399  6220 raft_consensus.cc:359] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.433517  6220 raft_consensus.cc:385] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.433866  6220 raft_consensus.cc:740] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.435513  6221 tablet_bootstrap.cc:492] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.440346  6222 tablet_bootstrap.cc:492] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.440883  6222 ts_tablet_manager.cc:1403] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.000s
I20260709 13:15:59.440071  6220 consensus_queue.cc:260] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.442726  6106 raft_consensus.cc:2468] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:15:59.444103  6222 raft_consensus.cc:359] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.444811  6222 raft_consensus.cc:385] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.445094  6222 raft_consensus.cc:740] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.445375  5996 leader_election.cc:304] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.446534  6226 raft_consensus.cc:2804] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.446705  6222 consensus_queue.cc:260] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.447070  6226 raft_consensus.cc:493] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.447572  6226 raft_consensus.cc:3060] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.452333  6220 ts_tablet_manager.cc:1434] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:15:59.453298  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a2bcee8dc14683a3de1e27aac3e496" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:15:59.453504  6220 tablet_bootstrap.cc:492] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
W20260709 13:15:59.455269  5994 leader_election.cc:343] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.457535  6221 tablet_bootstrap.cc:654] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.458069  5928 catalog_manager.cc:5697] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "848c6bc37a7d4316aa29e395d99215e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.460899  6226 raft_consensus.cc:515] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.461649  6220 tablet_bootstrap.cc:654] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:59.464156  5977 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:59.466503  6222 ts_tablet_manager.cc:1434] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:15:59.466766  6221 tablet_bootstrap.cc:492] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.467360  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a2bcee8dc14683a3de1e27aac3e496" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:15:59.467667  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a2bcee8dc14683a3de1e27aac3e496" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:15:59.469226  6106 raft_consensus.cc:3060] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.468259  6226 leader_election.cc:290] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.469115  6221 ts_tablet_manager.cc:1403] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.006s
I20260709 13:15:59.474079  6222 tablet_bootstrap.cc:492] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.476917  6228 raft_consensus.cc:493] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.477404  6228 raft_consensus.cc:515] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.479975  6228 leader_election.cc:290] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
W20260709 13:15:59.480981  5994 leader_election.cc:343] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.482889  6106 raft_consensus.cc:2468] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:15:59.484570  5996 leader_election.cc:304] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 2ab5a16eb5b04b97b6cfc1361853a74a
I20260709 13:15:59.485694  6226 raft_consensus.cc:2804] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.485776  6222 tablet_bootstrap.cc:654] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.500025  6226 raft_consensus.cc:697] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:15:59.501109  6226 consensus_queue.cc:237] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.502820  6222 tablet_bootstrap.cc:492] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.503405  6222 ts_tablet_manager.cc:1403] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.008s
I20260709 13:15:59.506168  6221 raft_consensus.cc:359] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.507596  6221 raft_consensus.cc:385] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.507992  6221 raft_consensus.cc:740] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.509070  6220 tablet_bootstrap.cc:492] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.506531  6222 raft_consensus.cc:359] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.509536  6222 raft_consensus.cc:385] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.509965  6222 raft_consensus.cc:740] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.510915  6222 consensus_queue.cc:260] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.508855  6221 consensus_queue.cc:260] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.511628  5927 catalog_manager.cc:5697] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.514438  6221 ts_tablet_manager.cc:1434] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:15:59.509557  6220 ts_tablet_manager.cc:1403] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.056s	user 0.006s	sys 0.014s
I20260709 13:15:59.515444  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" is_pre_election: true
I20260709 13:15:59.518733  6031 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 0.
I20260709 13:15:59.519268  6221 tablet_bootstrap.cc:492] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.520185  6145 leader_election.cc:304] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.521037  6228 raft_consensus.cc:2804] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.521400  6228 raft_consensus.cc:493] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.521828  6228 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.521481  6220 raft_consensus.cc:359] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.522374  6220 raft_consensus.cc:385] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.522718  6220 raft_consensus.cc:740] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.523592  6220 consensus_queue.cc:260] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.526566  6221 tablet_bootstrap.cc:654] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.528808  6228 raft_consensus.cc:515] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.531833  6220 ts_tablet_manager.cc:1434] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 13:15:59.531834  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:15:59.532805  6031 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.532977  6220 tablet_bootstrap.cc:492] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.518003  6222 ts_tablet_manager.cc:1434] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:15:59.536959  6222 tablet_bootstrap.cc:492] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.536803  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:15:59.537508  6106 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 0.
I20260709 13:15:59.538363  6228 leader_election.cc:290] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:15:59.540705  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:15:59.541272  6106 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.583498  6222 tablet_bootstrap.cc:654] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.582249  6221 tablet_bootstrap.cc:492] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.584192  6106 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 1.
I20260709 13:15:59.584444  6221 ts_tablet_manager.cc:1403] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.006s
I20260709 13:15:59.585249  6145 leader_election.cc:304] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:15:59.583504  6220 tablet_bootstrap.cc:654] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.585997  6228 raft_consensus.cc:2804] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.587735  6221 raft_consensus.cc:359] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.588690  6221 raft_consensus.cc:385] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.589066  6221 raft_consensus.cc:740] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.590039  6221 consensus_queue.cc:260] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.592949  6221 ts_tablet_manager.cc:1434] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:59.594254  6221 tablet_bootstrap.cc:492] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.596218  6031 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 1.
I20260709 13:15:59.600180  6228 raft_consensus.cc:697] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Becoming Leader. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:15:59.601095  6228 consensus_queue.cc:237] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.604295  6221 tablet_bootstrap.cc:654] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.609155  6222 tablet_bootstrap.cc:492] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.609894  6222 ts_tablet_manager.cc:1403] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.073s	user 0.039s	sys 0.010s
I20260709 13:15:59.612275  6220 tablet_bootstrap.cc:492] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.612680  6220 ts_tablet_manager.cc:1403] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.032s
I20260709 13:15:59.614723  6222 raft_consensus.cc:359] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.615836  6222 raft_consensus.cc:385] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.615618  6220 raft_consensus.cc:359] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.616449  6220 raft_consensus.cc:385] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.616420  6222 raft_consensus.cc:740] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.616941  6220 raft_consensus.cc:740] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.617848  6220 consensus_queue.cc:260] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.617945  5927 catalog_manager.cc:5697] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131). New cstate: current_term: 1 leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:59.618003  6222 consensus_queue.cc:260] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.621387  6222 ts_tablet_manager.cc:1434] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:15:59.623631  6222 tablet_bootstrap.cc:492] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.622159  6228 raft_consensus.cc:493] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.625505  6228 raft_consensus.cc:515] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.626328  6220 ts_tablet_manager.cc:1434] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:15:59.627745  6228 leader_election.cc:290] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:15:59.628888  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531ec449e905416c95437a5081a86db4" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" is_pre_election: true
I20260709 13:15:59.629637  6220 tablet_bootstrap.cc:492] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.629528  6031 raft_consensus.cc:2468] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 0.
I20260709 13:15:59.631036  6145 leader_election.cc:304] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.631824  6228 raft_consensus.cc:2804] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.634104  6228 raft_consensus.cc:493] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.634433  6228 raft_consensus.cc:3060] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.632095  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531ec449e905416c95437a5081a86db4" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:15:59.635167  6106 raft_consensus.cc:2468] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 0.
I20260709 13:15:59.637255  6222 tablet_bootstrap.cc:654] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.632685  6221 tablet_bootstrap.cc:492] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.639068  6221 ts_tablet_manager.cc:1403] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.015s
I20260709 13:15:59.638486  6220 tablet_bootstrap.cc:654] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.642323  6228 raft_consensus.cc:515] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.642685  6221 raft_consensus.cc:359] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.644176  6221 raft_consensus.cc:385] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.644563  6221 raft_consensus.cc:740] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.645193  6228 leader_election.cc:290] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:15:59.645367  6221 consensus_queue.cc:260] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.646322  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531ec449e905416c95437a5081a86db4" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:15:59.646895  6031 raft_consensus.cc:3060] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.646795  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531ec449e905416c95437a5081a86db4" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:15:59.647632  6106 raft_consensus.cc:3060] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.649737  6221 ts_tablet_manager.cc:1434] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:15:59.650722  6221 tablet_bootstrap.cc:492] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.653707  6031 raft_consensus.cc:2468] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 1.
I20260709 13:15:59.655051  6145 leader_election.cc:304] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.656147  6228 raft_consensus.cc:2804] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.656690  6228 raft_consensus.cc:697] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Becoming Leader. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:15:59.657755  6228 consensus_queue.cc:237] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.658286  6221 tablet_bootstrap.cc:654] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.668879  6222 tablet_bootstrap.cc:492] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.669369  6222 ts_tablet_manager.cc:1403] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.004s
I20260709 13:15:59.671346  6220 tablet_bootstrap.cc:492] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.671814  6220 ts_tablet_manager.cc:1403] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.042s	user 0.026s	sys 0.000s
I20260709 13:15:59.672077  6226 raft_consensus.cc:493] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.672223  6222 raft_consensus.cc:359] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.673084  6222 raft_consensus.cc:385] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.672796  6226 raft_consensus.cc:515] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.673413  6222 raft_consensus.cc:740] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.674825  6220 raft_consensus.cc:359] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.675527  6220 raft_consensus.cc:385] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.675484  6226 leader_election.cc:290] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.676024  6220 raft_consensus.cc:740] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.676855  6220 consensus_queue.cc:260] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.679435  6220 ts_tablet_manager.cc:1434] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:59.679302  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:15:59.680078  6105 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:15:59.680414  6220 tablet_bootstrap.cc:492] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.682108  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:15:59.683755  5996 leader_election.cc:304] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.684953  6243 raft_consensus.cc:493] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.685557  6243 raft_consensus.cc:515] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.687361  6220 tablet_bootstrap.cc:654] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.688262  6243 leader_election.cc:290] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.688922  6243 raft_consensus.cc:2804] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.689320  6243 raft_consensus.cc:493] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.689725  6243 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.692328  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:15:59.693212  6179 raft_consensus.cc:2468] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:15:59.692770  6222 consensus_queue.cc:260] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.695199  6222 ts_tablet_manager.cc:1434] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:15:59.695837  6180 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:15:59.696385  6222 tablet_bootstrap.cc:492] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.675923  6106 raft_consensus.cc:2468] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 1.
I20260709 13:15:59.698853  5994 leader_election.cc:304] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.702953  6221 tablet_bootstrap.cc:492] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.703471  6221 ts_tablet_manager.cc:1403] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.032s
I20260709 13:15:59.705067  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
W20260709 13:15:59.706480  5996 leader_election.cc:343] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.706439  6221 raft_consensus.cc:359] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.707382  6221 raft_consensus.cc:385] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.707818  6221 raft_consensus.cc:740] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.708665  6222 tablet_bootstrap.cc:654] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.708763  6221 consensus_queue.cc:260] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.707471  6226 raft_consensus.cc:2804] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.714936  6226 raft_consensus.cc:493] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.715539  6226 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.718267  6221 ts_tablet_manager.cc:1434] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:15:59.719367  6248 raft_consensus.cc:493] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.719966  6248 raft_consensus.cc:515] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.722458  6221 tablet_bootstrap.cc:492] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.722735  6226 raft_consensus.cc:515] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.727762  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a2bcee8dc14683a3de1e27aac3e496" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" is_pre_election: true
I20260709 13:15:59.729595  6221 tablet_bootstrap.cc:654] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.730476  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a2bcee8dc14683a3de1e27aac3e496" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:15:59.731655  6106 raft_consensus.cc:2393] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ab5a16eb5b04b97b6cfc1361853a74a in current term 1: Already voted for candidate b0f3ceeef7f64b8d80f68433a37e629b in this term.
I20260709 13:15:59.732529  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:15:59.734016  6145 leader_election.cc:304] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a; no voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:15:59.728937  6243 raft_consensus.cc:515] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
W20260709 13:15:59.734220  5996 leader_election.cc:343] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.735128  6228 raft_consensus.cc:3060] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.736516  5927 catalog_manager.cc:5697] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131). New cstate: current_term: 1 leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:59.740422  6248 leader_election.cc:290] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:15:59.741482  6221 tablet_bootstrap.cc:492] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.744362  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:15:59.745218  6180 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.750313  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:15:59.750934  6179 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.751936  6243 leader_election.cc:290] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.752620  6226 leader_election.cc:290] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.754849  6228 raft_consensus.cc:2749] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.755318  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:15:59.755885  6105 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.755806  6180 raft_consensus.cc:2468] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:15:59.742030  6221 ts_tablet_manager.cc:1403] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.002s
I20260709 13:15:59.758184  6222 tablet_bootstrap.cc:492] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.758184  6220 tablet_bootstrap.cc:492] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.758802  6220 ts_tablet_manager.cc:1403] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.050s
I20260709 13:15:59.758849  6221 raft_consensus.cc:359] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.759729  6221 raft_consensus.cc:385] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.760047  6221 raft_consensus.cc:740] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.760196  5994 leader_election.cc:304] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:15:59.758802  6222 ts_tablet_manager.cc:1403] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.063s	user 0.019s	sys 0.003s
I20260709 13:15:59.760849  6221 consensus_queue.cc:260] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.761857  6243 raft_consensus.cc:2804] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.762349  6243 raft_consensus.cc:697] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:15:59.763273  6243 consensus_queue.cc:237] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.763971  6222 raft_consensus.cc:359] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.764658  6222 raft_consensus.cc:385] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.764930  6105 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:15:59.764957  6222 raft_consensus.cc:740] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.765842  6220 raft_consensus.cc:359] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.766268  5996 leader_election.cc:304] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.766580  6220 raft_consensus.cc:385] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.766911  6220 raft_consensus.cc:740] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.767141  6226 raft_consensus.cc:2804] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.767521  6221 ts_tablet_manager.cc:1434] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:15:59.767675  6179 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:15:59.768366  6221 tablet_bootstrap.cc:492] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.767848  6220 consensus_queue.cc:260] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.767910  6226 raft_consensus.cc:697] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:15:59.772940  6226 consensus_queue.cc:237] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.778399  6222 consensus_queue.cc:260] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.779441  6221 tablet_bootstrap.cc:654] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.781591  6222 ts_tablet_manager.cc:1434] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:15:59.773921  6220 ts_tablet_manager.cc:1434] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:15:59.783579  6220 tablet_bootstrap.cc:492] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.788930  6258 raft_consensus.cc:493] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.789424  6258 raft_consensus.cc:515] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.791244  6222 tablet_bootstrap.cc:492] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.793531  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:15:59.794195  6105 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:15:59.795666  5996 leader_election.cc:304] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.795960  6221 tablet_bootstrap.cc:492] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.796355  6221 ts_tablet_manager.cc:1403] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.010s
I20260709 13:15:59.797000  6226 raft_consensus.cc:2804] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.797139  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:15:59.797405  6226 raft_consensus.cc:493] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.797767  6226 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:59.798704  5994 leader_election.cc:343] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.799031  6222 tablet_bootstrap.cc:654] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.799116  6221 raft_consensus.cc:359] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.800376  6220 tablet_bootstrap.cc:654] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.800835  6221 raft_consensus.cc:385] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.801185  6221 raft_consensus.cc:740] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.801980  6221 consensus_queue.cc:260] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.804697  6243 raft_consensus.cc:493] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.805147  6243 raft_consensus.cc:515] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.803961  6258 leader_election.cc:290] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.810423  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:15:59.815630  6179 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:15:59.810885  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:15:59.816280  6220 tablet_bootstrap.cc:492] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.816051  6227 consensus_queue.cc:1048] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:59.810766  5926 catalog_manager.cc:5697] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.816562  6105 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:15:59.820624  6221 ts_tablet_manager.cc:1434] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.003s
I20260709 13:15:59.812647  6226 raft_consensus.cc:515] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.820756  5996 leader_election.cc:304] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.822084  6258 raft_consensus.cc:2804] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.822428  6258 raft_consensus.cc:493] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.822765  6258 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.822816  6226 leader_election.cc:290] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.816702  6220 ts_tablet_manager.cc:1403] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.004s
I20260709 13:15:59.827847  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:15:59.827922  6220 raft_consensus.cc:359] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.828841  6220 raft_consensus.cc:385] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.829191  6220 raft_consensus.cc:740] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.829452  6221 tablet_bootstrap.cc:492] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b: Bootstrap starting.
I20260709 13:15:59.830065  6220 consensus_queue.cc:260] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.831681  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
W20260709 13:15:59.832432  6127 debug-util.cc:398] Leaking SignalData structure 0x7b080010fb40 after lost signal to thread 5413
I20260709 13:15:59.832870  6220 ts_tablet_manager.cc:1434] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:59.828485  6105 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.809022  6243 leader_election.cc:290] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.834096  6220 tablet_bootstrap.cc:492] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
W20260709 13:15:59.835659  5994 leader_election.cc:343] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.846451  6258 raft_consensus.cc:515] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.850854  6221 tablet_bootstrap.cc:654] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.865784  6222 tablet_bootstrap.cc:492] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.866267  6222 ts_tablet_manager.cc:1403] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.039s
I20260709 13:15:59.866906  6258 leader_election.cc:290] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:15:59.867679  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:15:59.868223  6106 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.868113  6105 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:15:59.868459  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:15:59.869017  6179 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.868935  6222 raft_consensus.cc:359] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.869628  6222 raft_consensus.cc:385] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.869889  6222 raft_consensus.cc:740] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.870528  6222 consensus_queue.cc:260] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.871968  6220 tablet_bootstrap.cc:654] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.872444  6222 ts_tablet_manager.cc:1434] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:15:59.873445  6222 tablet_bootstrap.cc:492] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:15:59.877326  5996 leader_election.cc:304] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 2ab5a16eb5b04b97b6cfc1361853a74a
I20260709 13:15:59.878367  6258 raft_consensus.cc:2804] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.878846  6258 raft_consensus.cc:697] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:15:59.879612  6258 consensus_queue.cc:237] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.881067  6227 consensus_queue.cc:1048] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:59.884657  6106 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:15:59.886080  5996 leader_election.cc:304] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:15:59.894726  6243 raft_consensus.cc:2804] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.895412  6243 raft_consensus.cc:697] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:15:59.896504  6243 consensus_queue.cc:237] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.904124  6179 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:15:59.916929  6226 consensus_queue.cc:1048] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:59.917650  6222 tablet_bootstrap.cc:654] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.933737  6220 tablet_bootstrap.cc:492] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.934294  6220 ts_tablet_manager.cc:1403] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.100s	user 0.028s	sys 0.032s
I20260709 13:15:59.921433  5929 catalog_manager.cc:5697] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.936956  6220 raft_consensus.cc:359] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.937618  6220 raft_consensus.cc:385] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.937880  6220 raft_consensus.cc:740] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.938668  6220 consensus_queue.cc:260] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.941519  6220 ts_tablet_manager.cc:1434] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:59.942812  6220 tablet_bootstrap.cc:492] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3: Bootstrap starting.
I20260709 13:15:59.949872  6220 tablet_bootstrap.cc:654] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.953955  6222 tablet_bootstrap.cc:492] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:15:59.954536  6222 ts_tablet_manager.cc:1403] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.000s
I20260709 13:15:59.956036  6221 tablet_bootstrap.cc:492] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b: No bootstrap required, opened a new log
I20260709 13:15:59.956473  6221 ts_tablet_manager.cc:1403] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent bootstrapping tablet: real 0.133s	user 0.016s	sys 0.035s
I20260709 13:15:59.960004  6221 raft_consensus.cc:359] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.960752  6221 raft_consensus.cc:385] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.961148  6221 raft_consensus.cc:740] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.962216  6221 consensus_queue.cc:260] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.964526  6221 ts_tablet_manager.cc:1434] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:59.965577  6220 tablet_bootstrap.cc:492] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3: No bootstrap required, opened a new log
I20260709 13:15:59.967502  6220 ts_tablet_manager.cc:1403] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.008s
I20260709 13:15:59.978163  6258 consensus_queue.cc:1048] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:59.978766  6220 raft_consensus.cc:359] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.979396  6220 raft_consensus.cc:385] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.979650  6220 raft_consensus.cc:740] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.980286  6220 consensus_queue.cc:260] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.984251  6222 raft_consensus.cc:359] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.987318  6222 raft_consensus.cc:385] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.987747  6222 raft_consensus.cc:740] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.991252  6220 ts_tablet_manager.cc:1434] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:15:59.992470  6222 consensus_queue.cc:260] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:15:59.995234  6222 ts_tablet_manager.cc:1434] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:15:59.996438  6222 tablet_bootstrap.cc:492] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a: Bootstrap starting.
I20260709 13:16:00.004122  6222 tablet_bootstrap.cc:654] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.010462  6222 tablet_bootstrap.cc:492] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a: No bootstrap required, opened a new log
I20260709 13:16:00.011183  6222 ts_tablet_manager.cc:1403] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.002s
I20260709 13:16:00.014501  6222 raft_consensus.cc:359] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.015280  6222 raft_consensus.cc:385] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.015596  6222 raft_consensus.cc:740] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.016297  6222 consensus_queue.cc:260] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.018174  6222 ts_tablet_manager.cc:1434] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:16:00.022099  5929 catalog_manager.cc:5697] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.023545  5929 catalog_manager.cc:5697] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.042092  6228 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.058980  6228 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.076454  6243 raft_consensus.cc:493] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.076920  6243 raft_consensus.cc:515] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.078863  6243 leader_election.cc:290] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:00.080052  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:16:00.080761  6179 raft_consensus.cc:2468] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:16:00.081527  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:16:00.082298  5994 leader_election.cc:304] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:00.082850  6106 raft_consensus.cc:2468] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:16:00.083098  6243 raft_consensus.cc:2804] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.083875  6243 raft_consensus.cc:493] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.084237  6243 raft_consensus.cc:3060] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.091696  6243 raft_consensus.cc:515] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.093886  6243 leader_election.cc:290] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:00.094775  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:00.095429  6106 raft_consensus.cc:3060] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.095777  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:00.096349  6179 raft_consensus.cc:3060] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.099997  6106 raft_consensus.cc:2468] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:16:00.101006  5996 leader_election.cc:304] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:00.101671  6243 raft_consensus.cc:2804] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.102097  6243 raft_consensus.cc:697] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:00.102921  6243 consensus_queue.cc:237] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.103585  6179 raft_consensus.cc:2468] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:16:00.110126  5926 catalog_manager.cc:5697] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.123807  6248 consensus_queue.cc:1048] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.137431  6248 consensus_queue.cc:1048] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.209216  6279 raft_consensus.cc:493] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.209746  6279 raft_consensus.cc:515] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.212026  6279 leader_election.cc:290] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:00.212728  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" is_pre_election: true
I20260709 13:16:00.213277  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:16:00.214092  6179 raft_consensus.cc:2393] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 848c6bc37a7d4316aa29e395d99215e3 in current term 1: Already voted for candidate b0f3ceeef7f64b8d80f68433a37e629b in this term.
I20260709 13:16:00.215098  6069 leader_election.cc:304] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3; no voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:00.215926  6279 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.220724  6279 raft_consensus.cc:2749] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:00.281718  6243 raft_consensus.cc:493] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.282274  6243 raft_consensus.cc:515] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.283950  6243 leader_election.cc:290] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:00.284911  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:16:00.285099  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" is_pre_election: true
I20260709 13:16:00.285441  6105 raft_consensus.cc:2468] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:16:00.285785  6179 raft_consensus.cc:2468] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 0.
I20260709 13:16:00.286479  5996 leader_election.cc:304] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:00.287254  6243 raft_consensus.cc:2804] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.287624  6243 raft_consensus.cc:493] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.287919  6243 raft_consensus.cc:3060] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.292279  6243 raft_consensus.cc:515] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.293992  6243 leader_election.cc:290] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:00.295061  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:00.294885  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:00.295550  6179 raft_consensus.cc:3060] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.295627  6105 raft_consensus.cc:3060] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.302132  6179 raft_consensus.cc:2468] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:16:00.302132  6105 raft_consensus.cc:2468] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 1.
I20260709 13:16:00.303444  5994 leader_election.cc:304] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:00.304277  6243 raft_consensus.cc:2804] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.304768  6243 raft_consensus.cc:697] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:00.305629  6243 consensus_queue.cc:237] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.314457  6243 consensus_queue.cc:1048] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.315536  5926 catalog_manager.cc:5697] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 0 to 1, leader changed from <none> to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.326828  6226 consensus_queue.cc:1048] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.348392  6258 consensus_queue.cc:1048] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.354086  6248 raft_consensus.cc:493] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.354566  6248 raft_consensus.cc:515] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.356600  6248 leader_election.cc:290] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:16:00.358120  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" is_pre_election: true
I20260709 13:16:00.361188  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "848c6bc37a7d4316aa29e395d99215e3" is_pre_election: true
I20260709 13:16:00.362196  6105 raft_consensus.cc:2393] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ab5a16eb5b04b97b6cfc1361853a74a in current term 1: Already voted for candidate b0f3ceeef7f64b8d80f68433a37e629b in this term.
I20260709 13:16:00.362839  6243 consensus_queue.cc:1048] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.363945  6145 leader_election.cc:304] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a; no voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:00.365402  6228 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.377678  6228 raft_consensus.cc:2749] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:00.403180  6258 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.416361  6243 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.493367  6243 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.509837  6243 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.511195  6106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:34140
I20260709 13:16:00.511794  6106 raft_consensus.cc:606] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:00.524722  6243 consensus_queue.cc:1048] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.542023  6266 consensus_queue.cc:1048] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.546952  6180 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:50004
I20260709 13:16:00.547592  6180 raft_consensus.cc:606] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:00.555699  6179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "531ec449e905416c95437a5081a86db4"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:50004
I20260709 13:16:00.556303  6179 raft_consensus.cc:606] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:00.577064  6031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:60156
I20260709 13:16:00.577576  6031 raft_consensus.cc:606] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Received request to transfer leadership to TS 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:16:00.582922  6031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:60156
I20260709 13:16:00.583395  6031 raft_consensus.cc:606] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Received request to transfer leadership to TS 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:16:00.588279  6031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:60156
I20260709 13:16:00.588747  6031 raft_consensus.cc:606] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Received request to transfer leadership to TS 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:16:00.593778  6031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:60156
I20260709 13:16:00.594234  6031 raft_consensus.cc:606] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Received request to transfer leadership to TS 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:16:00.599419  6031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:60156
I20260709 13:16:00.601166  6031 raft_consensus.cc:606] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Received request to transfer leadership to TS 2ab5a16eb5b04b97b6cfc1361853a74a
I20260709 13:16:00.736141  6248 raft_consensus.cc:993] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:00.736655  6228 raft_consensus.cc:1081] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:00.738166  6031 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "531ec449e905416c95437a5081a86db4"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60152
I20260709 13:16:00.738708  6031 raft_consensus.cc:493] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:00.739010  6031 raft_consensus.cc:3060] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.743486  6031 raft_consensus.cc:515] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.745612  6031 leader_election.cc:290] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 2 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:00.746546  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531ec449e905416c95437a5081a86db4" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:00.746837  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531ec449e905416c95437a5081a86db4" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:00.747229  6105 raft_consensus.cc:3060] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.747421  6179 raft_consensus.cc:3055] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:00.747819  6179 raft_consensus.cc:740] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:00.748525  6179 consensus_queue.cc:260] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.749439  6179 raft_consensus.cc:3060] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.754423  6179 raft_consensus.cc:2468] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 2.
I20260709 13:16:00.754637  6105 raft_consensus.cc:2468] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 2.
I20260709 13:16:00.756000  5994 leader_election.cc:304] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:00.756703  6266 raft_consensus.cc:2804] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:00.757170  6266 raft_consensus.cc:697] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:00.757956  6266 consensus_queue.cc:237] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.764864  6266 consensus_queue.cc:1048] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:00.765369  5929 catalog_manager.cc:5697] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 1 to 2, leader changed from 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.775301  6226 raft_consensus.cc:993] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:00.775748  6258 raft_consensus.cc:1081] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Signalling peer 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:00.778559  6179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:49992
I20260709 13:16:00.780375  6179 raft_consensus.cc:493] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:00.780823  6179 raft_consensus.cc:3060] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.788353  6179 raft_consensus.cc:515] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.792369  6179 leader_election.cc:290] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 2 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:16:00.794344  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:00.795060  6243 raft_consensus.cc:993] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:00.794399  6266 consensus_queue.cc:1048] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:00.795765  6243 raft_consensus.cc:1081] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Signalling peer 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:00.797559  6106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:34128
I20260709 13:16:00.798110  6106 raft_consensus.cc:493] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:00.798478  6106 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.794344  6105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e92aa120df4961922986aa3cb4e5a8" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:00.803607  6031 raft_consensus.cc:3055] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:00.803977  6031 raft_consensus.cc:740] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:00.804766  6031 consensus_queue.cc:260] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.805840  6031 raft_consensus.cc:3060] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.816432  6106 raft_consensus.cc:515] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.819339  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:00.819979  6030 raft_consensus.cc:3055] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:00.820320  6030 raft_consensus.cc:740] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:00.821107  6030 consensus_queue.cc:260] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.822216  6030 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.823145  6031 raft_consensus.cc:2468] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 2.
I20260709 13:16:00.824625  6106 leader_election.cc:290] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:00.824560  6145 leader_election.cc:304] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:16:00.826218  6228 raft_consensus.cc:2804] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:00.826387  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:00.826730  6228 raft_consensus.cc:697] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 LEADER]: Becoming Leader. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:00.827041  6180 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:00.827502  6228 consensus_queue.cc:237] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.834581  6180 raft_consensus.cc:2468] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:00.836776  6069 leader_election.cc:304] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:00.837482  6279 raft_consensus.cc:2804] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:00.841017  6030 raft_consensus.cc:2468] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:00.839018  5926 catalog_manager.cc:5697] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a reported cstate change: term changed from 1 to 2, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131). New cstate: current_term: 2 leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:00.842969  6279 raft_consensus.cc:697] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Becoming Leader. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:00.843946  6279 consensus_queue.cc:237] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:00.851336  5929 catalog_manager.cc:5697] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 reported cstate change: term changed from 1 to 2, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "848c6bc37a7d4316aa29e395d99215e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.014818  6258 raft_consensus.cc:993] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:01.015285  6258 raft_consensus.cc:1081] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Signalling peer 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:01.017053  6106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:34128
I20260709 13:16:01.017565  6106 raft_consensus.cc:493] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:01.017935  6106 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.020009  6306 raft_consensus.cc:993] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:01.020494  6306 raft_consensus.cc:1081] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:01.022153  6030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60144
I20260709 13:16:01.022768  6030 raft_consensus.cc:493] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:01.023167  6030 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.024441  6106 raft_consensus.cc:515] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.026757  6106 leader_election.cc:290] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:01.027912  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:01.028556  6179 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.029670  6031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:01.030265  6031 raft_consensus.cc:3055] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:01.030583  6031 raft_consensus.cc:740] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:01.031410  6031 consensus_queue.cc:260] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.032522  6031 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.037180  6179 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:01.038522  6180 raft_consensus.cc:1240] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Rejecting Update request from peer b0f3ceeef7f64b8d80f68433a37e629b for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:01.039280  6226 raft_consensus.cc:993] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:01.039076  6069 leader_election.cc:304] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:01.039705  6226 raft_consensus.cc:1081] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Signalling peer 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:01.040113  6279 raft_consensus.cc:2804] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:01.040639  6279 raft_consensus.cc:697] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Becoming Leader. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:01.037204  6030 raft_consensus.cc:515] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.041482  6104 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:34128
I20260709 13:16:01.042115  6104 raft_consensus.cc:493] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:01.041756  6279 consensus_queue.cc:237] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.042526  6104 raft_consensus.cc:3060] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.043654  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:01.044210  6106 raft_consensus.cc:3055] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:01.044534  6106 raft_consensus.cc:740] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:01.045310  6106 consensus_queue.cc:260] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.046361  6106 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.050118  6104 raft_consensus.cc:515] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.051291  6030 leader_election.cc:290] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 2 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:01.052693  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:01.053282  6179 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.053426  6106 raft_consensus.cc:2468] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 2.
I20260709 13:16:01.054754  5996 leader_election.cc:304] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:01.055742  6266 raft_consensus.cc:2804] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:01.056185  6266 raft_consensus.cc:697] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:01.056953  6266 consensus_queue.cc:237] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.057272  5929 catalog_manager.cc:5697] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 reported cstate change: term changed from 1 to 2, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "848c6bc37a7d4316aa29e395d99215e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.065109  6248 raft_consensus.cc:993] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:01.065549  6248 raft_consensus.cc:1081] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:01.067348  6030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60152
I20260709 13:16:01.067960  6030 raft_consensus.cc:493] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:01.068352  6030 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.073699  6031 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:01.076602  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:01.076713  6029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b44862ac6c4b79a974ee6f3513430b" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:01.077229  6180 raft_consensus.cc:3060] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.077292  6029 raft_consensus.cc:3055] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:01.077700  6029 raft_consensus.cc:740] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:01.078447  6029 consensus_queue.cc:260] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.079576  6029 raft_consensus.cc:3060] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.082310  5929 catalog_manager.cc:5697] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 1 to 2, leader changed from 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.085595  6180 raft_consensus.cc:2468] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:01.086841  6069 leader_election.cc:304] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:01.087742  6179 raft_consensus.cc:2468] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 2.
I20260709 13:16:01.088224  6029 raft_consensus.cc:2468] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:01.089119  6279 raft_consensus.cc:2804] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:01.089653  6279 raft_consensus.cc:697] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Becoming Leader. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:01.089816  6104 leader_election.cc:290] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:01.090559  6279 consensus_queue.cc:237] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.092515  6030 raft_consensus.cc:515] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.095323  6104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:01.095856  6104 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.096407  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:01.096958  6180 raft_consensus.cc:3055] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:01.097301  6180 raft_consensus.cc:740] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:01.095888  6030 leader_election.cc:290] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 2 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:01.098215  6180 consensus_queue.cc:260] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.100468  6180 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.101305  5929 catalog_manager.cc:5697] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 reported cstate change: term changed from 1 to 2, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "848c6bc37a7d4316aa29e395d99215e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.102746  6104 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 2.
I20260709 13:16:01.103879  5996 leader_election.cc:304] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:01.104720  6266 raft_consensus.cc:2804] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:01.105189  6266 raft_consensus.cc:697] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:01.105983  6266 consensus_queue.cc:237] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.107576  6180 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 2.
I20260709 13:16:01.117115  5929 catalog_manager.cc:5697] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 1 to 2, leader changed from 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.124961  6226 raft_consensus.cc:993] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:01.125428  6226 raft_consensus.cc:1081] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Signalling peer 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:01.126973  6104 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:34128
I20260709 13:16:01.127424  6104 raft_consensus.cc:493] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:01.127738  6104 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.132756  6104 raft_consensus.cc:515] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.134511  6104 leader_election.cc:290] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:01.136471  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:01.136503  6180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:01.137017  6030 raft_consensus.cc:3055] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:01.137167  6180 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.137348  6030 raft_consensus.cc:740] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:01.138157  6030 consensus_queue.cc:260] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.139060  6030 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.143597  6030 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:01.144873  6071 leader_election.cc:304] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:01.145552  6279 raft_consensus.cc:2804] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:01.145870  6180 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 2.
I20260709 13:16:01.146004  6279 raft_consensus.cc:697] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Becoming Leader. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:01.147249  6279 consensus_queue.cc:237] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:01.153887  5929 catalog_manager.cc:5697] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 reported cstate change: term changed from 1 to 2, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "848c6bc37a7d4316aa29e395d99215e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.218899  6104 raft_consensus.cc:1275] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.220141  6226 consensus_queue.cc:1048] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.228629  6180 raft_consensus.cc:1275] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.230800  6258 consensus_queue.cc:1048] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.235865  6030 raft_consensus.cc:1275] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.237861  6284 consensus_queue.cc:1048] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:01.257476  6104 raft_consensus.cc:3060] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:01.264102  6104 raft_consensus.cc:1275] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.265425  6284 consensus_queue.cc:1048] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.356353  6030 raft_consensus.cc:1275] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.357590  6279 consensus_queue.cc:1048] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.369962  6180 raft_consensus.cc:1275] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.372011  6279 consensus_queue.cc:1048] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.472683  6180 raft_consensus.cc:1275] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.473855  6312 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.481292  6104 raft_consensus.cc:1275] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.482792  6258 consensus_queue.cc:1048] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.485841  6030 raft_consensus.cc:1275] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.487920  6306 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.520845  6180 raft_consensus.cc:1275] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.522404  6258 consensus_queue.cc:1048] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.530347  6106 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.532361  6258 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.544076  6180 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.550583  6258 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:01.562695  6180 raft_consensus.cc:1275] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.564044  6312 consensus_queue.cc:1048] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.575258  6030 raft_consensus.cc:1275] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.576673  6279 consensus_queue.cc:1048] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.665512  6030 raft_consensus.cc:1275] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.666885  6279 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:01.681650  6180 raft_consensus.cc:1275] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:01.683271  6306 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:03.629845  5410 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:16:03.630760  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:03.655786  6030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:47364
I20260709 13:16:03.656424  6030 raft_consensus.cc:606] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Received request to transfer leadership to TS 2ab5a16eb5b04b97b6cfc1361853a74a
I20260709 13:16:03.659075  6030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:47364
I20260709 13:16:03.659649  6030 raft_consensus.cc:606] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Received request to transfer leadership to TS 2ab5a16eb5b04b97b6cfc1361853a74a
I20260709 13:16:03.660539  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:03.660995  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:03.742585  6258 raft_consensus.cc:993] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:03.743182  6328 raft_consensus.cc:1081] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Signalling peer 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:03.744617  6179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:49992
I20260709 13:16:03.745040  6179 raft_consensus.cc:493] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.745282  6179 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.749650  6179 raft_consensus.cc:515] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:03.751312  6179 leader_election.cc:290] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 2 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:16:03.752116  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:03.752427  6104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:03.752724  6030 raft_consensus.cc:3055] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:03.752982  6104 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.753077  6030 raft_consensus.cc:740] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:03.753939  6030 consensus_queue.cc:260] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:03.755215  6030 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:03.759979  6104 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 2.
I20260709 13:16:03.761283  6145 leader_election.cc:304] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:03.762018  6030 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 2.
I20260709 13:16:03.762108  6284 raft_consensus.cc:2804] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:03.763358  6284 raft_consensus.cc:697] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 LEADER]: Becoming Leader. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:03.764147  6284 consensus_queue.cc:237] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:03.770315  5926 catalog_manager.cc:5697] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a reported cstate change: term changed from 1 to 2, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131). New cstate: current_term: 2 leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:03.987190  6258 raft_consensus.cc:993] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:03.987583  6328 raft_consensus.cc:1081] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Signalling peer 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:03.988983  6179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:49992
I20260709 13:16:03.989523  6179 raft_consensus.cc:493] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:03.989845  6179 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.994467  6179 raft_consensus.cc:515] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:03.995819  6179 leader_election.cc:290] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 3 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:16:03.996668  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:03.996904  6104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:03.997248  6030 raft_consensus.cc:3055] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:03.997473  6104 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:03.997617  6030 raft_consensus.cc:740] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:03.998500  6030 consensus_queue.cc:260] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:03.999596  6030 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.002774  6104 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 3.
I20260709 13:16:04.003729  6145 leader_election.cc:304] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:04.004470  6284 raft_consensus.cc:2804] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:04.004580  6030 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 3.
I20260709 13:16:04.004868  6284 raft_consensus.cc:697] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 LEADER]: Becoming Leader. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:04.005718  6284 consensus_queue.cc:237] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.013952  5926 catalog_manager.cc:5697] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a reported cstate change: term changed from 2 to 3, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131). New cstate: current_term: 3 leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:04.291718  6030 raft_consensus.cc:1275] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.292852  6284 consensus_queue.cc:1048] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.304823  6104 raft_consensus.cc:1275] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:04.306454  6284 consensus_queue.cc:1048] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:04.470314  6030 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:04.471555  6284 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.482435  6106 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:04.485213  6338 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:04.661854  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:04.674415  6179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:50004
I20260709 13:16:04.674871  6179 raft_consensus.cc:606] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:04.683970  6179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:50004
I20260709 13:16:04.684377  6179 raft_consensus.cc:606] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:04.688563  6106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:34140
I20260709 13:16:04.688989  6106 raft_consensus.cc:606] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:04.693219  6106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:34140
I20260709 13:16:04.693642  6106 raft_consensus.cc:606] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:04.698218  6106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
mode: GRACEFUL
new_leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:34140
I20260709 13:16:04.698743  6106 raft_consensus.cc:606] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Received request to transfer leadership to TS b0f3ceeef7f64b8d80f68433a37e629b
I20260709 13:16:04.852972  6306 raft_consensus.cc:993] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:04.853399  6344 raft_consensus.cc:1081] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:04.854799  6030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60144
I20260709 13:16:04.855334  6030 raft_consensus.cc:493] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:04.855587  6030 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.860196  6030 raft_consensus.cc:515] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.861733  6329 raft_consensus.cc:993] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:04.862035  6030 leader_election.cc:290] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:04.862221  6329 raft_consensus.cc:1081] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:04.862877  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:04.863210  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcf0341ab4e4003ba68968236997a97" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:04.863579  6106 raft_consensus.cc:3055] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:04.863761  6179 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.863897  6030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60152
I20260709 13:16:04.864033  6106 raft_consensus.cc:740] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:04.864446  6030 raft_consensus.cc:493] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:04.864831  6030 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.864892  6106 consensus_queue.cc:260] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.865899  6106 raft_consensus.cc:3060] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.870033  6179 raft_consensus.cc:2468] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:04.870636  6106 raft_consensus.cc:2468] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:04.871282  5994 leader_election.cc:304] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:04.871205  6030 raft_consensus.cc:515] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.872116  6328 raft_consensus.cc:2804] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:04.873171  6328 raft_consensus.cc:697] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:04.873472  6030 leader_election.cc:290] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:04.873833  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:04.874352  6106 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.874106  6328 consensus_queue.cc:237] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.875157  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0583fd471a2f48afa3d1a211dd84ff1d" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:04.875687  6179 raft_consensus.cc:3055] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:04.875981  6179 raft_consensus.cc:740] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:04.876629  6179 consensus_queue.cc:260] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.877707  6179 raft_consensus.cc:3060] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.880335  6106 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:04.881352  5996 leader_election.cc:304] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:04.881634  5926 catalog_manager.cc:5697] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 2 to 3, leader changed from 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 3 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:04.882825  6328 raft_consensus.cc:2804] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:04.883299  6328 raft_consensus.cc:697] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:04.883684  6179 raft_consensus.cc:2468] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:04.884097  6328 consensus_queue.cc:237] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.895665  5926 catalog_manager.cc:5697] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 2 to 3, leader changed from 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 3 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:04.904210  6306 raft_consensus.cc:993] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:04.904610  6344 raft_consensus.cc:1081] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:04.905787  6030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60144
I20260709 13:16:04.906199  6030 raft_consensus.cc:493] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:04.906476  6030 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.910938  6030 raft_consensus.cc:515] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.912808  6030 leader_election.cc:290] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:04.913564  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:04.913805  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:04.914109  6106 raft_consensus.cc:3055] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:04.914258  6179 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.914428  6106 raft_consensus.cc:740] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:04.915189  6106 consensus_queue.cc:260] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.916234  6106 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:04.920228  6179 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:04.921280  5994 leader_election.cc:304] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:04.921866  6106 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:04.921957  6328 raft_consensus.cc:2804] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:04.922475  6328 raft_consensus.cc:697] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:04.923262  6328 consensus_queue.cc:237] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:04.929345  5926 catalog_manager.cc:5697] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 2 to 3, leader changed from 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 3 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:05.035530  6306 raft_consensus.cc:993] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:05.036027  6306 raft_consensus.cc:1081] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:05.037883  6030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60144
I20260709 13:16:05.038401  6030 raft_consensus.cc:493] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:05.038817  6030 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.043262  6030 raft_consensus.cc:515] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:05.045184  6030 leader_election.cc:290] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:05.045954  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:05.046520  6106 raft_consensus.cc:3055] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:05.046819  6106 raft_consensus.cc:740] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:05.047372  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:05.047513  6106 consensus_queue.cc:260] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:05.047919  6179 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.048485  6106 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:05.052330  6179 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:05.052392  6106 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 3.
I20260709 13:16:05.053409  5996 leader_election.cc:304] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:05.054278  6345 raft_consensus.cc:2804] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:05.054816  6345 raft_consensus.cc:697] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:05.055665  6345 consensus_queue.cc:237] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:05.062642  5926 catalog_manager.cc:5697] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 2 to 3, leader changed from 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 3 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:05.105304  6338 raft_consensus.cc:993] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: : Instructing follower b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:05.106094  6329 raft_consensus.cc:1081] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 LEADER]: Signalling peer b0f3ceeef7f64b8d80f68433a37e629b to start an election
I20260709 13:16:05.109064  6030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
 from {username='slave'} at 127.0.0.1:60152
I20260709 13:16:05.110294  6030 raft_consensus.cc:493] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:05.110951  6030 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:05.115433  6030 raft_consensus.cc:515] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:05.116981  6030 leader_election.cc:290] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 4 election: Requested vote from peers 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:05.118098  6106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:05.118000  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:05.118711  6179 raft_consensus.cc:3055] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:05.118692  6106 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:05.119175  6179 raft_consensus.cc:740] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:05.119925  6179 consensus_queue.cc:260] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:05.121008  6179 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:05.125279  6106 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 4.
I20260709 13:16:05.126415  5996 leader_election.cc:304] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 848c6bc37a7d4316aa29e395d99215e3, b0f3ceeef7f64b8d80f68433a37e629b; no voters: 
I20260709 13:16:05.126897  6179 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0f3ceeef7f64b8d80f68433a37e629b in term 4.
I20260709 13:16:05.127156  6345 raft_consensus.cc:2804] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:05.127650  6345 raft_consensus.cc:697] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 LEADER]: Becoming Leader. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:05.128377  6345 consensus_queue.cc:237] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:05.133869  5926 catalog_manager.cc:5697] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b reported cstate change: term changed from 3 to 4, leader changed from 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131) to b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129). New cstate: current_term: 4 leader_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:05.262658  6106 raft_consensus.cc:1275] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.263880  6345 consensus_queue.cc:1048] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.274696  6179 raft_consensus.cc:1275] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.276116  6180 raft_consensus.cc:1275] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.278481  6353 consensus_queue.cc:1048] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.278847  6345 consensus_queue.cc:1048] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.291306  6106 raft_consensus.cc:1275] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.293398  6345 consensus_queue.cc:1048] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.343217  6179 raft_consensus.cc:1275] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.344300  6345 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.352896  6106 raft_consensus.cc:1275] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.354439  6353 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:05.502038  6179 raft_consensus.cc:1275] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.503335  6353 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.511636  6106 raft_consensus.cc:1275] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:05.513027  6345 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:05.666080  6180 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:05.667191  6345 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:05.675257  6106 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Refusing update from remote peer b0f3ceeef7f64b8d80f68433a37e629b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:05.677294  6328 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:07.705500  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:07.713567  6030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:47364
I20260709 13:16:07.714151  6030 raft_consensus.cc:606] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 LEADER]: Received request to transfer leadership to TS 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:16:07.716648  6030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:47364
I20260709 13:16:07.717023  6030 raft_consensus.cc:606] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Received request to transfer leadership to TS 2ab5a16eb5b04b97b6cfc1361853a74a
I20260709 13:16:07.719071  6030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:47364
I20260709 13:16:07.719455  6030 raft_consensus.cc:606] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Received request to transfer leadership to TS 2ab5a16eb5b04b97b6cfc1361853a74a
I20260709 13:16:07.721450  6030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea"
dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
mode: GRACEFUL
new_leader_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:47364
I20260709 13:16:07.721800  6030 raft_consensus.cc:606] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Received request to transfer leadership to TS 848c6bc37a7d4316aa29e395d99215e3
I20260709 13:16:07.722754  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:16:07.723134  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:07.826324  6367 raft_consensus.cc:993] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:07.826876  6364 raft_consensus.cc:1081] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Signalling peer 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:07.828330  6179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:49992
I20260709 13:16:07.828717  6179 raft_consensus.cc:493] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:07.828981  6179 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:07.833894  6179 raft_consensus.cc:515] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:07.835371  6179 leader_election.cc:290] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 4 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:16:07.836367  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:07.836669  6104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5057b87f1464fc0ac3c6943975bcdd5" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:07.837030  6030 raft_consensus.cc:3055] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:07.837255  6104 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:07.837383  6030 raft_consensus.cc:740] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:07.838289  6030 consensus_queue.cc:260] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:07.839540  6030 raft_consensus.cc:3060] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:07.843268  6104 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 4.
I20260709 13:16:07.844610  6145 leader_election.cc:304] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:07.845260  6030 raft_consensus.cc:2468] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 4.
I20260709 13:16:07.845700  6338 raft_consensus.cc:2804] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:07.847692  6338 raft_consensus.cc:697] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 LEADER]: Becoming Leader. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:07.848707  6338 consensus_queue.cc:237] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:07.855731  5929 catalog_manager.cc:5697] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a reported cstate change: term changed from 3 to 4, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131). New cstate: current_term: 4 leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:07.867761  6364 raft_consensus.cc:993] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:07.868530  6372 raft_consensus.cc:1081] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 LEADER]: Signalling peer 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:07.870244  6104 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:34128
I20260709 13:16:07.870882  6104 raft_consensus.cc:493] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:07.871279  6104 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:07.876420  6104 raft_consensus.cc:515] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:07.877991  6104 leader_election.cc:290] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 5 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:07.878717  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:07.878997  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aefd4e81722495ea9fc155e3b924c72" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:07.879288  6030 raft_consensus.cc:3055] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:07.879530  6179 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:07.879616  6030 raft_consensus.cc:740] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:07.880380  6030 consensus_queue.cc:260] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:07.881196  6030 raft_consensus.cc:3060] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:07.883792  6179 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 5.
I20260709 13:16:07.884981  6069 leader_election.cc:304] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:07.885797  6030 raft_consensus.cc:2468] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 5.
I20260709 13:16:07.885831  6306 raft_consensus.cc:2804] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:07.887207  6306 raft_consensus.cc:697] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 5 LEADER]: Becoming Leader. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:07.888067  6306 consensus_queue.cc:237] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:07.894500  5929 catalog_manager.cc:5697] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 reported cstate change: term changed from 4 to 5, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130). New cstate: current_term: 5 leader_uuid: "848c6bc37a7d4316aa29e395d99215e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.162434  6364 raft_consensus.cc:993] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:08.162883  6372 raft_consensus.cc:1081] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Signalling peer 848c6bc37a7d4316aa29e395d99215e3 to start an election
I20260709 13:16:08.164712  6104 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea"
dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
 from {username='slave'} at 127.0.0.1:34128
I20260709 13:16:08.165175  6104 raft_consensus.cc:493] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:08.165474  6104 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:08.170161  6104 raft_consensus.cc:515] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:08.171810  6104 leader_election.cc:290] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 3 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131:42733)
I20260709 13:16:08.172564  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:08.172806  6179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae5f9d1e9824af4a395601857899eea" candidate_uuid: "848c6bc37a7d4316aa29e395d99215e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
I20260709 13:16:08.173097  6030 raft_consensus.cc:3055] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:08.173350  6179 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:08.173434  6030 raft_consensus.cc:740] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:08.174335  6030 consensus_queue.cc:260] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:08.175398  6030 raft_consensus.cc:3060] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:08.178442  6179 raft_consensus.cc:2468] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 3.
I20260709 13:16:08.179495  6069 leader_election.cc:304] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:08.180066  6030 raft_consensus.cc:2468] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 848c6bc37a7d4316aa29e395d99215e3 in term 3.
I20260709 13:16:08.180152  6306 raft_consensus.cc:2804] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:08.180640  6306 raft_consensus.cc:697] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 3 LEADER]: Becoming Leader. State: Replica: 848c6bc37a7d4316aa29e395d99215e3, State: Running, Role: LEADER
I20260709 13:16:08.181519  6306 consensus_queue.cc:237] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:08.187849  5929 catalog_manager.cc:5697] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 reported cstate change: term changed from 2 to 3, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "848c6bc37a7d4316aa29e395d99215e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.201367  6372 raft_consensus.cc:993] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: : Instructing follower 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:08.201782  6372 raft_consensus.cc:1081] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Signalling peer 2ab5a16eb5b04b97b6cfc1361853a74a to start an election
I20260709 13:16:08.203343  6179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb"
dest_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a"
 from {username='slave'} at 127.0.0.1:49992
I20260709 13:16:08.203938  6179 raft_consensus.cc:493] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:08.204310  6179 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:08.209331  6179 raft_consensus.cc:515] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:08.210966  6179 leader_election.cc:290] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 4 election: Requested vote from peers b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797), 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487)
I20260709 13:16:08.211799  6030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b0f3ceeef7f64b8d80f68433a37e629b"
I20260709 13:16:08.211896  6104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d394d62b444ec9b807d39ea09362cb" candidate_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "848c6bc37a7d4316aa29e395d99215e3"
I20260709 13:16:08.212468  6030 raft_consensus.cc:3055] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:08.212508  6104 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:08.212918  6030 raft_consensus.cc:740] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b0f3ceeef7f64b8d80f68433a37e629b, State: Running, Role: LEADER
I20260709 13:16:08.213660  6030 consensus_queue.cc:260] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:08.214856  6030 raft_consensus.cc:3060] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:08.219424  6104 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 4.
I20260709 13:16:08.220600  6145 leader_election.cc:304] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab5a16eb5b04b97b6cfc1361853a74a, 848c6bc37a7d4316aa29e395d99215e3; no voters: 
I20260709 13:16:08.220897  6030 raft_consensus.cc:2468] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab5a16eb5b04b97b6cfc1361853a74a in term 4.
I20260709 13:16:08.221361  6338 raft_consensus.cc:2804] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:08.221920  6338 raft_consensus.cc:697] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 LEADER]: Becoming Leader. State: Replica: 2ab5a16eb5b04b97b6cfc1361853a74a, State: Running, Role: LEADER
I20260709 13:16:08.222710  6338 consensus_queue.cc:237] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } }
I20260709 13:16:08.229408  5929 catalog_manager.cc:5697] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a reported cstate change: term changed from 3 to 4, leader changed from b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129) to 2ab5a16eb5b04b97b6cfc1361853a74a (127.5.72.131). New cstate: current_term: 4 leader_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.405241  6104 raft_consensus.cc:1275] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:08.408411  6338 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:08.427770  6030 raft_consensus.cc:1275] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:08.429338  6368 consensus_queue.cc:1048] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:08.433040  6179 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 5 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:16:08.434909  6306 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:08.445842  6030 raft_consensus.cc:1275] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 5 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:16:08.447405  6375 consensus_queue.cc:1048] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:08.594810  6030 raft_consensus.cc:1275] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:08.594905  6179 raft_consensus.cc:1275] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Refusing update from remote peer 848c6bc37a7d4316aa29e395d99215e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:08.596211  6375 consensus_queue.cc:1048] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:08.596727  6306 consensus_queue.cc:1048] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab5a16eb5b04b97b6cfc1361853a74a" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:08.703305  6030 raft_consensus.cc:1275] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:08.704408  6338 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0f3ceeef7f64b8d80f68433a37e629b" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:08.713282  6104 raft_consensus.cc:1275] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Refusing update from remote peer 2ab5a16eb5b04b97b6cfc1361853a74a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:08.714740  6338 consensus_queue.cc:1048] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "848c6bc37a7d4316aa29e395d99215e3" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 36487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:08.723834  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:08.730679  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
I20260709 13:16:08.760111  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:08.760782  5410 tablet_replica.cc:333] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.761473  5410 raft_consensus.cc:2243] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.761984  5410 raft_consensus.cc:2272] T d0e92aa120df4961922986aa3cb4e5a8 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.764091  5410 tablet_replica.cc:333] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.764628  5410 raft_consensus.cc:2243] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.765025  5410 raft_consensus.cc:2272] T 46b44862ac6c4b79a974ee6f3513430b P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.766776  5410 tablet_replica.cc:333] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.767311  5410 raft_consensus.cc:2243] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.767663  5410 raft_consensus.cc:2272] T b5057b87f1464fc0ac3c6943975bcdd5 P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.769902  5410 tablet_replica.cc:333] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.770375  5410 raft_consensus.cc:2243] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.771183  5410 raft_consensus.cc:2272] T 5bcf0341ab4e4003ba68968236997a97 P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.772984  5410 tablet_replica.cc:333] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.773512  5410 raft_consensus.cc:2243] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.774202  5410 raft_consensus.cc:2272] T 0583fd471a2f48afa3d1a211dd84ff1d P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.775991  5410 tablet_replica.cc:333] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.776461  5410 raft_consensus.cc:2243] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.777173  5410 raft_consensus.cc:2272] T 531ec449e905416c95437a5081a86db4 P b0f3ceeef7f64b8d80f68433a37e629b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.779049  5410 tablet_replica.cc:333] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.779719  5410 raft_consensus.cc:2243] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.780094  5410 raft_consensus.cc:2272] T 85d394d62b444ec9b807d39ea09362cb P b0f3ceeef7f64b8d80f68433a37e629b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.782143  5410 tablet_replica.cc:333] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.782691  5410 raft_consensus.cc:2243] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.783069  5410 raft_consensus.cc:2272] T 0aefd4e81722495ea9fc155e3b924c72 P b0f3ceeef7f64b8d80f68433a37e629b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.785065  5410 tablet_replica.cc:333] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.785501  5410 raft_consensus.cc:2243] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.785837  5410 raft_consensus.cc:2272] T cae5f9d1e9824af4a395601857899eea P b0f3ceeef7f64b8d80f68433a37e629b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.787956  5410 tablet_replica.cc:333] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b: stopping tablet replica
I20260709 13:16:08.788412  5410 raft_consensus.cc:2243] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.789147  5410 raft_consensus.cc:2272] T 34a2bcee8dc14683a3de1e27aac3e496 P b0f3ceeef7f64b8d80f68433a37e629b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.818145  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
I20260709 13:16:08.837903  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
I20260709 13:16:08.860638  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:08.861315  5410 tablet_replica.cc:333] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.861989  5410 raft_consensus.cc:2243] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.862494  5410 raft_consensus.cc:2272] T d0e92aa120df4961922986aa3cb4e5a8 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.864902  5410 tablet_replica.cc:333] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.865525  5410 raft_consensus.cc:2243] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.866303  5410 raft_consensus.cc:2272] T 46b44862ac6c4b79a974ee6f3513430b P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.866860  6071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.72.129:33797: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260709 13:16:08.868238  5410 tablet_replica.cc:333] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.868804  5410 raft_consensus.cc:2243] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.869220  5410 raft_consensus.cc:2272] T 5bcf0341ab4e4003ba68968236997a97 P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.870458  6071 consensus_peers.cc:597] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 -> Peer b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797): Couldn't send request to peer b0f3ceeef7f64b8d80f68433a37e629b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:33797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.871506  5410 tablet_replica.cc:333] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.872025  5410 raft_consensus.cc:2243] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.872467  5410 raft_consensus.cc:2272] T b5057b87f1464fc0ac3c6943975bcdd5 P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.874985  5410 tablet_replica.cc:333] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.875651  5410 raft_consensus.cc:2243] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.876104  5410 raft_consensus.cc:2272] T 0583fd471a2f48afa3d1a211dd84ff1d P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.878329  5410 tablet_replica.cc:333] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
W20260709 13:16:08.878317  6145 consensus_peers.cc:597] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a -> Peer 848c6bc37a7d4316aa29e395d99215e3 (127.5.72.130:36487): Couldn't send request to peer 848c6bc37a7d4316aa29e395d99215e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.879114  5410 raft_consensus.cc:2243] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.879559  5410 raft_consensus.cc:2272] T 531ec449e905416c95437a5081a86db4 P 848c6bc37a7d4316aa29e395d99215e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.881481  5410 tablet_replica.cc:333] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.882009  5410 raft_consensus.cc:2243] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.882416  5410 raft_consensus.cc:2272] T 34a2bcee8dc14683a3de1e27aac3e496 P 848c6bc37a7d4316aa29e395d99215e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.884126  5410 tablet_replica.cc:333] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.884595  5410 raft_consensus.cc:2243] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.885006  5410 raft_consensus.cc:2272] T 85d394d62b444ec9b807d39ea09362cb P 848c6bc37a7d4316aa29e395d99215e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.887550  5410 tablet_replica.cc:333] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.888010  5410 raft_consensus.cc:2243] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.888681  5410 raft_consensus.cc:2272] T 0aefd4e81722495ea9fc155e3b924c72 P 848c6bc37a7d4316aa29e395d99215e3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.890470  5410 tablet_replica.cc:333] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3: stopping tablet replica
I20260709 13:16:08.890973  5410 raft_consensus.cc:2243] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.891636  5410 raft_consensus.cc:2272] T cae5f9d1e9824af4a395601857899eea P 848c6bc37a7d4316aa29e395d99215e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.935645  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
W20260709 13:16:08.948277  6145 consensus_peers.cc:597] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a -> Peer b0f3ceeef7f64b8d80f68433a37e629b (127.5.72.129:33797): Couldn't send request to peer b0f3ceeef7f64b8d80f68433a37e629b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:33797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.955912  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
I20260709 13:16:08.979249  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:08.979987  5410 tablet_replica.cc:333] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.980568  5410 raft_consensus.cc:2243] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.981329  5410 raft_consensus.cc:2272] T d0e92aa120df4961922986aa3cb4e5a8 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.983161  5410 tablet_replica.cc:333] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.983697  5410 raft_consensus.cc:2243] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.984102  5410 raft_consensus.cc:2272] T 46b44862ac6c4b79a974ee6f3513430b P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.985685  5410 tablet_replica.cc:333] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.986133  5410 raft_consensus.cc:2243] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.986884  5410 raft_consensus.cc:2272] T b5057b87f1464fc0ac3c6943975bcdd5 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.988595  5410 tablet_replica.cc:333] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.989113  5410 raft_consensus.cc:2243] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.989504  5410 raft_consensus.cc:2272] T 0583fd471a2f48afa3d1a211dd84ff1d P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.991235  5410 tablet_replica.cc:333] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.991703  5410 raft_consensus.cc:2243] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.992098  5410 raft_consensus.cc:2272] T 5bcf0341ab4e4003ba68968236997a97 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.993691  5410 tablet_replica.cc:333] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.994165  5410 raft_consensus.cc:2243] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.994549  5410 raft_consensus.cc:2272] T 531ec449e905416c95437a5081a86db4 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.996189  5410 tablet_replica.cc:333] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.996646  5410 raft_consensus.cc:2243] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.997030  5410 raft_consensus.cc:2272] T 34a2bcee8dc14683a3de1e27aac3e496 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.998487  5410 tablet_replica.cc:333] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:08.998991  5410 raft_consensus.cc:2243] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.999670  5410 raft_consensus.cc:2272] T 85d394d62b444ec9b807d39ea09362cb P 2ab5a16eb5b04b97b6cfc1361853a74a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.001405  5410 tablet_replica.cc:333] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:09.001896  5410 raft_consensus.cc:2243] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.002298  5410 raft_consensus.cc:2272] T 0aefd4e81722495ea9fc155e3b924c72 P 2ab5a16eb5b04b97b6cfc1361853a74a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.004413  5410 tablet_replica.cc:333] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a: stopping tablet replica
I20260709 13:16:09.004841  5410 raft_consensus.cc:2243] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:09.005228  5410 raft_consensus.cc:2272] T cae5f9d1e9824af4a395601857899eea P 2ab5a16eb5b04b97b6cfc1361853a74a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.032771  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
I20260709 13:16:09.050434  5410 master.cc:562] Master@127.5.72.190:42289 shutting down...
I20260709 13:16:09.070729  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:09.071275  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:09.071604  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 55249a2acd5b4f5abc0fc7ecb58824b8: stopping tablet replica
I20260709 13:16:09.091112  5410 master.cc:584] Master@127.5.72.190:42289 shutdown complete.
2ab5a16eb5b04b97b6cfc1361853a74a  1
848c6bc37a7d4316aa29e395d99215e3  4
b0f3ceeef7f64b8d80f68433a37e629b  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10816 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:16:09.134409  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:46511
I20260709 13:16:09.135779  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:09.141556  6395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:09.141964  6396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:09.143070  6398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:09.144394  5410 server_base.cc:1061] running on GCE node
I20260709 13:16:09.145160  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:09.145419  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:09.145536  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602969145524 us; error 0 us; skew 500 ppm
I20260709 13:16:09.145980  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:09.148298  5410 webserver.cc:533] Webserver started at http://127.5.72.190:42231/ using document root <none> and password file <none>
I20260709 13:16:09.148729  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:09.148880  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:09.149086  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:09.150110  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "10caaf50f65c441b844e3e6055117d18"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.154354  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:09.157503  6403 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.158242  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:09.158526  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "10caaf50f65c441b844e3e6055117d18"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.158845  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:09.190446  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:09.191716  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:09.228668  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:46511
I20260709 13:16:09.228766  6454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:46511 every 8 connection(s)
I20260709 13:16:09.232417  6455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.241640  6455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Bootstrap starting.
I20260709 13:16:09.246349  6455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.250448  6455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: No bootstrap required, opened a new log
I20260709 13:16:09.252542  6455 raft_consensus.cc:359] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10caaf50f65c441b844e3e6055117d18" member_type: VOTER }
I20260709 13:16:09.252919  6455 raft_consensus.cc:385] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:09.253213  6455 raft_consensus.cc:740] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10caaf50f65c441b844e3e6055117d18, State: Initialized, Role: FOLLOWER
I20260709 13:16:09.253801  6455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10caaf50f65c441b844e3e6055117d18" member_type: VOTER }
I20260709 13:16:09.254225  6455 raft_consensus.cc:399] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:09.254468  6455 raft_consensus.cc:493] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:09.254773  6455 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.258951  6455 raft_consensus.cc:515] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10caaf50f65c441b844e3e6055117d18" member_type: VOTER }
I20260709 13:16:09.259487  6455 leader_election.cc:304] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10caaf50f65c441b844e3e6055117d18; no voters: 
I20260709 13:16:09.260664  6455 leader_election.cc:290] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:09.261056  6458 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:09.262406  6458 raft_consensus.cc:697] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 1 LEADER]: Becoming Leader. State: Replica: 10caaf50f65c441b844e3e6055117d18, State: Running, Role: LEADER
I20260709 13:16:09.263232  6458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10caaf50f65c441b844e3e6055117d18" member_type: VOTER }
I20260709 13:16:09.263779  6455 sys_catalog.cc:565] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:09.268708  6460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10caaf50f65c441b844e3e6055117d18. Latest consensus state: current_term: 1 leader_uuid: "10caaf50f65c441b844e3e6055117d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10caaf50f65c441b844e3e6055117d18" member_type: VOTER } }
I20260709 13:16:09.268895  6459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10caaf50f65c441b844e3e6055117d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10caaf50f65c441b844e3e6055117d18" member_type: VOTER } }
I20260709 13:16:09.269462  6460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:09.269658  6459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:09.271811  6466 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:09.280258  6466 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:09.282725  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:09.298382  6466 catalog_manager.cc:1383] Generated new cluster ID: 480bb63290424a29bfdb32601d2608b6
I20260709 13:16:09.299119  6466 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:09.317255  6466 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:09.318579  6466 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:09.333181  6466 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Generated new TSK 0
I20260709 13:16:09.333783  6466 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:09.350580  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:09.356477  6476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:09.357720  6477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:09.359445  5410 server_base.cc:1061] running on GCE node
W20260709 13:16:09.359834  6479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:09.360731  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:09.360965  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:09.361142  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602969361125 us; error 0 us; skew 500 ppm
I20260709 13:16:09.361855  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:09.364166  5410 webserver.cc:533] Webserver started at http://127.5.72.129:41455/ using document root <none> and password file <none>
I20260709 13:16:09.364696  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:09.364924  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:09.365197  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:09.366304  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "921c423406da4f289884afca51ca198f"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.370762  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:16:09.373920  6484 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.374805  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:16:09.375149  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "921c423406da4f289884afca51ca198f"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.375406  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:09.402004  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:09.403270  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:09.404665  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:09.406989  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:09.407163  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.407469  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:09.407641  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.447285  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:34121
I20260709 13:16:09.447397  6546 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:34121 every 8 connection(s)
I20260709 13:16:09.452165  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:09.459806  6551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:09.461330  6552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:09.464150  6554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:09.464510  5410 server_base.cc:1061] running on GCE node
I20260709 13:16:09.465361  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:09.465538  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:09.465657  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602969465647 us; error 0 us; skew 500 ppm
I20260709 13:16:09.466090  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:09.468642  5410 webserver.cc:533] Webserver started at http://127.5.72.130:33463/ using document root <none> and password file <none>
I20260709 13:16:09.469149  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:09.469375  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:09.469652  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:09.471206  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "3ceee54e49d041be9f7f48a2052709c5"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.471225  6547 heartbeater.cc:344] Connected to a master server at 127.5.72.190:46511
I20260709 13:16:09.471711  6547 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:09.472612  6547 heartbeater.cc:507] Master 127.5.72.190:46511 requested a full tablet report, sending...
I20260709 13:16:09.475071  6420 ts_manager.cc:194] Registered new tserver with Master: 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:09.476963  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:16:09.477638  6420 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35464
I20260709 13:16:09.481091  6559 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.481828  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:09.482131  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "3ceee54e49d041be9f7f48a2052709c5"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.482425  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:09.498395  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:09.499609  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:09.501037  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:09.503455  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:09.503679  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.503924  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:09.504092  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.558446  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:37977
I20260709 13:16:09.558661  6621 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:37977 every 8 connection(s)
I20260709 13:16:09.563563  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:09.572434  6625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:09.574084  6626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:09.577833  6629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:09.582142  5410 server_base.cc:1061] running on GCE node
I20260709 13:16:09.582238  6622 heartbeater.cc:344] Connected to a master server at 127.5.72.190:46511
I20260709 13:16:09.582793  6622 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:09.583142  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:09.583360  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:09.583537  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602969583505 us; error 0 us; skew 500 ppm
I20260709 13:16:09.583606  6622 heartbeater.cc:507] Master 127.5.72.190:46511 requested a full tablet report, sending...
I20260709 13:16:09.584153  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:09.585956  6420 ts_manager.cc:194] Registered new tserver with Master: 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:09.586941  5410 webserver.cc:533] Webserver started at http://127.5.72.131:42975/ using document root <none> and password file <none>
I20260709 13:16:09.587564  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:09.587800  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:09.587872  6420 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35466
I20260709 13:16:09.588186  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:09.589648  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "53fee9894731461a821a131be24dbbca"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.594049  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:16:09.597167  6633 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.597934  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:09.598220  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "53fee9894731461a821a131be24dbbca"
format_stamp: "Formatted at 2026-07-09 13:16:09 on dist-test-slave-n5kz"
I20260709 13:16:09.598488  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:09.628695  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:09.629925  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:09.631455  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:09.633641  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:09.633854  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.634114  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:09.634270  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:09.674404  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:38019
I20260709 13:16:09.674496  6695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:38019 every 8 connection(s)
I20260709 13:16:09.688303  6696 heartbeater.cc:344] Connected to a master server at 127.5.72.190:46511
I20260709 13:16:09.688716  6696 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:09.689463  6696 heartbeater.cc:507] Master 127.5.72.190:46511 requested a full tablet report, sending...
I20260709 13:16:09.691445  6420 ts_manager.cc:194] Registered new tserver with Master: 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:09.692317  5410 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014533518s
I20260709 13:16:09.692741  6420 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35476
I20260709 13:16:09.717238  6420 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35482:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:09.723345  6420 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:09.923121  6587 tablet_service.cc:1511] Processing CreateTablet for tablet f4f0dad199224b8182ae036d97281b72 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:09.925117  6587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f0dad199224b8182ae036d97281b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.928820  6661 tablet_service.cc:1511] Processing CreateTablet for tablet f4f0dad199224b8182ae036d97281b72 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:09.929776  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:09.931689  6586 tablet_service.cc:1511] Processing CreateTablet for tablet 2c0a546cb4d749bc83ffa9cbd0f40297 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:09.933238  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c0a546cb4d749bc83ffa9cbd0f40297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.932955  6660 tablet_service.cc:1511] Processing CreateTablet for tablet 2c0a546cb4d749bc83ffa9cbd0f40297 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:09.934273  6653 tablet_service.cc:1511] Processing CreateTablet for tablet 1a2845fc76124deab8a20512c6112df7 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:09.935173  6659 tablet_service.cc:1511] Processing CreateTablet for tablet d83f8b945cd641db90647da0461be4a3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:09.936000  6657 tablet_service.cc:1511] Processing CreateTablet for tablet 835ac063488a40878ccb02b662d3898b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:09.937356  6654 tablet_service.cc:1511] Processing CreateTablet for tablet b2c67ec2db9643088bc65fe9d83aa732 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:09.938292  6652 tablet_service.cc:1511] Processing CreateTablet for tablet 40b7cf612223418ea8fe2d636986bb3f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:09.939601  6656 tablet_service.cc:1511] Processing CreateTablet for tablet bab5f01bd4774a168d3c01940a615cce (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:09.941471  6655 tablet_service.cc:1511] Processing CreateTablet for tablet 004d58f566fd4b6d954bc425675471fa (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:09.930238  6661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f0dad199224b8182ae036d97281b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.943173  6654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c67ec2db9643088bc65fe9d83aa732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.943775  6656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab5f01bd4774a168d3c01940a615cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.945290  6652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40b7cf612223418ea8fe2d636986bb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.947497  6659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8b945cd641db90647da0461be4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.949532  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835ac063488a40878ccb02b662d3898b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.950917  6660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c0a546cb4d749bc83ffa9cbd0f40297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.955020  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.957008  6711 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:09.959044  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a2845fc76124deab8a20512c6112df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.962977  6655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 004d58f566fd4b6d954bc425675471fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.965025  6711 tablet_bootstrap.cc:654] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.974589  6586 tablet_service.cc:1511] Processing CreateTablet for tablet d83f8b945cd641db90647da0461be4a3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:09.976603  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8b945cd641db90647da0461be4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.977826  6578 tablet_service.cc:1511] Processing CreateTablet for tablet f9f5da6098b64b229cd872c6346351f3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:09.979172  6578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f5da6098b64b229cd872c6346351f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.979007  6584 tablet_service.cc:1511] Processing CreateTablet for tablet 835ac063488a40878ccb02b662d3898b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:09.981129  6583 tablet_service.cc:1511] Processing CreateTablet for tablet bab5f01bd4774a168d3c01940a615cce (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:09.982434  6583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab5f01bd4774a168d3c01940a615cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.977823  6585 tablet_service.cc:1511] Processing CreateTablet for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:09.983807  6582 tablet_service.cc:1511] Processing CreateTablet for tablet 004d58f566fd4b6d954bc425675471fa (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:09.985471  6581 tablet_service.cc:1511] Processing CreateTablet for tablet b2c67ec2db9643088bc65fe9d83aa732 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:09.994774  6585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.995430  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 004d58f566fd4b6d954bc425675471fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:09.987151  6580 tablet_service.cc:1511] Processing CreateTablet for tablet 1a2845fc76124deab8a20512c6112df7 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:09.988755  6579 tablet_service.cc:1511] Processing CreateTablet for tablet 40b7cf612223418ea8fe2d636986bb3f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:10.000424  6580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a2845fc76124deab8a20512c6112df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.003196  6579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40b7cf612223418ea8fe2d636986bb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.007162  6581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c67ec2db9643088bc65fe9d83aa732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.010979  6584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835ac063488a40878ccb02b662d3898b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.024901  6657 tablet_service.cc:1511] Processing CreateTablet for tablet ce959bc5e0b4481abad1a3bef3085967 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:10.026849  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce959bc5e0b4481abad1a3bef3085967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.028662  6506 tablet_service.cc:1511] Processing CreateTablet for tablet 004d58f566fd4b6d954bc425675471fa (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:10.032253  6506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 004d58f566fd4b6d954bc425675471fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.032900  6512 tablet_service.cc:1511] Processing CreateTablet for tablet f4f0dad199224b8182ae036d97281b72 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:10.034212  6512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f0dad199224b8182ae036d97281b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.034031  6508 tablet_service.cc:1511] Processing CreateTablet for tablet 835ac063488a40878ccb02b662d3898b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:10.041230  6503 tablet_service.cc:1511] Processing CreateTablet for tablet 40b7cf612223418ea8fe2d636986bb3f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:10.046209  6510 tablet_service.cc:1511] Processing CreateTablet for tablet d83f8b945cd641db90647da0461be4a3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:10.046836  6659 tablet_service.cc:1511] Processing CreateTablet for tablet ffa871a3f5334bd49b9c75539f218332 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:10.049083  6659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffa871a3f5334bd49b9c75539f218332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.049338  6507 tablet_service.cc:1511] Processing CreateTablet for tablet bab5f01bd4774a168d3c01940a615cce (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:10.052094  6509 tablet_service.cc:1511] Processing CreateTablet for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:10.032902  6511 tablet_service.cc:1511] Processing CreateTablet for tablet 2c0a546cb4d749bc83ffa9cbd0f40297 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:10.039045  6505 tablet_service.cc:1511] Processing CreateTablet for tablet b2c67ec2db9643088bc65fe9d83aa732 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:10.047076  6504 tablet_service.cc:1511] Processing CreateTablet for tablet 1a2845fc76124deab8a20512c6112df7 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:10.056224  6508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835ac063488a40878ccb02b662d3898b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.057224  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 85490b85caca40d19a25af108b1da46a (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:10.057801  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40b7cf612223418ea8fe2d636986bb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.058214  6582 tablet_service.cc:1511] Processing CreateTablet for tablet ce959bc5e0b4481abad1a3bef3085967 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:10.058934  6511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c0a546cb4d749bc83ffa9cbd0f40297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.059469  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce959bc5e0b4481abad1a3bef3085967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.042119  6587 tablet_service.cc:1511] Processing CreateTablet for tablet ffa871a3f5334bd49b9c75539f218332 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:10.066859  6505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c67ec2db9643088bc65fe9d83aa732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.067498  6510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8b945cd641db90647da0461be4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.071022  6509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.072767  6507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab5f01bd4774a168d3c01940a615cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.074923  6504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a2845fc76124deab8a20512c6112df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.079384  6581 tablet_service.cc:1511] Processing CreateTablet for tablet 85490b85caca40d19a25af108b1da46a (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:10.080641  6581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85490b85caca40d19a25af108b1da46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.081506  6656 tablet_service.cc:1511] Processing CreateTablet for tablet f9f5da6098b64b229cd872c6346351f3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:10.091398  6713 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:10.095691  6586 tablet_service.cc:1511] Processing CreateTablet for tablet ff27e205e84e4edf946bc02c573a2c25 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:10.100836  6660 tablet_service.cc:1511] Processing CreateTablet for tablet ff27e205e84e4edf946bc02c573a2c25 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:10.121289  6503 tablet_service.cc:1511] Processing CreateTablet for tablet f9f5da6098b64b229cd872c6346351f3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:10.122756  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f5da6098b64b229cd872c6346351f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.066978  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85490b85caca40d19a25af108b1da46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.127596  6509 tablet_service.cc:1511] Processing CreateTablet for tablet ce959bc5e0b4481abad1a3bef3085967 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:10.142827  6509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce959bc5e0b4481abad1a3bef3085967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.143965  6711 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:10.144435  6711 ts_tablet_manager.cc:1403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.188s	user 0.018s	sys 0.000s
I20260709 13:16:10.131335  6587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffa871a3f5334bd49b9c75539f218332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.132094  6653 tablet_service.cc:1511] Processing CreateTablet for tablet f8a48eeccdac45838ef8456e5e9332b5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:10.133698  6661 tablet_service.cc:1511] Processing CreateTablet for tablet 17e76729a2fc4273abedd8be9331a882 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:10.134817  6657 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb598c0b9b54f788ae6eefe2894e3a5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:10.135634  6654 tablet_service.cc:1511] Processing CreateTablet for tablet 627436404a154d80a882f4900dff2948 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:10.130429  6655 tablet_service.cc:1511] Processing CreateTablet for tablet 91d990297ed0410d894aa69d60271fe7 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:10.147081  6659 tablet_service.cc:1511] Processing CreateTablet for tablet c84605b85a2f4c4b839274e0724ef52b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:10.152631  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff27e205e84e4edf946bc02c573a2c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.150465  6579 tablet_service.cc:1511] Processing CreateTablet for tablet f8a48eeccdac45838ef8456e5e9332b5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:10.150967  6656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f5da6098b64b229cd872c6346351f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.187273  6660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff27e205e84e4edf946bc02c573a2c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.169348  6580 tablet_service.cc:1511] Processing CreateTablet for tablet 17e76729a2fc4273abedd8be9331a882 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:10.133739  6578 tablet_service.cc:1511] Processing CreateTablet for tablet 91d990297ed0410d894aa69d60271fe7 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:10.191699  6578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d990297ed0410d894aa69d60271fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.192245  6654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627436404a154d80a882f4900dff2948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.198906  6579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a48eeccdac45838ef8456e5e9332b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.201395  6711 raft_consensus.cc:359] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.207931  6655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d990297ed0410d894aa69d60271fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.208598  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a48eeccdac45838ef8456e5e9332b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.213300  6661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e76729a2fc4273abedd8be9331a882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.213342  6510 tablet_service.cc:1511] Processing CreateTablet for tablet ffa871a3f5334bd49b9c75539f218332 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:10.214735  6510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffa871a3f5334bd49b9c75539f218332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.210089  6580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e76729a2fc4273abedd8be9331a882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.202157  6711 raft_consensus.cc:385] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.194486  6652 tablet_service.cc:1511] Processing CreateTablet for tablet d598d50f158e42a6b8d5e297f271951d (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:10.219084  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb598c0b9b54f788ae6eefe2894e3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.217585  6711 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.220453  6711 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.221921  6652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d598d50f158e42a6b8d5e297f271951d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.223027  6659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84605b85a2f4c4b839274e0724ef52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.227833  6507 tablet_service.cc:1511] Processing CreateTablet for tablet ff27e205e84e4edf946bc02c573a2c25 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:10.229226  6507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff27e205e84e4edf946bc02c573a2c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.234414  6713 tablet_bootstrap.cc:654] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.240001  6583 tablet_service.cc:1511] Processing CreateTablet for tablet 627436404a154d80a882f4900dff2948 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:10.240401  6584 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb598c0b9b54f788ae6eefe2894e3a5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:10.245349  6583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627436404a154d80a882f4900dff2948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.247205  6714 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:10.256472  6585 tablet_service.cc:1511] Processing CreateTablet for tablet c84605b85a2f4c4b839274e0724ef52b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:10.257853  6585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84605b85a2f4c4b839274e0724ef52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.268323  6711 ts_tablet_manager.cc:1434] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.124s	user 0.007s	sys 0.008s
I20260709 13:16:10.275118  6509 tablet_service.cc:1511] Processing CreateTablet for tablet 91d990297ed0410d894aa69d60271fe7 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
W20260709 13:16:10.277153  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:10.278219  6509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d990297ed0410d894aa69d60271fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.283962  6584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb598c0b9b54f788ae6eefe2894e3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.289819  6622 heartbeater.cc:499] Master 127.5.72.190:46511 was elected leader, sending a full tablet report...
I20260709 13:16:10.296146  6508 tablet_service.cc:1511] Processing CreateTablet for tablet 85490b85caca40d19a25af108b1da46a (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:10.298524  6508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85490b85caca40d19a25af108b1da46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.317654  6512 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb598c0b9b54f788ae6eefe2894e3a5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:10.318501  6656 tablet_service.cc:1511] Processing CreateTablet for tablet 4c870451bc2945e8b0a88e8fedf73ce6 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:10.303879  6711 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:10.323330  6512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb598c0b9b54f788ae6eefe2894e3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.332286  6504 tablet_service.cc:1511] Processing CreateTablet for tablet f8a48eeccdac45838ef8456e5e9332b5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:10.333698  6504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a48eeccdac45838ef8456e5e9332b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.334808  6656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c870451bc2945e8b0a88e8fedf73ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.337596  6714 tablet_bootstrap.cc:654] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.349574  6505 tablet_service.cc:1511] Processing CreateTablet for tablet 627436404a154d80a882f4900dff2948 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:10.350365  6711 tablet_bootstrap.cc:654] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.340914  6582 tablet_service.cc:1511] Processing CreateTablet for tablet d598d50f158e42a6b8d5e297f271951d (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:10.351799  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d598d50f158e42a6b8d5e297f271951d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.355252  6510 tablet_service.cc:1511] Processing CreateTablet for tablet 17e76729a2fc4273abedd8be9331a882 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:10.355901  6506 tablet_service.cc:1511] Processing CreateTablet for tablet c84605b85a2f4c4b839274e0724ef52b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:10.358714  6580 tablet_service.cc:1511] Processing CreateTablet for tablet 4c870451bc2945e8b0a88e8fedf73ce6 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:10.359966  6580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c870451bc2945e8b0a88e8fedf73ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.359644  6656 tablet_service.cc:1511] Processing CreateTablet for tablet 17b598f21a79476cabdf3f5191b4f9a8 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:10.360878  6512 tablet_service.cc:1511] Processing CreateTablet for tablet d598d50f158e42a6b8d5e297f271951d (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:10.362386  6656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b598f21a79476cabdf3f5191b4f9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.367990  6586 tablet_service.cc:1511] Processing CreateTablet for tablet 6be297bb363c43c99d04cccfc55bec8f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:10.369529  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6be297bb363c43c99d04cccfc55bec8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.366312  6506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84605b85a2f4c4b839274e0724ef52b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.372499  6658 tablet_service.cc:1511] Processing CreateTablet for tablet c0e818d5c3014869b85a1d5b725c58cf (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:10.383556  6510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e76729a2fc4273abedd8be9331a882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.394577  6714 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:10.395160  6714 ts_tablet_manager.cc:1403] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.148s	user 0.005s	sys 0.007s
I20260709 13:16:10.398103  6711 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:10.398690  6711 ts_tablet_manager.cc:1403] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.100s	user 0.016s	sys 0.000s
I20260709 13:16:10.398484  6579 tablet_service.cc:1511] Processing CreateTablet for tablet 8885ae897ec84eca940e2aa65f7311b5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:10.399749  6579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8885ae897ec84eca940e2aa65f7311b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.399441  6578 tablet_service.cc:1511] Processing CreateTablet for tablet c0e818d5c3014869b85a1d5b725c58cf (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:10.400748  6578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e818d5c3014869b85a1d5b725c58cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.401777  6711 raft_consensus.cc:359] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.402580  6711 raft_consensus.cc:385] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.402906  6711 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.404031  6711 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.408110  6711 ts_tablet_manager.cc:1434] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:16:10.408912  6505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627436404a154d80a882f4900dff2948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.411517  6661 tablet_service.cc:1511] Processing CreateTablet for tablet 6be297bb363c43c99d04cccfc55bec8f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:10.413622  6713 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:10.414163  6713 ts_tablet_manager.cc:1403] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.323s	user 0.020s	sys 0.017s
I20260709 13:16:10.414733  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e818d5c3014869b85a1d5b725c58cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.417539  6713 raft_consensus.cc:359] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.422879  6713 raft_consensus.cc:385] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.429584  6713 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.398213  6714 raft_consensus.cc:359] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.431716  6711 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:10.436425  6508 tablet_service.cc:1511] Processing CreateTablet for tablet 17b598f21a79476cabdf3f5191b4f9a8 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:10.436715  6581 tablet_service.cc:1511] Processing CreateTablet for tablet 60fb01da2cc5490ebbbca7ec57b64a20 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:10.438673  6581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fb01da2cc5490ebbbca7ec57b64a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.431020  6512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d598d50f158e42a6b8d5e297f271951d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.443230  6653 tablet_service.cc:1511] Processing CreateTablet for tablet 8885ae897ec84eca940e2aa65f7311b5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:10.443799  6711 tablet_bootstrap.cc:654] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.433571  6714 raft_consensus.cc:385] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.444868  6587 tablet_service.cc:1511] Processing CreateTablet for tablet 17b598f21a79476cabdf3f5191b4f9a8 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:10.445379  6714 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.446029  6508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b598f21a79476cabdf3f5191b4f9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.446308  6587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b598f21a79476cabdf3f5191b4f9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.446437  6714 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.448706  6657 tablet_service.cc:1511] Processing CreateTablet for tablet 60fb01da2cc5490ebbbca7ec57b64a20 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:10.453645  6661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6be297bb363c43c99d04cccfc55bec8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.459286  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8885ae897ec84eca940e2aa65f7311b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.462000  6713 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.465624  6511 tablet_service.cc:1511] Processing CreateTablet for tablet 4c870451bc2945e8b0a88e8fedf73ce6 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:10.468918  6511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c870451bc2945e8b0a88e8fedf73ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.469633  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fb01da2cc5490ebbbca7ec57b64a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.480579  6547 heartbeater.cc:499] Master 127.5.72.190:46511 was elected leader, sending a full tablet report...
W20260709 13:16:10.487192  6407 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 51394.9us
I20260709 13:16:10.502365  6713 ts_tablet_manager.cc:1434] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.088s	user 0.008s	sys 0.004s
I20260709 13:16:10.504637  6582 tablet_service.cc:1511] Processing CreateTablet for tablet fd28ad0fdaad48d586176f4a8e3f1aa3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:10.505966  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd28ad0fdaad48d586176f4a8e3f1aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.511598  6714 ts_tablet_manager.cc:1434] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.116s	user 0.007s	sys 0.006s
I20260709 13:16:10.512615  6714 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:10.527104  6503 tablet_service.cc:1511] Processing CreateTablet for tablet 6be297bb363c43c99d04cccfc55bec8f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:10.528924  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6be297bb363c43c99d04cccfc55bec8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.535138  6696 heartbeater.cc:499] Master 127.5.72.190:46511 was elected leader, sending a full tablet report...
I20260709 13:16:10.536981  6659 tablet_service.cc:1511] Processing CreateTablet for tablet fd28ad0fdaad48d586176f4a8e3f1aa3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:10.538285  6659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd28ad0fdaad48d586176f4a8e3f1aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.540032  6715 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.540477  6715 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.543079  6715 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:10.546588  6713 tablet_bootstrap.cc:492] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:10.568881  6715 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.570720  6715 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.571126  6654 tablet_service.cc:1511] Processing CreateTablet for tablet 0d4a8b90b128485fa69bc10112de24e9 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:10.572376  6654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4a8b90b128485fa69bc10112de24e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.572693  6715 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:10.574720  6713 tablet_bootstrap.cc:654] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.588979  6711 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:10.592554  6711 ts_tablet_manager.cc:1403] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.166s	user 0.018s	sys 0.032s
I20260709 13:16:10.605307  6711 raft_consensus.cc:359] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.610870  6711 raft_consensus.cc:385] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.611467  6711 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.608295  6505 tablet_service.cc:1511] Processing CreateTablet for tablet c0e818d5c3014869b85a1d5b725c58cf (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:10.608532  6507 tablet_service.cc:1511] Processing CreateTablet for tablet 8885ae897ec84eca940e2aa65f7311b5 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:10.602005  6660 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9d0fb5a0864567ac4cc6ee615911b6 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:10.616312  6660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9d0fb5a0864567ac4cc6ee615911b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.623636  6582 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9d0fb5a0864567ac4cc6ee615911b6 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:10.625044  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9d0fb5a0864567ac4cc6ee615911b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.627022  6586 tablet_service.cc:1511] Processing CreateTablet for tablet 0d4a8b90b128485fa69bc10112de24e9 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:10.628355  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4a8b90b128485fa69bc10112de24e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.643968  6711 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.652320  6505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e818d5c3014869b85a1d5b725c58cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.652887  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:10.654843  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:10.655259  6561 leader_election.cc:343] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.655287  6711 ts_tablet_manager.cc:1434] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:16:10.656430  6714 tablet_bootstrap.cc:654] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:10.656494  6561 leader_election.cc:343] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.657028  6711 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:10.660244  6721 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.660748  6721 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.663849  6508 tablet_service.cc:1511] Processing CreateTablet for tablet 60fb01da2cc5490ebbbca7ec57b64a20 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:10.665423  6508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fb01da2cc5490ebbbca7ec57b64a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.667251  6503 tablet_service.cc:1511] Processing CreateTablet for tablet 0d4a8b90b128485fa69bc10112de24e9 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:10.668540  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d4a8b90b128485fa69bc10112de24e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.669543  6584 tablet_service.cc:1511] Processing CreateTablet for tablet 943ce377e70a422a8376b294da2840fa (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:10.675843  6584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943ce377e70a422a8376b294da2840fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.675984  6510 tablet_service.cc:1511] Processing CreateTablet for tablet 943ce377e70a422a8376b294da2840fa (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:10.677590  6510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943ce377e70a422a8376b294da2840fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.681373  6711 tablet_bootstrap.cc:654] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.687036  6721 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:10.691685  6511 tablet_service.cc:1511] Processing CreateTablet for tablet fd28ad0fdaad48d586176f4a8e3f1aa3 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:10.694057  6509 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9d0fb5a0864567ac4cc6ee615911b6 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:10.695032  6653 tablet_service.cc:1511] Processing CreateTablet for tablet 943ce377e70a422a8376b294da2840fa (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:10.696338  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943ce377e70a422a8376b294da2840fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.699689  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:10.700029  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:10.704193  6563 leader_election.cc:343] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.704806  6563 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:10.705868  6715 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:10.716954  6713 tablet_bootstrap.cc:492] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:10.717510  6713 ts_tablet_manager.cc:1403] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.171s	user 0.025s	sys 0.010s
W20260709 13:16:10.711069  6563 leader_election.cc:343] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.726385  6563 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:10.729637  6715 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:10.729445  6713 raft_consensus.cc:359] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.730204  6713 raft_consensus.cc:385] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.730536  6713 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.735872  6507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8885ae897ec84eca940e2aa65f7311b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.742172  6509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9d0fb5a0864567ac4cc6ee615911b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.742913  6511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd28ad0fdaad48d586176f4a8e3f1aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.748585  6506 tablet_service.cc:1511] Processing CreateTablet for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:10.750393  6506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.761544  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:10.767954  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.775210  6587 tablet_service.cc:1511] Processing CreateTablet for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:10.776554  6587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.772972  6713 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.781220  6656 tablet_service.cc:1511] Processing CreateTablet for tablet e1a87665f0ba4296b0ba819fcb712842 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:10.782526  6656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a87665f0ba4296b0ba819fcb712842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.784034  6512 tablet_service.cc:1511] Processing CreateTablet for tablet e1a87665f0ba4296b0ba819fcb712842 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:10.786986  6512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a87665f0ba4296b0ba819fcb712842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.789861  6719 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.791077  6719 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.811292  6719 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:10.831959  6714 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:10.831610  6503 tablet_service.cc:1511] Processing CreateTablet for tablet f67b820295ac47a4b54e21400f8b4201 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:10.832451  6714 ts_tablet_manager.cc:1403] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.320s	user 0.017s	sys 0.002s
I20260709 13:16:10.833423  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f67b820295ac47a4b54e21400f8b4201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.835587  6714 raft_consensus.cc:359] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.836381  6714 raft_consensus.cc:385] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.836728  6714 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.837440  6714 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.844063  6713 ts_tablet_manager.cc:1434] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.126s	user 0.002s	sys 0.009s
I20260709 13:16:10.845495  6713 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:10.849980  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:10.850767  6521 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:10.852130  6637 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:10.853088  6721 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:10.853442  6721 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.853772  6721 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:10.859377  6715 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.859917  6715 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.867215  6715 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:10.873417  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:10.874351  6660 tablet_service.cc:1511] Processing CreateTablet for tablet f67b820295ac47a4b54e21400f8b4201 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:10.875653  6660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f67b820295ac47a4b54e21400f8b4201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.877269  6509 tablet_service.cc:1511] Processing CreateTablet for tablet a758d1835d6148668ab495504660d72a (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:10.878539  6509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a758d1835d6148668ab495504660d72a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.880085  6654 tablet_service.cc:1511] Processing CreateTablet for tablet a758d1835d6148668ab495504660d72a (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:10.881376  6654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a758d1835d6148668ab495504660d72a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:10.883026  6561 leader_election.cc:343] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.883106  6582 tablet_service.cc:1511] Processing CreateTablet for tablet e1a87665f0ba4296b0ba819fcb712842 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:10.884498  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a87665f0ba4296b0ba819fcb712842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.884702  6504 tablet_service.cc:1511] Processing CreateTablet for tablet b75fb632ebbc415092b08b38d0835144 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:10.885928  6504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75fb632ebbc415092b08b38d0835144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.896497  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:10.904671  6713 tablet_bootstrap.cc:654] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.905555  6578 tablet_service.cc:1511] Processing CreateTablet for tablet a758d1835d6148668ab495504660d72a (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:10.900039  6657 tablet_service.cc:1511] Processing CreateTablet for tablet b75fb632ebbc415092b08b38d0835144 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:10.906915  6578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a758d1835d6148668ab495504660d72a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.910719  6721 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.911242  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75fb632ebbc415092b08b38d0835144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.912197  6711 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:10.912688  6711 ts_tablet_manager.cc:1403] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.256s	user 0.017s	sys 0.021s
I20260709 13:16:10.915719  6711 raft_consensus.cc:359] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.916457  6711 raft_consensus.cc:385] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.916747  6711 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.917575  6711 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:10.919736  6711 ts_tablet_manager.cc:1434] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:16:10.923655  6584 tablet_service.cc:1511] Processing CreateTablet for tablet f67b820295ac47a4b54e21400f8b4201 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:10.932597  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:10.933235  6521 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:10.934571  6503 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:10.935103  6711 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:10.941138  6721 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:10.943955  6521 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:10.945089  6637 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:10.946004  6721 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:10.947448  6714 ts_tablet_manager.cc:1434] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.115s	user 0.006s	sys 0.002s
I20260709 13:16:10.952195  6586 tablet_service.cc:1511] Processing CreateTablet for tablet b75fb632ebbc415092b08b38d0835144 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:10.954407  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:10.955086  6596 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:10.948419  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:10.960968  6584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f67b820295ac47a4b54e21400f8b4201. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:10.969790  6563 leader_election.cc:343] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.971298  6563 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:10.970953  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75fb632ebbc415092b08b38d0835144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.983799  6596 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:10.991797  6715 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:10.986632  6711 tablet_bootstrap.cc:654] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.991236  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:10.987161  6714 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:10.997462  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:10.990972  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:11.001391  6578 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:11.004181  6578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.004637  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.006783  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.010576  6596 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:11.024547  6656 tablet_service.cc:1511] Processing CreateTablet for tablet c9bbf06685e047fdaa1b6bc881dc7a93 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:11.025903  6656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9bbf06685e047fdaa1b6bc881dc7a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.028134  6587 tablet_service.cc:1511] Processing CreateTablet for tablet c9bbf06685e047fdaa1b6bc881dc7a93 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:11.029498  6587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9bbf06685e047fdaa1b6bc881dc7a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.029877  6714 tablet_bootstrap.cc:654] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.033540  6582 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc1d5a094e64eadb9c93f5e81f30fbc (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:11.034894  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc1d5a094e64eadb9c93f5e81f30fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.035813  6506 tablet_service.cc:1511] Processing CreateTablet for tablet 389b5012cd8548a099f7dec661e47e4f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:11.036229  6657 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc1d5a094e64eadb9c93f5e81f30fbc (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:11.037113  6506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389b5012cd8548a099f7dec661e47e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.037401  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc1d5a094e64eadb9c93f5e81f30fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.050875  6580 tablet_service.cc:1511] Processing CreateTablet for tablet 389b5012cd8548a099f7dec661e47e4f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:11.052187  6580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389b5012cd8548a099f7dec661e47e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.007956  6721 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:11.058880  6719 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.059413  6719 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.062566  6713 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:11.063072  6713 ts_tablet_manager.cc:1403] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.218s	user 0.019s	sys 0.018s
I20260709 13:16:11.065104  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 389b5012cd8548a099f7dec661e47e4f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:11.065989  6713 raft_consensus.cc:359] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.070066  6713 raft_consensus.cc:385] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.070274  6713 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.066565  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389b5012cd8548a099f7dec661e47e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.068375  6719 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.069128  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.069725  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:11.072582  6488 leader_election.cc:343] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.072957  6506 tablet_service.cc:1511] Processing CreateTablet for tablet 938e1826b2ce4babab3a70e5219411a4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 13:16:11.073274  6486 leader_election.cc:343] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.074270  6506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938e1826b2ce4babab3a70e5219411a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.074163  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:11.076700  6509 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc1d5a094e64eadb9c93f5e81f30fbc (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:11.078183  6512 tablet_service.cc:1511] Processing CreateTablet for tablet 23ba4e226865438d85ee0c061786f3db (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:11.081817  6512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23ba4e226865438d85ee0c061786f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.083578  6582 tablet_service.cc:1511] Processing CreateTablet for tablet 938e1826b2ce4babab3a70e5219411a4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:11.086239  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938e1826b2ce4babab3a70e5219411a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.079362  6719 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:11.080509  6654 tablet_service.cc:1511] Processing CreateTablet for tablet 938e1826b2ce4babab3a70e5219411a4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:11.070961  6713 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.094549  6509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc1d5a094e64eadb9c93f5e81f30fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.085886  6721 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.095106  6711 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.095597  6711 ts_tablet_manager.cc:1403] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.161s	user 0.020s	sys 0.015s
I20260709 13:16:11.098994  6657 tablet_service.cc:1511] Processing CreateTablet for tablet 23ba4e226865438d85ee0c061786f3db (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:11.100142  6586 tablet_service.cc:1511] Processing CreateTablet for tablet 23ba4e226865438d85ee0c061786f3db (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:11.100580  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23ba4e226865438d85ee0c061786f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.101434  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23ba4e226865438d85ee0c061786f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.102157  6507 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb5773be2294301bdb1bbedf5765666 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:11.098776  6711 raft_consensus.cc:359] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.103157  6711 raft_consensus.cc:385] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.103543  6507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb5773be2294301bdb1bbedf5765666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.104331  6508 tablet_service.cc:1511] Processing CreateTablet for tablet c9bbf06685e047fdaa1b6bc881dc7a93 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:11.105504  6508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9bbf06685e047fdaa1b6bc881dc7a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.111720  6713 ts_tablet_manager.cc:1434] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.048s	user 0.002s	sys 0.005s
I20260709 13:16:11.103468  6711 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.112774  6711 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.115187  6654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938e1826b2ce4babab3a70e5219411a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.120841  6713 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:11.123660  6585 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb5773be2294301bdb1bbedf5765666 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:11.124987  6585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb5773be2294301bdb1bbedf5765666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.133366  6656 tablet_service.cc:1511] Processing CreateTablet for tablet 6eb5773be2294301bdb1bbedf5765666 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:11.133049  6420 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.134850  6656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb5773be2294301bdb1bbedf5765666. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:11.141464  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 69048.2us
I20260709 13:16:11.157858  6582 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3ba394f290437b90045c4a98fb75ca (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:11.159191  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ba394f290437b90045c4a98fb75ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.159437  6504 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3ba394f290437b90045c4a98fb75ca (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:11.160113  6660 tablet_service.cc:1511] Processing CreateTablet for tablet 1c3ba394f290437b90045c4a98fb75ca (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:11.160697  6504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ba394f290437b90045c4a98fb75ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.161121  6486 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:11.161392  6660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ba394f290437b90045c4a98fb75ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.160630  6721 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.165864  6721 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.168107  6721 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:11.168162  6740 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:11.170770  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:11.172240  6714 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:11.172052  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.172802  6714 ts_tablet_manager.cc:1403] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.186s	user 0.006s	sys 0.015s
I20260709 13:16:11.172850  6503 tablet_service.cc:1511] Processing CreateTablet for tablet 56ba1578b97b4faa966168899769e067 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 13:16:11.173985  6637 leader_election.cc:343] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.174335  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ba1578b97b4faa966168899769e067. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:11.174785  6637 leader_election.cc:343] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.175222  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:11.175930  6714 raft_consensus.cc:359] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.176399  6721 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:11.176646  6714 raft_consensus.cc:385] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.183135  6587 tablet_service.cc:1511] Processing CreateTablet for tablet 56ba1578b97b4faa966168899769e067 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:11.184789  6660 tablet_service.cc:1511] Processing CreateTablet for tablet 56ba1578b97b4faa966168899769e067 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:11.158340  6713 tablet_bootstrap.cc:654] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.186180  6714 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.197458  6587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ba1578b97b4faa966168899769e067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.197585  6711 ts_tablet_manager.cc:1434] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.001s
I20260709 13:16:11.203853  6578 tablet_service.cc:1511] Processing CreateTablet for tablet 34cabfbde16f4e10affcaec358aa0dd4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:11.204617  6653 tablet_service.cc:1511] Processing CreateTablet for tablet 34cabfbde16f4e10affcaec358aa0dd4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:11.200757  6714 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.201089  6510 tablet_service.cc:1511] Processing CreateTablet for tablet 34cabfbde16f4e10affcaec358aa0dd4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:11.206509  6510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34cabfbde16f4e10affcaec358aa0dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.201684  6660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ba1578b97b4faa966168899769e067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.208210  6711 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.212291  6511 tablet_service.cc:1511] Processing CreateTablet for tablet f4112f607ad043a3889d117a4fb6ca59 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:11.213218  6714 ts_tablet_manager.cc:1434] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:16:11.213572  6511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4112f607ad043a3889d117a4fb6ca59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.215379  6578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34cabfbde16f4e10affcaec358aa0dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.217326  6653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34cabfbde16f4e10affcaec358aa0dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.239972  6582 tablet_service.cc:1511] Processing CreateTablet for tablet f4112f607ad043a3889d117a4fb6ca59 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:11.241679  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4112f607ad043a3889d117a4fb6ca59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.238421  6659 tablet_service.cc:1511] Processing CreateTablet for tablet f4112f607ad043a3889d117a4fb6ca59 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:11.242733  6714 tablet_bootstrap.cc:492] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:11.243459  6659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4112f607ad043a3889d117a4fb6ca59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.242054  6711 tablet_bootstrap.cc:654] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.256357  6509 tablet_service.cc:1511] Processing CreateTablet for tablet 7a72647203b143708b26072192a9c102 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:11.257620  6509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a72647203b143708b26072192a9c102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.259816  6654 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbaf7daabfe40028228e412b8f9214b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:11.261312  6654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbaf7daabfe40028228e412b8f9214b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.261797  6512 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbaf7daabfe40028228e412b8f9214b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:11.263267  6512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbaf7daabfe40028228e412b8f9214b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.265785  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 7a72647203b143708b26072192a9c102 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:11.267102  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a72647203b143708b26072192a9c102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.266849  6580 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbaf7daabfe40028228e412b8f9214b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:11.268155  6580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbaf7daabfe40028228e412b8f9214b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.279641  6711 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.280211  6711 ts_tablet_manager.cc:1403] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.005s
I20260709 13:16:11.283193  6711 raft_consensus.cc:359] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.284001  6711 raft_consensus.cc:385] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.284309  6711 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.285125  6711 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.287314  6711 ts_tablet_manager.cc:1434] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:11.288713  6711 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.294186  6582 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc18602dfbd4fc9897b6b696b497f73 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:11.295166  6711 tablet_bootstrap.cc:654] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.295567  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc18602dfbd4fc9897b6b696b497f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.295912  6586 tablet_service.cc:1511] Processing CreateTablet for tablet 7a72647203b143708b26072192a9c102 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:11.297241  6586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a72647203b143708b26072192a9c102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.295912  6657 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc18602dfbd4fc9897b6b696b497f73 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:11.298907  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc18602dfbd4fc9897b6b696b497f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.296906  6507 tablet_service.cc:1511] Processing CreateTablet for tablet 9fc18602dfbd4fc9897b6b696b497f73 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:11.301092  6507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fc18602dfbd4fc9897b6b696b497f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.303038  6508 tablet_service.cc:1511] Processing CreateTablet for tablet e15a9820f1e1414f829e7fecece15dc0 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:11.309613  6585 tablet_service.cc:1511] Processing CreateTablet for tablet e15a9820f1e1414f829e7fecece15dc0 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:11.310386  6661 tablet_service.cc:1511] Processing CreateTablet for tablet e15a9820f1e1414f829e7fecece15dc0 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:11.310962  6508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15a9820f1e1414f829e7fecece15dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.311432  6585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15a9820f1e1414f829e7fecece15dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.311712  6661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15a9820f1e1414f829e7fecece15dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.330017  6713 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:11.330536  6713 ts_tablet_manager.cc:1403] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.210s	user 0.011s	sys 0.009s
I20260709 13:16:11.353998  6506 tablet_service.cc:1511] Processing CreateTablet for tablet 542a321aae654f82803540bffcf7968f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:11.355412  6506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542a321aae654f82803540bffcf7968f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.356014  6505 tablet_service.cc:1511] Processing CreateTablet for tablet c7d55fddd552409bbf822686f45eda69 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:11.357733  6505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d55fddd552409bbf822686f45eda69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.358220  6714 tablet_bootstrap.cc:654] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.339512  6721 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.359148  6721 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.359735  6504 tablet_service.cc:1511] Processing CreateTablet for tablet 35df31b6c7b2475cac55c1e114ba24dc (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:11.363508  6721 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:11.365820  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:11.366585  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:11.369441  6637 leader_election.cc:343] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.362452  6504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35df31b6c7b2475cac55c1e114ba24dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:11.371320  6637 leader_election.cc:343] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.371244  6658 tablet_service.cc:1511] Processing CreateTablet for tablet c98ecd701e924e049261801a6e8b1e3b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:11.371806  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:11.372624  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98ecd701e924e049261801a6e8b1e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.374806  6721 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:11.367471  6511 tablet_service.cc:1511] Processing CreateTablet for tablet c98ecd701e924e049261801a6e8b1e3b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:11.376391  6511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98ecd701e924e049261801a6e8b1e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.397060  6585 tablet_service.cc:1511] Processing CreateTablet for tablet c98ecd701e924e049261801a6e8b1e3b (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:11.398458  6585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98ecd701e924e049261801a6e8b1e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.398197  6584 tablet_service.cc:1511] Processing CreateTablet for tablet c7d55fddd552409bbf822686f45eda69 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:11.399554  6584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d55fddd552409bbf822686f45eda69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.400421  6578 tablet_service.cc:1511] Processing CreateTablet for tablet 35df31b6c7b2475cac55c1e114ba24dc (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:11.401733  6578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35df31b6c7b2475cac55c1e114ba24dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.403563  6582 tablet_service.cc:1511] Processing CreateTablet for tablet 542a321aae654f82803540bffcf7968f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:11.404502  6715 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.404729  6582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542a321aae654f82803540bffcf7968f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.405006  6715 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.416875  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 35df31b6c7b2475cac55c1e114ba24dc (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:11.417893  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35df31b6c7b2475cac55c1e114ba24dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.419263  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:11.420280  6652 tablet_service.cc:1511] Processing CreateTablet for tablet 542a321aae654f82803540bffcf7968f (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 13:16:11.413767  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:11.426481  6713 raft_consensus.cc:359] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.428138  6713 raft_consensus.cc:385] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.428493  6713 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.427824  6652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542a321aae654f82803540bffcf7968f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.436167  6715 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.436780  6713 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.437427  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:11.439190  6561 leader_election.cc:343] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.445549  6504 tablet_service.cc:1511] Processing CreateTablet for tablet 7365731c56504a678dc1a5ceac3f8da4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:11.446660  6713 ts_tablet_manager.cc:1434] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.116s	user 0.007s	sys 0.000s
I20260709 13:16:11.446913  6504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7365731c56504a678dc1a5ceac3f8da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.474279  6660 tablet_service.cc:1511] Processing CreateTablet for tablet 7365731c56504a678dc1a5ceac3f8da4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:11.476104  6660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7365731c56504a678dc1a5ceac3f8da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.482851  6658 tablet_service.cc:1511] Processing CreateTablet for tablet 520b1aa490ee4869a2814c313fac4a0e (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:11.484014  6658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520b1aa490ee4869a2814c313fac4a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.482775  6713 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
W20260709 13:16:11.457289  6563 leader_election.cc:343] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.495985  6657 tablet_service.cc:1511] Processing CreateTablet for tablet e5a761916a9242a99d2d5b2e54ffda61 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:11.497320  6657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a761916a9242a99d2d5b2e54ffda61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.499609  6511 tablet_service.cc:1511] Processing CreateTablet for tablet 520b1aa490ee4869a2814c313fac4a0e (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:11.500844  6511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520b1aa490ee4869a2814c313fac4a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.501811  6503 tablet_service.cc:1511] Processing CreateTablet for tablet e5a761916a9242a99d2d5b2e54ffda61 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:11.503038  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a761916a9242a99d2d5b2e54ffda61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.494508  6563 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:11.524377  6715 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:11.524528  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 68414.8us
I20260709 13:16:11.524989  6714 tablet_bootstrap.cc:492] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:11.525710  6714 ts_tablet_manager.cc:1403] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.283s	user 0.011s	sys 0.009s
I20260709 13:16:11.530076  6584 tablet_service.cc:1511] Processing CreateTablet for tablet 520b1aa490ee4869a2814c313fac4a0e (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:11.530717  6587 tablet_service.cc:1511] Processing CreateTablet for tablet e5a761916a9242a99d2d5b2e54ffda61 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:11.531525  6584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520b1aa490ee4869a2814c313fac4a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.532341  6587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a761916a9242a99d2d5b2e54ffda61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.508033  6659 tablet_service.cc:1511] Processing CreateTablet for tablet c7d55fddd552409bbf822686f45eda69 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:11.544307  6659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7d55fddd552409bbf822686f45eda69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.529090  6714 raft_consensus.cc:359] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.547472  6714 raft_consensus.cc:385] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.550200  6714 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.553043  6715 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.553573  6715 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.555822  6715 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.553172  6714 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.557731  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:11.570405  6561 leader_election.cc:343] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.572538  6714 ts_tablet_manager.cc:1434] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:16:11.578773  6714 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:11.578202  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:11.588279  6563 leader_election.cc:343] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.589287  6563 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:11.589084  6721 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:11.590788  6715 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:11.585435  6713 tablet_bootstrap.cc:654] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.583726  6711 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.602706  6711 ts_tablet_manager.cc:1403] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.314s	user 0.051s	sys 0.066s
I20260709 13:16:11.609575  6711 raft_consensus.cc:359] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.610716  6711 raft_consensus.cc:385] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.611130  6711 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.614838  6711 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.618010  6711 ts_tablet_manager.cc:1434] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:16:11.619412  6711 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.586050  6585 tablet_service.cc:1511] Processing CreateTablet for tablet 7365731c56504a678dc1a5ceac3f8da4 (DEFAULT_TABLE table=test-workload [id=b35ecb8e11bf4c77a827717b0226b074]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:11.623008  6585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7365731c56504a678dc1a5ceac3f8da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:11.625978  6740 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.626487  6740 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.628178  6711 tablet_bootstrap.cc:654] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.628818  6740 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.630096  6714 tablet_bootstrap.cc:654] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.630551  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.632319  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:11.636945  6670 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:11.638088  6486 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:16:11.639384  6488 leader_election.cc:343] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.640287  6740 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.643213  6740 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.644909  6740 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.652998  6721 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:11.658571  6740 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.661418  6740 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.662151  6740 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.662439  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:16:11.664319  6488 leader_election.cc:343] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.665555  6711 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.665835  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:11.666204  6711 ts_tablet_manager.cc:1403] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.005s
I20260709 13:16:11.666581  6670 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.669855  6711 raft_consensus.cc:359] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.670794  6711 raft_consensus.cc:385] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.671120  6711 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.671952  6711 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.673754  6740 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.675259  6670 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:11.677488  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.678333  6711 ts_tablet_manager.cc:1434] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
W20260709 13:16:11.679077  6488 leader_election.cc:343] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.679422  6740 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.681058  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:11.681355  6486 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:11.681946  6671 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:11.683593  6740 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.683974  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:11.684296  6740 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:11.685607  6740 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.687599  6711 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.694666  6713 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:11.694936  6711 tablet_bootstrap.cc:654] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.707226  6713 ts_tablet_manager.cc:1403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.253s	user 0.052s	sys 0.036s
I20260709 13:16:11.709015  6714 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:11.709573  6714 ts_tablet_manager.cc:1403] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.131s	user 0.009s	sys 0.005s
I20260709 13:16:11.716589  6711 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.717021  6711 ts_tablet_manager.cc:1403] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.004s
I20260709 13:16:11.718150  6713 raft_consensus.cc:359] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.718938  6713 raft_consensus.cc:385] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.719362  6713 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.717940  6418 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:11.720340  6713 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.720916  6714 raft_consensus.cc:359] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.721861  6714 raft_consensus.cc:385] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.722251  6714 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.722831  6713 ts_tablet_manager.cc:1434] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 13:16:11.723348  6714 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.723846  6713 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:11.701203  6740 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.724370  6740 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.724820  6740 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.725543  6714 ts_tablet_manager.cc:1434] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.003s
I20260709 13:16:11.726483  6714 tablet_bootstrap.cc:492] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:11.720098  6711 raft_consensus.cc:359] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.731881  6740 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.732368  6711 raft_consensus.cc:385] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.732693  6711 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.733500  6711 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.735354  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:11.735707  6711 ts_tablet_manager.cc:1434] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:16:11.736568  6711 tablet_bootstrap.cc:492] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
W20260709 13:16:11.736726  6488 leader_election.cc:343] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.742214  6711 tablet_bootstrap.cc:654] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.747340  6714 tablet_bootstrap.cc:654] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.753510  6740 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.754248  6713 tablet_bootstrap.cc:654] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.755437  6711 tablet_bootstrap.cc:492] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.756016  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:11.756623  6670 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.758493  6711 ts_tablet_manager.cc:1403] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 13:16:11.763824  6713 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:11.764349  6713 ts_tablet_manager.cc:1403] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.041s	user 0.004s	sys 0.007s
I20260709 13:16:11.765862  6670 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:11.767323  6713 raft_consensus.cc:359] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.767659  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:11.768057  6713 raft_consensus.cc:385] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.768167  6714 tablet_bootstrap.cc:492] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:11.764674  6711 raft_consensus.cc:359] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.768365  6713 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.768575  6711 raft_consensus.cc:385] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.768684  6714 ts_tablet_manager.cc:1403] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.007s
I20260709 13:16:11.768910  6711 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.768970  6740 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.769659  6711 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.771687  6711 ts_tablet_manager.cc:1434] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:16:11.771857  6714 raft_consensus.cc:359] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.772567  6711 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.772504  6714 raft_consensus.cc:385] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.775867  6714 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.775486  6713 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.770027  6740 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:11.778378  6740 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.780122  6713 ts_tablet_manager.cc:1434] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:16:11.781131  6713 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:11.776711  6714 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.790014  6714 ts_tablet_manager.cc:1434] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:16:11.791079  6711 tablet_bootstrap.cc:654] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.791239  6713 tablet_bootstrap.cc:654] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.797290  6714 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:11.802348  6420 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:11.807003  6715 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.807554  6715 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.810081  6715 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.811070  6714 tablet_bootstrap.cc:654] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.812063  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:11.812776  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:11.813387  6522 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
W20260709 13:16:11.814900  6561 leader_election.cc:343] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.815825  6563 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:11.817286  6715 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.817466  6713 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:11.817788  6715 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.818186  6713 ts_tablet_manager.cc:1403] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.003s
I20260709 13:16:11.818301  6715 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.822252  6713 raft_consensus.cc:359] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.822958  6713 raft_consensus.cc:385] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.823253  6713 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.824270  6714 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:11.824757  6714 ts_tablet_manager.cc:1403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.002s
I20260709 13:16:11.827951  6714 raft_consensus.cc:359] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.828650  6714 raft_consensus.cc:385] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.828946  6714 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.830375  6713 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.833244  6721 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.833498  6713 ts_tablet_manager.cc:1434] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:16:11.829638  6714 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.833705  6721 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.835917  6714 ts_tablet_manager.cc:1434] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:16:11.836074  6713 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:11.836913  6714 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:11.837598  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:11.838938  6637 leader_election.cc:343] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.838901  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.839809  6596 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:11.842821  6714 tablet_bootstrap.cc:654] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.845829  6713 tablet_bootstrap.cc:654] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.847529  6637 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:11.836095  6721 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:11.848948  6759 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.849480  6759 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.849915  6759 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.855909  6715 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.859843  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:11.860723  6521 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.862076  6759 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.867483  6711 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.868187  6711 ts_tablet_manager.cc:1403] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.096s	user 0.021s	sys 0.024s
I20260709 13:16:11.868973  6713 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:11.869436  6713 ts_tablet_manager.cc:1403] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.019s
I20260709 13:16:11.869736  6740 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.870196  6740 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.871124  6711 raft_consensus.cc:359] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.871737  6711 raft_consensus.cc:385] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.872001  6711 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.872324  6740 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.872619  6711 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.873693  6521 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:11.875376  6563 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:11.876500  6759 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:11.876822  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.877194  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:11.877597  6596 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:11.878875  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:11.879638  6597 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.878175  6713 raft_consensus.cc:359] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.881788  6713 raft_consensus.cc:385] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.882107  6713 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.874362  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:11.882988  6670 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:11.882826  6713 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.884336  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:16:11.885649  6561 leader_election.cc:343] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.886307  6740 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.886713  6740 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.887058  6740 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.888494  6774 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.888962  6774 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.890650  6713 ts_tablet_manager.cc:1434] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:16:11.891569  6713 tablet_bootstrap.cc:492] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:11.892259  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:11.892884  6670 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:11.894090  6561 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:11.896155  6773 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.896617  6773 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:11.897420  6773 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.897948  6714 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:11.898437  6714 ts_tablet_manager.cc:1403] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.022s
I20260709 13:16:11.900264  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:11.900940  6714 raft_consensus.cc:359] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.901495  6597 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
W20260709 13:16:11.901641  6637 leader_election.cc:343] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.902302  6715 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.907109  6711 ts_tablet_manager.cc:1434] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.039s	user 0.012s	sys 0.004s
I20260709 13:16:11.907405  6713 tablet_bootstrap.cc:654] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.908406  6711 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.908946  6776 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.909354  6776 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.909684  6776 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.909624  6740 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.901571  6714 raft_consensus.cc:385] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.914270  6714 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.915134  6711 tablet_bootstrap.cc:654] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.915330  6714 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.916172  6637 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:11.917351  6759 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.917786  6759 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:11.918593  6759 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.919903  6756 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.920471  6756 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.922765  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:11.923043  6756 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:11.923367  6670 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.923697  6740 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.925287  6714 ts_tablet_manager.cc:1434] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:16:11.926448  6714 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:11.928078  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:11.928723  6521 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:11.929924  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:11.930781  6756 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.931124  6756 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.931484  6756 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.933465  6714 tablet_bootstrap.cc:654] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.940076  6711 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.940629  6711 ts_tablet_manager.cc:1403] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.000s
I20260709 13:16:11.943486  6711 raft_consensus.cc:359] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.944424  6711 raft_consensus.cc:385] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.944766  6711 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.945534  6711 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.946887  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:11.947503  6597 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:11.945894  6670 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:11.948655  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:11.949451  6740 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.949697  6711 ts_tablet_manager.cc:1434] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:16:11.949967  6740 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:11.951176  6711 tablet_bootstrap.cc:492] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.951222  6740 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.949810  6756 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.954608  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:11.955260  6521 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.958775  6714 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:11.958822  6711 tablet_bootstrap.cc:654] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.959546  6714 ts_tablet_manager.cc:1403] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.014s
I20260709 13:16:11.966066  6521 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:11.968407  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:11.969300  6721 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.971451  6711 tablet_bootstrap.cc:492] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:11.972168  6711 ts_tablet_manager.cc:1403] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.009s
I20260709 13:16:11.972237  6775 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.972827  6775 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.975026  6775 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:11.978001  6418 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:11.979133  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:11.980783  6561 leader_election.cc:343] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.982309  6714 raft_consensus.cc:359] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.984169  6714 raft_consensus.cc:385] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.984555  6714 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.983286  6711 raft_consensus.cc:359] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.985425  6711 raft_consensus.cc:385] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.985786  6711 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.986608  6711 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.987739  6721 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:11.989465  6721 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:11.990518  6711 ts_tablet_manager.cc:1434] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:16:11.991715  6711 tablet_bootstrap.cc:492] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:11.992512  6419 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.993971  6756 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:16:11.988514  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 113232us
I20260709 13:16:11.994520  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:11.995134  6597 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.997722  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:11.998456  6711 tablet_bootstrap.cc:654] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.998759  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:11.998941  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:11.987928  6714 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.002089  6714 ts_tablet_manager.cc:1434] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:16:12.002995  6714 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.004530  6597 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:12.009244  6774 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:16:12.010592  6563 leader_election.cc:343] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.011039  6563 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.012060  6774 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.016615  6420 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:12.017760  6713 tablet_bootstrap.cc:492] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.018808  6713 ts_tablet_manager.cc:1403] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.127s	user 0.028s	sys 0.027s
I20260709 13:16:12.021545  6776 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.021637  6713 raft_consensus.cc:359] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.022835  6596 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:12.022883  6713 raft_consensus.cc:385] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.023308  6713 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.023919  6711 tablet_bootstrap.cc:492] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.024405  6711 ts_tablet_manager.cc:1403] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.006s
I20260709 13:16:12.024122  6713 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.026006  6713 ts_tablet_manager.cc:1434] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:16:12.026849  6713 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.027194  6711 raft_consensus.cc:359] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.027882  6711 raft_consensus.cc:385] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.028182  6711 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.028890  6714 tablet_bootstrap.cc:654] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.029031  6711 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.031267  6711 ts_tablet_manager.cc:1434] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:12.032435  6711 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:12.033111  6713 tablet_bootstrap.cc:654] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.036325  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:12.040042  6776 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.040901  6740 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.041111  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:12.041383  6740 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.041811  6670 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:12.043098  6561 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.049561  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.049547  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.049856  6711 tablet_bootstrap.cc:654] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.050545  6596 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:12.051435  6721 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.049466  6418 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:12.051831  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.051880  6721 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.052839  6755 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.053155  6755 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.053427  6755 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.053978  6721 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:12.054593  6670 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:12.055586  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.056185  6521 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:12.057457  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.058830  6721 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.059291  6721 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.059716  6721 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.060403  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.061018  6596 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:12.058420  6740 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.063928  6755 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.064771  6714 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.065217  6714 ts_tablet_manager.cc:1403] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.000s
I20260709 13:16:12.066268  6776 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.067332  6755 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.067811  6714 raft_consensus.cc:359] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.068035  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:12.068461  6714 raft_consensus.cc:385] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.068737  6714 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.068811  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:12.069355  6670 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.069438  6714 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.073179  6596 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.076982  6756 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.077610  6756 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.079231  6714 ts_tablet_manager.cc:1434] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:16:12.080236  6756 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:12.081792  6713 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.082062  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.082454  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.083340  6713 ts_tablet_manager.cc:1403] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.001s
I20260709 13:16:12.083688  6597 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
W20260709 13:16:12.084916  6637 leader_election.cc:343] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.085180  6670 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:12.086263  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.087284  6755 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.087785  6755 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:12.087718  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:12.088987  6756 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.088632  6755 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.089381  6756 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.089932  6756 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.089193  6713 raft_consensus.cc:359] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.090577  6713 raft_consensus.cc:385] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.091001  6713 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.091912  6713 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.093452  6714 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.087692  6721 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.099031  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:12.099216  6713 ts_tablet_manager.cc:1434] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:16:12.099077  6721 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:12.099696  6597 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.100244  6713 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.101706  6756 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.103278  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:12.104344  6521 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.109987  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:16:12.111332  6637 leader_election.cc:343] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.117969  6713 tablet_bootstrap.cc:654] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.118754  6521 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:12.118796  6597 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:12.111469  6596 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:12.120712  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:12.120795  6420 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:12.121644  6756 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:12.122598  6756 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.123132  6756 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:12.123216  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:12.123970  6596 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.124384  6711 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.124202  6756 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.124840  6711 ts_tablet_manager.cc:1403] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.093s	user 0.014s	sys 0.019s
I20260709 13:16:12.131789  6596 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:12.127430  6714 tablet_bootstrap.cc:654] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.132916  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:12.133729  6711 raft_consensus.cc:359] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.134115  6721 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.134464  6711 raft_consensus.cc:385] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.134696  6721 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:12.134789  6711 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.135792  6711 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.135689  6721 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.149068  6711 ts_tablet_manager.cc:1434] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:16:12.150483  6711 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:12.150292  6418 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:12.154242  6714 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.154757  6714 ts_tablet_manager.cc:1403] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.061s	user 0.023s	sys 0.003s
I20260709 13:16:12.157441  6714 raft_consensus.cc:359] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.158092  6714 raft_consensus.cc:385] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.158305  6711 tablet_bootstrap.cc:654] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.158391  6714 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.159220  6714 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.169695  6711 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.170260  6711 ts_tablet_manager.cc:1403] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 13:16:12.171155  6714 ts_tablet_manager.cc:1434] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:16:12.172971  6711 raft_consensus.cc:359] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.173663  6596 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.173761  6711 raft_consensus.cc:385] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.174235  6711 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.174989  6711 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.177034  6711 ts_tablet_manager.cc:1434] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:12.178409  6711 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:12.182924  6713 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.183038  6776 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.183395  6713 ts_tablet_manager.cc:1403] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.005s
I20260709 13:16:12.183549  6776 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.184495  6711 tablet_bootstrap.cc:654] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.185786  6776 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.186239  6713 raft_consensus.cc:359] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.186939  6713 raft_consensus.cc:385] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.187216  6713 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.187850  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.187903  6713 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.188655  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.189416  6670 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:12.190127  6713 ts_tablet_manager.cc:1434] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:12.191038  6713 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.195060  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.195240  6755 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.197185  6713 tablet_bootstrap.cc:654] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.196830  6740 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.200475  6774 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.181552  6417 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:12.202750  6714 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.206794  6740 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.213397  6714 tablet_bootstrap.cc:654] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.219127  6774 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.230571  6711 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.235383  6711 ts_tablet_manager.cc:1403] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.026s
I20260709 13:16:12.243722  6774 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.244241  6774 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.244642  6714 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.245195  6714 ts_tablet_manager.cc:1403] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.008s
I20260709 13:16:12.246371  6774 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.248634  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:12.250149  6561 leader_election.cc:343] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.251781  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:12.254402  6563 leader_election.cc:343] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.254880  6563 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.255918  6776 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.253957  6774 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.258795  6774 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.260339  6714 raft_consensus.cc:359] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.261071  6714 raft_consensus.cc:385] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.261400  6714 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.262084  6714 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.264114  6714 ts_tablet_manager.cc:1434] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:16:12.267383  6714 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.261359  6774 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.271960  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.272075  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:12.273473  6561 leader_election.cc:343] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:12.274108  6563 leader_election.cc:343] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.274487  6563 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.276816  6713 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.277226  6713 ts_tablet_manager.cc:1403] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.014s
I20260709 13:16:12.275833  6774 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.280947  6713 raft_consensus.cc:359] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.281641  6713 raft_consensus.cc:385] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.281931  6713 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.282655  6713 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:12.284125  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54906.5us
I20260709 13:16:12.284861  6713 ts_tablet_manager.cc:1434] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:12.285261  6711 raft_consensus.cc:359] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.285743  6713 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.285992  6711 raft_consensus.cc:385] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.286044  6714 tablet_bootstrap.cc:654] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.286442  6711 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.287317  6711 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.293290  6711 ts_tablet_manager.cc:1434] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 13:16:12.294186  6803 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.294780  6803 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.298250  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.298900  6669 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:12.300264  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.301105  6740 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.301455  6740 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.301772  6740 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.309059  6755 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.309580  6755 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.311796  6755 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.311873  6803 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.311770  6714 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.312793  6714 ts_tablet_manager.cc:1403] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.011s
I20260709 13:16:12.312844  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.313611  6711 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:12.313699  6669 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:12.309067  6713 tablet_bootstrap.cc:654] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.316812  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.316838  6714 raft_consensus.cc:359] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.317863  6803 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.312767  6740 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.318413  6803 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.318800  6803 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.319150  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.319772  6596 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:12.319976  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.320816  6595 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:12.320771  6755 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.317927  6714 raft_consensus.cc:385] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.322253  6714 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.323200  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:12.323756  6669 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.326269  6803 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.327437  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:12.328066  6595 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.329630  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:12.330255  6594 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.331817  6669 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:12.333129  6740 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.335008  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:12.335664  6671 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.338212  6711 tablet_bootstrap.cc:654] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.339005  6714 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.340418  6595 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:12.342289  6774 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.342226  6488 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.343034  6774 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.345585  6740 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.346521  6740 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:12.348043  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.348884  6714 ts_tablet_manager.cc:1434] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:16:12.349190  6755 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.349643  6755 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.350023  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.352104  6803 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.351753  6740 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:12.353242  6561 leader_election.cc:343] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.354595  6671 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:12.355043  6755 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:16:12.355659  6563 leader_election.cc:343] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.356106  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.356884  6776 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.358718  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.358834  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.359617  6803 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.359854  6594 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:12.361141  6803 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.362294  6803 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:12.363523  6803 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.350143  6714 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.374418  6417 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:12.351037  6774 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.376916  6774 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.378629  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.379704  6594 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:12.394389  6714 tablet_bootstrap.cc:654] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.386404  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:12.397850  6740 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.404338  6711 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.404827  6711 ts_tablet_manager.cc:1403] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.002s
I20260709 13:16:12.406289  6713 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.406836  6713 ts_tablet_manager.cc:1403] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.121s	user 0.014s	sys 0.035s
I20260709 13:16:12.410598  6713 raft_consensus.cc:359] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.411336  6713 raft_consensus.cc:385] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.411660  6713 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.414111  6713 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.416962  6713 ts_tablet_manager.cc:1434] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:16:12.414799  6711 raft_consensus.cc:359] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.420359  6711 raft_consensus.cc:385] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.420698  6711 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.421525  6711 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.423462  6711 ts_tablet_manager.cc:1434] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:16:12.424358  6713 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.419826  6740 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.426249  6721 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:16:12.429461  6740 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.430469  6740 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.425117  6803 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:12.439867  6711 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:12.478412  6774 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.479110  6774 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.483332  6774 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.483446  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:12.484993  6561 leader_election.cc:343] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.486997  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.452566  6740 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.502521  6594 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:12.507009  6711 tablet_bootstrap.cc:654] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.501794  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.508777  6669 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.509966  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:12.512050  6563 leader_election.cc:343] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.512499  6563 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.513242  6773 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.514879  6488 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:12.515697  6740 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.519548  6714 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.520059  6714 ts_tablet_manager.cc:1403] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.170s	user 0.039s	sys 0.035s
I20260709 13:16:12.522264  6721 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.522686  6714 raft_consensus.cc:359] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.523499  6714 raft_consensus.cc:385] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.523814  6714 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.524605  6714 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.525161  6713 tablet_bootstrap.cc:654] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.526682  6714 ts_tablet_manager.cc:1434] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:12.535873  6714 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.535246  6774 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.536442  6416 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:12.546440  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:12.555197  6756 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.559923  6740 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.578925  6756 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.570565  6813 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:16:12.581352  6759 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.583081  6759 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.584462  6756 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.585319  6759 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:12.583235  6740 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:16:12.592901  6713 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.592844  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.593578  6713 ts_tablet_manager.cc:1403] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.169s	user 0.030s	sys 0.006s
I20260709 13:16:12.595492  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.596160  6595 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:12.599720  6713 raft_consensus.cc:359] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.600579  6713 raft_consensus.cc:385] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.600909  6713 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.601704  6713 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.602632  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:12.605163  6714 tablet_bootstrap.cc:654] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.609426  6711 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.609472  6713 ts_tablet_manager.cc:1434] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
W20260709 13:16:12.610131  6637 leader_election.cc:343] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.610369  6756 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.621397  6756 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.621567  6711 ts_tablet_manager.cc:1403] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.182s	user 0.021s	sys 0.015s
I20260709 13:16:12.623512  6713 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.621793  6756 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.626101  6711 raft_consensus.cc:359] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.628024  6711 raft_consensus.cc:385] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.628423  6711 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.629485  6711 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.633591  6756 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.633632  6711 ts_tablet_manager.cc:1434] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.007s
I20260709 13:16:12.637099  6711 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:12.636886  6821 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.666316  6821 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.666836  6821 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.668519  6756 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:12.670156  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:12.671808  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:12.672477  6596 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.675213  6714 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.675698  6714 ts_tablet_manager.cc:1403] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.148s	user 0.011s	sys 0.009s
I20260709 13:16:12.678095  6714 raft_consensus.cc:359] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.678791  6714 raft_consensus.cc:385] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.679044  6714 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
W20260709 13:16:12.679342  6637 leader_election.cc:343] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.641629  6713 tablet_bootstrap.cc:654] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.682857  6821 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:12.685564  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.687881  6597 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:12.689538  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:12.689875  6713 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.690327  6713 ts_tablet_manager.cc:1403] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.067s	user 0.004s	sys 0.005s
I20260709 13:16:12.686219  6714 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.692338  6759 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.692720  6759 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.693076  6759 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.691588  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.693904  6713 raft_consensus.cc:359] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.694595  6713 raft_consensus.cc:385] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.693955  6596 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:12.695436  6713 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.696120  6713 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.696543  6755 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.696996  6755 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.697827  6714 ts_tablet_manager.cc:1434] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:16:12.698760  6714 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.699611  6755 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.701313  6711 tablet_bootstrap.cc:654] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.701421  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:12.704501  6486 leader_election.cc:343] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.704668  6756 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.704535  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:16:12.707676  6637 leader_election.cc:343] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.708802  6721 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.711695  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:12.713075  6488 leader_election.cc:343] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.713514  6488 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:12.714488  6713 ts_tablet_manager.cc:1434] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:16:12.715415  6713 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.716154  6759 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.718776  6740 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.719332  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:16:12.720866  6637 leader_election.cc:343] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.729985  6759 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:12.732146  6721 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:12.738206  6711 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.738601  6711 ts_tablet_manager.cc:1403] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.102s	user 0.006s	sys 0.008s
I20260709 13:16:12.736620  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:12.741735  6594 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.742554  6721 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.746217  6713 tablet_bootstrap.cc:654] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.747895  6759 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.750248  6594 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:12.752846  6711 raft_consensus.cc:359] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.753530  6711 raft_consensus.cc:385] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.753798  6711 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.754734  6714 tablet_bootstrap.cc:654] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.760592  6711 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.763774  6711 ts_tablet_manager.cc:1434] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.005s
I20260709 13:16:12.763847  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:12.764770  6711 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:12.764992  6759 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.765131  6774 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.765508  6759 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:12.765674  6774 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.766340  6759 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.767710  6774 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.772213  6740 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.769016  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:12.778191  6563 leader_election.cc:343] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.775614  6740 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.790046  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.795145  6759 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.798468  6759 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.794651  6774 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.804991  6774 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.807616  6774 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.807502  6420 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:12.797629  6561 leader_election.cc:343] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.809151  6561 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.809345  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.809908  6774 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.809942  6522 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:12.811321  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.811476  6740 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.813174  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.814214  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:12.815719  6486 leader_election.cc:343] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.815027  6774 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.818512  6774 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.818882  6774 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.818964  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.819912  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:12.820124  6561 leader_election.cc:343] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.822165  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:12.823349  6637 leader_election.cc:343] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:12.824568  6637 leader_election.cc:343] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.824754  6740 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.825069  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:12.824570  6713 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.826542  6713 ts_tablet_manager.cc:1403] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.012s
I20260709 13:16:12.817492  6759 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:12.826772  6756 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.829481  6713 raft_consensus.cc:359] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.830125  6713 raft_consensus.cc:385] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.830382  6713 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.832681  6711 tablet_bootstrap.cc:654] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.837520  6713 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.837679  6420 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:12.836441  6714 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.836251  6774 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.839833  6714 ts_tablet_manager.cc:1403] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.141s	user 0.017s	sys 0.022s
I20260709 13:16:12.840425  6596 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:12.842422  6714 raft_consensus.cc:359] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.843111  6714 raft_consensus.cc:385] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.843175  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:12.843428  6714 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.844053  6714 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.844408  6740 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.844947  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:12.846695  6774 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.839841  6713 ts_tablet_manager.cc:1434] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:16:12.847621  6714 ts_tablet_manager.cc:1434] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:12.847497  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:12.848177  6713 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.848568  6669 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.850054  6714 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.855443  6713 tablet_bootstrap.cc:654] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.855541  6714 tablet_bootstrap.cc:654] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.855544  6669 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:12.868472  6713 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:12.869123  6713 ts_tablet_manager.cc:1403] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:16:12.871618  6774 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.872290  6774 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.876312  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:12.877192  6521 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:12.878939  6563 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:12.880244  6561 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:12.881467  6776 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.881970  6776 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.882856  6773 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.855556  6740 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.882417  6776 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.900928  6711 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:12.901713  6711 ts_tablet_manager.cc:1403] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.137s	user 0.017s	sys 0.043s
I20260709 13:16:12.905376  6711 raft_consensus.cc:359] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.906355  6711 raft_consensus.cc:385] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.906800  6711 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.907797  6711 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.908900  6522 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:12.910506  6776 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.914815  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:16:12.916365  6561 leader_election.cc:343] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.917387  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:12.917709  6755 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.918009  6522 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.923316  6774 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.924224  6813 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.924688  6813 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.931574  6803 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.932652  6813 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.933461  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:12.935222  6803 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.936800  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:12.943895  6561 leader_election.cc:343] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:12.948307  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:12.949092  6596 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:12.952615  6488 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:12.954062  6755 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.960443  6776 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:12.972675  6713 raft_consensus.cc:359] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.973461  6713 raft_consensus.cc:385] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.973774  6713 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.978370  6714 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:12.979012  6714 ts_tablet_manager.cc:1403] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.129s	user 0.058s	sys 0.060s
I20260709 13:16:12.982502  6714 raft_consensus.cc:359] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.984326  6714 raft_consensus.cc:385] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.984678  6714 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.985482  6714 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.987331  6713 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.987766  6714 ts_tablet_manager.cc:1434] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:12.988926  6714 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:12.989336  6713 ts_tablet_manager.cc:1434] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.120s	user 0.005s	sys 0.000s
I20260709 13:16:12.990168  6713 tablet_bootstrap.cc:492] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:12.990780  6755 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.991429  6773 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:12.992316  6773 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:12.994782  6714 tablet_bootstrap.cc:654] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.004058  6522 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:13.005303  6563 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:13.006735  6843 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.007239  6843 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:13.012496  6843 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.015040  6714 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.015491  6714 ts_tablet_manager.cc:1403] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.011s
I20260709 13:16:13.017177  6711 ts_tablet_manager.cc:1434] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.115s	user 0.022s	sys 0.061s
I20260709 13:16:13.018092  6711 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.018216  6713 tablet_bootstrap.cc:654] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.017879  6714 raft_consensus.cc:359] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.018704  6714 raft_consensus.cc:385] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.018975  6714 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.019585  6714 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.021595  6714 ts_tablet_manager.cc:1434] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:16:13.022444  6714 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.024024  6711 tablet_bootstrap.cc:654] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.028048  6714 tablet_bootstrap.cc:654] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.029395  6418 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.050405  6711 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.050915  6711 ts_tablet_manager.cc:1403] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.033s	user 0.005s	sys 0.015s
I20260709 13:16:13.051245  6714 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.051705  6714 ts_tablet_manager.cc:1403] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.002s
I20260709 13:16:13.053447  6711 raft_consensus.cc:359] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.054178  6711 raft_consensus.cc:385] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.054503  6711 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.054234  6714 raft_consensus.cc:359] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.054951  6714 raft_consensus.cc:385] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.055208  6714 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.055192  6711 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.055981  6714 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.057803  6714 ts_tablet_manager.cc:1434] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:13.057818  6711 ts_tablet_manager.cc:1434] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:16:13.058818  6714 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.058818  6711 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.065174  6711 tablet_bootstrap.cc:654] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.071805  6418 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.074098  6714 tablet_bootstrap.cc:654] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.086155  6713 tablet_bootstrap.cc:492] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.087211  6713 ts_tablet_manager.cc:1403] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.097s	user 0.034s	sys 0.027s
I20260709 13:16:13.088675  6711 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.089308  6713 raft_consensus.cc:359] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.089737  6711 ts_tablet_manager.cc:1403] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.000s
I20260709 13:16:13.089996  6713 raft_consensus.cc:385] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.090301  6713 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.091032  6713 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.093205  6713 ts_tablet_manager.cc:1434] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:13.093264  6711 raft_consensus.cc:359] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.094060  6713 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.094180  6711 raft_consensus.cc:385] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.094517  6711 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.097750  6711 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.099663  6713 tablet_bootstrap.cc:654] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.100273  6711 ts_tablet_manager.cc:1434] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:16:13.101472  6711 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.130926  6711 tablet_bootstrap.cc:654] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.131821  6713 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.132346  6713 ts_tablet_manager.cc:1403] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:16:13.132638  6714 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.133098  6714 ts_tablet_manager.cc:1403] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.075s	user 0.022s	sys 0.011s
I20260709 13:16:13.135051  6713 raft_consensus.cc:359] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.135711  6713 raft_consensus.cc:385] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.135784  6711 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.136049  6713 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.135769  6714 raft_consensus.cc:359] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.136498  6711 ts_tablet_manager.cc:1403] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 13:16:13.136608  6714 raft_consensus.cc:385] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.136947  6714 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.136837  6713 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.137979  6843 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.137601  6714 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.138460  6843 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.139773  6714 ts_tablet_manager.cc:1434] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:13.140518  6843 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.141614  6713 ts_tablet_manager.cc:1434] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:16:13.141882  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.142545  6713 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
W20260709 13:16:13.143229  6561 leader_election.cc:343] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.144594  6711 raft_consensus.cc:359] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.145368  6711 raft_consensus.cc:385] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.145654  6711 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.146250  6843 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.146389  6711 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.146728  6843 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.148785  6713 tablet_bootstrap.cc:654] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.149186  6711 ts_tablet_manager.cc:1434] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:16:13.150012  6711 tablet_bootstrap.cc:492] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.153561  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.154207  6669 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:13.155570  6711 tablet_bootstrap.cc:654] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.155445  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.157109  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:13.157307  6563 leader_election.cc:343] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.157778  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:13.158552  6773 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:13.158990  6563 leader_election.cc:343] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.161029  6714 tablet_bootstrap.cc:492] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.161150  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:13.162437  6713 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.162869  6713 ts_tablet_manager.cc:1403] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.015s
I20260709 13:16:13.163036  6773 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.163405  6773 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.163726  6773 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.165675  6713 raft_consensus.cc:359] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.166359  6713 raft_consensus.cc:385] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.166603  6713 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.168087  6714 tablet_bootstrap.cc:654] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.167735  6713 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.168274  6711 tablet_bootstrap.cc:492] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.168700  6711 ts_tablet_manager.cc:1403] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.002s
I20260709 13:16:13.169608  6713 ts_tablet_manager.cc:1434] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:13.171396  6711 raft_consensus.cc:359] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.172273  6711 raft_consensus.cc:385] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.172602  6711 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.173476  6711 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.175951  6711 ts_tablet_manager.cc:1434] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:16:13.176877  6711 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.180256  6713 tablet_bootstrap.cc:492] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.180600  6843 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.182834  6711 tablet_bootstrap.cc:654] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.186210  6713 tablet_bootstrap.cc:654] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.192004  6740 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.192709  6740 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.194818  6740 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.196228  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.202351  6594 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:13.203604  6488 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.204617  6711 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.204699  6740 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.205221  6740 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.205346  6711 ts_tablet_manager.cc:1403] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.007s
I20260709 13:16:13.205576  6740 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.207150  6843 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.196660  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.197206  6759 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:16:13.208930  6711 raft_consensus.cc:359] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:13.209313  6486 leader_election.cc:343] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.209843  6711 raft_consensus.cc:385] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.210216  6711 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.211102  6843 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.216806  6740 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.217617  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.218241  6522 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:13.219337  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.221130  6845 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.221648  6845 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.221222  6711 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.222059  6845 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.223053  6843 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.226435  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:13.224273  6711 ts_tablet_manager.cc:1434] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:16:13.224547  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.228040  6711 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.225001  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:13.228658  6595 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.231261  6845 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.225330  6740 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.225865  6773 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:13.233362  6486 leader_election.cc:343] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:13.233440  6561 leader_election.cc:343] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.245424  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:13.237095  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:13.246047  6845 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:16:13.294229  6563 leader_election.cc:343] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.295481  6595 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:13.246050  6521 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.298108  6714 tablet_bootstrap.cc:492] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.298446  6714 ts_tablet_manager.cc:1403] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.138s	user 0.019s	sys 0.034s
I20260709 13:16:13.300504  6714 raft_consensus.cc:359] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.300922  6714 raft_consensus.cc:385] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.301185  6714 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.301847  6714 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.246378  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:13.303182  6488 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:16:13.303858  6561 leader_election.cc:343] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.304416  6714 ts_tablet_manager.cc:1434] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:13.304823  6813 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.305449  6813 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:13.305848  6713 tablet_bootstrap.cc:492] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.306322  6713 ts_tablet_manager.cc:1403] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.001s
I20260709 13:16:13.306514  6813 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.309226  6759 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:13.310125  6714 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.313077  6713 raft_consensus.cc:359] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.313769  6713 raft_consensus.cc:385] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.314085  6713 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.317378  6759 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:13.318423  6773 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.314909  6713 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.318959  6773 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.319386  6773 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.322577  6773 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.323371  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.323649  6711 tablet_bootstrap.cc:654] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.324111  6721 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:13.324841  6563 leader_election.cc:343] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.324573  6721 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.326764  6521 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:13.326875  6721 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.328009  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.328455  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.328630  6670 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:13.327656  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:13.329810  6669 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.332989  6714 tablet_bootstrap.cc:654] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.337069  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:16:13.338703  6637 leader_election.cc:343] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:13.339594  6637 consensus_peers.cc:597] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:13.340241  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.341238  6773 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.341599  6773 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.341917  6773 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.340859  6595 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:13.345940  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:13.346735  6759 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.347105  6759 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.347443  6759 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.349947  6669 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:13.351583  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:13.352727  6845 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.353175  6845 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:13.354288  6721 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.353982  6845 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.354761  6721 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.357189  6721 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.359702  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.360548  6520 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:13.362464  6713 ts_tablet_manager.cc:1434] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.000s
I20260709 13:16:13.364051  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:13.364465  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.365350  6843 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.375144  6595 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:13.368680  6773 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.378739  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:16:13.379958  6563 leader_election.cc:343] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.380009  6759 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.373267  6711 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.381050  6711 ts_tablet_manager.cc:1403] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.153s	user 0.010s	sys 0.009s
I20260709 13:16:13.375803  6843 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:13.382045  6759 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.383136  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.383541  6740 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.384195  6721 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.383977  6740 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.384543  6721 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.385701  6721 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.385998  6740 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.374843  6713 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.387812  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.388513  6594 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:13.389688  6488 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.384428  6843 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.395296  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:13.395598  6714 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.395871  6594 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.396214  6714 ts_tablet_manager.cc:1403] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.086s	user 0.022s	sys 0.039s
I20260709 13:16:13.390790  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:16:13.397432  6637 leader_election.cc:343] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.391479  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.398252  6669 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:13.398833  6714 raft_consensus.cc:359] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.399533  6714 raft_consensus.cc:385] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.391966  6740 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.392474  6713 tablet_bootstrap.cc:654] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.399909  6714 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.399760  6417 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.400686  6714 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.402801  6714 ts_tablet_manager.cc:1434] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:13.402922  6740 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.403136  6594 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:13.403689  6714 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.403348  6740 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:13.407009  6865 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:16:13.409210  6773 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.406530  6711 raft_consensus.cc:359] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.410303  6711 raft_consensus.cc:385] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.410575  6711 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.410851  6714 tablet_bootstrap.cc:654] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.411536  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:13.412142  6669 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.414183  6711 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.416957  6711 ts_tablet_manager.cc:1434] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:16:13.418085  6711 tablet_bootstrap.cc:492] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.425529  6813 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.426134  6721 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.426177  6813 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.427984  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:13.430164  6721 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.429193  6418 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.430776  6721 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.431452  6721 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:13.431250  6418 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.432423  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:13.432276  6721 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.432813  6669 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:13.434839  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:13.437216  6488 leader_election.cc:343] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.433331  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:13.437713  6714 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.438035  6521 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.438272  6714 ts_tablet_manager.cc:1403] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.012s
I20260709 13:16:13.438213  6594 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.440976  6714 raft_consensus.cc:359] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.441618  6714 raft_consensus.cc:385] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.441867  6714 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.442528  6714 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.444543  6521 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:13.444780  6714 ts_tablet_manager.cc:1434] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:13.445683  6714 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.445668  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.446811  6871 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.447463  6871 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:13.448100  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:13.435293  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.434145  6813 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.448313  6871 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.437680  6713 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.459662  6713 ts_tablet_manager.cc:1403] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.001s
I20260709 13:16:13.461503  6714 tablet_bootstrap.cc:654] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.461344  6776 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.462113  6776 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:13.462270  6713 raft_consensus.cc:359] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.434592  6740 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.463693  6713 raft_consensus.cc:385] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.464001  6713 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
W20260709 13:16:13.464545  6486 leader_election.cc:343] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.463303  6776 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.465039  6486 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:13.466847  6813 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:13.469452  6713 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.490059  6713 ts_tablet_manager.cc:1434] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:16:13.491379  6713 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.493410  6756 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.494035  6756 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.497143  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:13.498021  6596 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.499954  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.501006  6521 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:13.503386  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.503736  6756 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.505226  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:13.509624  6773 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:13.514710  6773 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:13.506688  6756 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.527170  6596 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:13.533502  6756 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:13.534371  6773 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.518373  6594 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:13.544893  6740 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.546936  6713 tablet_bootstrap.cc:654] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.547824  6773 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.550061  6488 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.550509  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:13.551085  6669 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.552906  6740 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.553579  6740 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:13.554528  6740 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.555776  6773 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.569746  6711 tablet_bootstrap.cc:654] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.574244  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.571962  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.579945  6714 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.580382  6714 ts_tablet_manager.cc:1403] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.135s	user 0.032s	sys 0.056s
I20260709 13:16:13.583114  6417 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.582309  6418 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:13.586447  6561 leader_election.cc:343] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:13.588686  6563 leader_election.cc:343] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.589125  6563 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:13.589871  6773 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:13.599588  6714 raft_consensus.cc:359] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.600471  6714 raft_consensus.cc:385] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.600876  6714 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.601606  6714 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.603227  6714 ts_tablet_manager.cc:1434] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:16:13.604435  6714 tablet_bootstrap.cc:492] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.609336  6845 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:13.616921  6669 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:13.638535  6713 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.639075  6713 ts_tablet_manager.cc:1403] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.148s	user 0.012s	sys 0.006s
I20260709 13:16:13.644189  6845 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:13.644492  6871 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.646777  6713 raft_consensus.cc:359] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.648118  6713 raft_consensus.cc:385] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.648576  6713 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.648114  6871 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.649339  6713 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.651216  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.651780  6871 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.651959  6522 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:13.653666  6711 tablet_bootstrap.cc:492] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.653888  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.654134  6711 ts_tablet_manager.cc:1403] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.236s	user 0.020s	sys 0.011s
I20260709 13:16:13.654523  6594 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:13.655959  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:13.655488  6420 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.656708  6871 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.657105  6871 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.657501  6871 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.657473  6711 raft_consensus.cc:359] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.658236  6711 raft_consensus.cc:385] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.658517  6711 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.660995  6711 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.664799  6871 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.667456  6711 ts_tablet_manager.cc:1434] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:16:13.667960  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:16:13.667418  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:13.668493  6711 tablet_bootstrap.cc:492] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.668576  6520 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.671033  6714 tablet_bootstrap.cc:654] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.651767  6713 ts_tablet_manager.cc:1434] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:16:13.675608  6713 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.679256  6520 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:13.681535  6713 tablet_bootstrap.cc:654] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.680179  6871 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.683311  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:13.683894  6594 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.684818  6711 tablet_bootstrap.cc:654] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.689373  6714 tablet_bootstrap.cc:492] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.689810  6714 ts_tablet_manager.cc:1403] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.086s	user 0.008s	sys 0.002s
I20260709 13:16:13.692378  6714 raft_consensus.cc:359] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.693023  6714 raft_consensus.cc:385] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.693302  6714 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.693923  6714 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.700361  6711 tablet_bootstrap.cc:492] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.700418  6714 ts_tablet_manager.cc:1434] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:16:13.700939  6711 ts_tablet_manager.cc:1403] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.001s
I20260709 13:16:13.701473  6714 tablet_bootstrap.cc:492] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.702661  6713 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.703379  6711 raft_consensus.cc:359] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.704229  6711 raft_consensus.cc:385] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.704313  6713 ts_tablet_manager.cc:1403] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 13:16:13.704629  6711 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.702486  6417 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:13.705302  6711 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.707473  6711 ts_tablet_manager.cc:1434] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:13.707683  6740 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.707757  6713 raft_consensus.cc:359] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.708339  6711 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.708125  6740 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.708616  6713 raft_consensus.cc:385] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.709002  6713 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.709813  6713 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.711622  6740 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.710738  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.711475  6594 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:13.712532  6596 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:13.714828  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.714919  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.715791  6871 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:13.716229  6871 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:13.716388  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:13.717003  6871 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.717453  6740 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.720077  6755 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.722265  6755 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.724841  6755 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.725060  6713 ts_tablet_manager.cc:1434] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:16:13.725955  6713 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.732553  6740 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:13.733491  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.734499  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.740752  6486 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:13.742578  6740 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:13.751803  6711 tablet_bootstrap.cc:654] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.756217  6714 tablet_bootstrap.cc:654] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.760151  6713 tablet_bootstrap.cc:654] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.763092  6711 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.763577  6711 ts_tablet_manager.cc:1403] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.000s
I20260709 13:16:13.765934  6714 tablet_bootstrap.cc:492] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.766373  6714 ts_tablet_manager.cc:1403] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 13:16:13.766115  6711 raft_consensus.cc:359] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.766829  6711 raft_consensus.cc:385] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.767105  6711 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.767733  6711 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.769006  6714 raft_consensus.cc:359] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.769512  6843 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:16:13.769191  6420 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:13.772102  6713 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.772528  6713 ts_tablet_manager.cc:1403] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 13:16:13.774945  6713 raft_consensus.cc:359] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.775669  6713 raft_consensus.cc:385] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.775941  6713 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.776604  6713 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.769626  6740 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:16:13.779109  6713 ts_tablet_manager.cc:1434] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:13.780030  6713 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.782342  6521 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.785765  6713 tablet_bootstrap.cc:654] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.787077  6670 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.795691  6843 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:13.770208  6711 ts_tablet_manager.cc:1434] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:13.769662  6714 raft_consensus.cc:385] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.801954  6714 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.802536  6711 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.803659  6714 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.809914  6711 tablet_bootstrap.cc:654] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.815523  6713 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.815974  6713 ts_tablet_manager.cc:1403] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.015s
I20260709 13:16:13.818423  6713 raft_consensus.cc:359] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.819078  6713 raft_consensus.cc:385] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.819319  6713 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.819964  6713 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.821915  6713 ts_tablet_manager.cc:1434] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:16:13.822893  6713 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.828508  6711 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:13.829010  6711 ts_tablet_manager.cc:1403] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.004s
I20260709 13:16:13.831534  6711 raft_consensus.cc:359] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.832170  6711 raft_consensus.cc:385] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.832414  6711 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.833012  6711 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.834770  6711 ts_tablet_manager.cc:1434] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:16:13.835528  6670 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.836337  6714 ts_tablet_manager.cc:1434] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.001s
I20260709 13:16:13.836495  6713 tablet_bootstrap.cc:654] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.839980  6714 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.848683  6740 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:13.855213  6711 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:13.860351  6713 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.860812  6713 ts_tablet_manager.cc:1403] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.014s
I20260709 13:16:13.861382  6711 tablet_bootstrap.cc:654] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.864557  6843 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.864495  6713 raft_consensus.cc:359] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.865170  6843 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.865700  6713 raft_consensus.cc:385] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.866086  6713 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.867197  6843 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.866927  6713 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.869186  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:13.870690  6563 leader_election.cc:343] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.870849  6713 ts_tablet_manager.cc:1434] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:16:13.846565  6714 tablet_bootstrap.cc:654] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.872043  6713 tablet_bootstrap.cc:492] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.877475  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.880899  6713 tablet_bootstrap.cc:654] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.883188  6721 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:13.884505  6721 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:13.892517  6561 leader_election.cc:343] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.892959  6561 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:13.893567  6714 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:13.894129  6714 ts_tablet_manager.cc:1403] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.057s	user 0.004s	sys 0.011s
I20260709 13:16:13.895926  6845 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:13.897476  6756 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.897933  6756 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.898150  6714 raft_consensus.cc:359] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.899051  6714 raft_consensus.cc:385] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.899375  6714 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.900007  6756 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:13.902547  6713 tablet_bootstrap.cc:492] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:13.902509  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.903276  6713 ts_tablet_manager.cc:1403] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.005s
W20260709 13:16:13.904194  6637 leader_election.cc:343] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:13.906093  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.906680  6713 raft_consensus.cc:359] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.907543  6713 raft_consensus.cc:385] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:13.907876  6713 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:13.908627  6713 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.911372  6713 ts_tablet_manager.cc:1434] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:16:13.911715  6843 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.912362  6713 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:13.912173  6843 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.915216  6843 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.906719  6596 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:13.920805  6713 tablet_bootstrap.cc:654] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.921761  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.922955  6714 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.925016  6714 ts_tablet_manager.cc:1434] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:16:13.926237  6714 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:13.927009  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:13.927685  6521 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:13.929138  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:13.930526  6845 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.930902  6845 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.931206  6845 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.934113  6714 tablet_bootstrap.cc:654] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:13.956717  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:13.958591  6759 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:13.958981  6759 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.959296  6759 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:13.978806  6813 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.979348  6813 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.979771  6756 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:13.981973  6813 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.982026  6740 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.982594  6740 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:13.984400  6740 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:13.991864  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.995568  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:13.995132  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:13.998996  6669 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:14.001796  6714 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:14.002279  6714 ts_tablet_manager.cc:1403] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.076s	user 0.015s	sys 0.016s
I20260709 13:16:14.002352  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:14.010540  6713 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.011014  6713 ts_tablet_manager.cc:1403] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.099s	user 0.016s	sys 0.033s
I20260709 13:16:13.999650  6845 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.014568  6713 raft_consensus.cc:359] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.015179  6713 raft_consensus.cc:385] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.017464  6520 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.017554  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:14.017860  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:14.018442  6522 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.018862  6671 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:16:14.018983  6561 leader_election.cc:343] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.020082  6843 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.022100  6759 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.015437  6713 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.023034  6740 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.023684  6713 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.024196  6845 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.024798  6486 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:14.028086  6813 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.028870  6711 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.029299  6711 ts_tablet_manager.cc:1403] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.174s	user 0.064s	sys 0.070s
I20260709 13:16:14.030097  6740 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.031234  6871 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.031390  6776 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.031226  6843 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.032424  6843 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.033092  6759 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.034385  6843 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.035686  6813 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.035862  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.038870  6519 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:14.040243  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:14.038138  6522 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:14.042785  6776 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.043167  6776 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.043489  6776 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.049674  6714 raft_consensus.cc:359] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.050455  6714 raft_consensus.cc:385] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.049674  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:14.051663  6843 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.052287  6843 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:14.053184  6843 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.054240  6714 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.055227  6714 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.057591  6714 ts_tablet_manager.cc:1434] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260709 13:16:14.058584  6714 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:14.058651  6845 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:14.062422  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:14.063062  6669 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.055121  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:14.065141  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:14.065830  6670 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:14.037159  6711 raft_consensus.cc:359] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.071619  6711 raft_consensus.cc:385] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.071834  6711 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.072285  6711 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:14.071079  6637 leader_election.cc:343] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.074561  6669 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:14.075344  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:14.075585  6713 ts_tablet_manager.cc:1434] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 13:16:14.075937  6595 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.076555  6713 tablet_bootstrap.cc:492] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:14.082754  6595 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.083658  6776 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.093551  6776 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.095382  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:14.096102  6522 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.084612  6714 tablet_bootstrap.cc:654] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.089567  6773 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:16:14.085856  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:14.104094  6909 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.104699  6909 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:14.107146  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:14.107892  6670 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.113394  6740 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.105813  6909 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.141584  6713 tablet_bootstrap.cc:654] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.164028  6776 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.179065  6522 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:14.175654  6776 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.181555  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:14.183269  6845 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.183764  6845 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:14.191020  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:14.191743  6845 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:14.192605  6561 leader_election.cc:343] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.197075  6714 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:14.197870  6670 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:14.182474  6776 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.205673  6714 ts_tablet_manager.cc:1403] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.147s	user 0.006s	sys 0.012s
I20260709 13:16:14.204262  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.221671  6714 raft_consensus.cc:359] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.226496  6756 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:14.231734  6714 raft_consensus.cc:385] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.232097  6714 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.238327  6711 ts_tablet_manager.cc:1434] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.209s	user 0.038s	sys 0.076s
I20260709 13:16:14.239293  6711 tablet_bootstrap.cc:492] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.232985  6714 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:14.255076  6563 leader_election.cc:343] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.255573  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:14.244196  6871 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:16:14.259335  6714 ts_tablet_manager.cc:1434] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.003s
I20260709 13:16:14.260197  6714 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:14.263370  6845 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.275532  6909 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.287565  6740 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.288003  6740 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.289721  6740 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.293219  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:14.300711  6711 tablet_bootstrap.cc:654] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.298020  6417 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.305308  6417 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.306804  6713 tablet_bootstrap.cc:492] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.307461  6713 ts_tablet_manager.cc:1403] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.231s	user 0.018s	sys 0.011s
W20260709 13:16:14.295349  6486 leader_election.cc:343] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.309432  6721 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.298720  6909 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.319078  6721 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.325291  6721 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.331344  6711 tablet_bootstrap.cc:492] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.331825  6711 ts_tablet_manager.cc:1403] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.093s	user 0.003s	sys 0.011s
I20260709 13:16:14.337462  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.338122  6594 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:14.339015  6713 raft_consensus.cc:359] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.339700  6713 raft_consensus.cc:385] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.339962  6713 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.340215  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.340782  6521 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:14.340566  6713 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.340922  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.341758  6594 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:14.342643  6713 ts_tablet_manager.cc:1434] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.000s
I20260709 13:16:14.343458  6713 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:14.345033  6909 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.349327  6637 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:14.351665  6756 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.352058  6756 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.337001  6711 raft_consensus.cc:359] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.352412  6756 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.352640  6711 raft_consensus.cc:385] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.352921  6711 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.353555  6740 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.353963  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.355580  6488 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:14.354001  6740 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.356052  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.356652  6595 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:14.357465  6740 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.357901  6740 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.358177  6740 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.358469  6740 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:14.359346  6637 leader_election.cc:343] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.360906  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:14.361433  6714 tablet_bootstrap.cc:654] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.361714  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.362547  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:14.363360  6871 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.363803  6871 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:14.363834  6486 leader_election.cc:343] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.364210  6871 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.363838  6596 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:14.353536  6711 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.366044  6756 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.367889  6756 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.368012  6711 ts_tablet_manager.cc:1434] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:16:14.368763  6711 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.369954  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:14.370697  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:14.370805  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:14.371632  6813 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.371995  6813 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.372314  6813 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.370740  6596 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.374976  6740 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.374593  6521 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.376386  6845 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.376876  6845 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.380040  6740 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.381749  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.382557  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:14.381582  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:14.384440  6595 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:14.387461  6486 leader_election.cc:343] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:14.386543  6563 leader_election.cc:343] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.396755  6711 tablet_bootstrap.cc:654] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.401378  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:14.402544  6561 leader_election.cc:343] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.403007  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:14.403776  6776 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.408341  6595 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:14.408476  6521 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.398051  6845 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.414224  6596 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.414224  6871 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.416741  6871 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.418051  6813 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.418408  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:14.418963  6594 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.420603  6637 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:14.421624  6909 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.422106  6909 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:14.422960  6909 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.424961  6813 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.426456  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:16:14.429318  6486 leader_election.cc:343] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.430105  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:14.431680  6488 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:14.432813  6813 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.425583  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:14.433449  6813 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:14.431169  6596 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.435196  6813 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:14.437510  6637 leader_election.cc:343] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:14.439280  6594 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.441300  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:14.441565  6713 tablet_bootstrap.cc:654] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.442646  6756 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.443166  6756 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:14.446380  6756 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.450626  6714 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:14.451094  6714 ts_tablet_manager.cc:1403] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.191s	user 0.009s	sys 0.021s
I20260709 13:16:14.451953  6711 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.452358  6711 ts_tablet_manager.cc:1403] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.011s
I20260709 13:16:14.455147  6711 raft_consensus.cc:359] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.455816  6711 raft_consensus.cc:385] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.456051  6711 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.457353  6714 raft_consensus.cc:359] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.457999  6714 raft_consensus.cc:385] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.458237  6714 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.456604  6711 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.458840  6714 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.461315  6714 ts_tablet_manager.cc:1434] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 13:16:14.462188  6714 tablet_bootstrap.cc:492] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:14.464430  6711 ts_tablet_manager.cc:1434] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:16:14.465197  6711 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.453027  6596 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:14.468743  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:14.470562  6740 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.471007  6740 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:14.471731  6740 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.474454  6845 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.476190  6420 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.476280  6416 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:14.493954  6711 tablet_bootstrap.cc:654] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.499187  6714 tablet_bootstrap.cc:654] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.515563  6843 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.516687  6713 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.517143  6713 ts_tablet_manager.cc:1403] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.174s	user 0.011s	sys 0.005s
I20260709 13:16:14.521111  6713 raft_consensus.cc:359] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.521840  6713 raft_consensus.cc:385] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.522224  6713 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.523176  6713 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.531421  6713 ts_tablet_manager.cc:1434] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:16:14.535220  6713 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:14.557592  6714 tablet_bootstrap.cc:492] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:14.558099  6714 ts_tablet_manager.cc:1403] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.096s	user 0.024s	sys 0.024s
I20260709 13:16:14.560662  6714 raft_consensus.cc:359] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.561295  6714 raft_consensus.cc:385] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.561556  6714 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.562201  6714 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.564288  6714 ts_tablet_manager.cc:1434] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:14.565224  6714 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:14.565672  6713 tablet_bootstrap.cc:654] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.570927  6714 tablet_bootstrap.cc:654] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.583168  6711 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.583776  6711 ts_tablet_manager.cc:1403] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.010s
I20260709 13:16:14.586918  6711 raft_consensus.cc:359] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.587903  6711 raft_consensus.cc:385] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.588243  6711 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.589110  6711 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.592578  6713 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.592866  6711 ts_tablet_manager.cc:1434] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:16:14.593335  6713 ts_tablet_manager.cc:1403] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.016s
I20260709 13:16:14.594218  6909 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.593871  6416 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.594887  6909 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.596469  6713 raft_consensus.cc:359] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.596972  6711 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.597225  6713 raft_consensus.cc:385] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.597548  6713 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.597518  6909 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.598646  6713 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.601235  6713 ts_tablet_manager.cc:1434] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:14.602187  6713 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:14.602159  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.603195  6596 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:14.601222  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.607501  6521 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:14.608227  6711 tablet_bootstrap.cc:654] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.609674  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:14.610591  6713 tablet_bootstrap.cc:654] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.611346  6909 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.611691  6909 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.612043  6909 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.613240  6714 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:14.613761  6714 ts_tablet_manager.cc:1403] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.015s
I20260709 13:16:14.617343  6714 raft_consensus.cc:359] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.618229  6714 raft_consensus.cc:385] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.618554  6714 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.619324  6714 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.621363  6714 ts_tablet_manager.cc:1434] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:14.622321  6714 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:14.625413  6711 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.626013  6711 ts_tablet_manager.cc:1403] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.029s	user 0.001s	sys 0.020s
I20260709 13:16:14.626991  6909 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.629189  6714 tablet_bootstrap.cc:654] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.629050  6711 raft_consensus.cc:359] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.629784  6711 raft_consensus.cc:385] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.630084  6711 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.629915  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:14.630753  6520 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.630945  6711 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.633387  6711 ts_tablet_manager.cc:1434] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:16:14.634214  6711 tablet_bootstrap.cc:492] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.639901  6711 tablet_bootstrap.cc:654] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.640789  6714 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:14.641264  6714 ts_tablet_manager.cc:1403] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.010s
I20260709 13:16:14.641706  6417 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:14.643923  6714 raft_consensus.cc:359] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.644660  6714 raft_consensus.cc:385] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.644927  6714 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.644883  6417 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:14.648001  6714 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.643705  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:14.651008  6596 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.647083  6909 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.661613  6596 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.662062  6714 ts_tablet_manager.cc:1434] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:16:14.662134  6520 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.662739  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:14.663538  6714 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:14.666500  6909 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.666998  6909 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:14.672191  6909 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.673731  6711 tablet_bootstrap.cc:492] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.674196  6711 ts_tablet_manager.cc:1403] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.016s
I20260709 13:16:14.675990  6713 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.676285  6713 ts_tablet_manager.cc:1403] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.074s	user 0.021s	sys 0.008s
I20260709 13:16:14.678889  6713 raft_consensus.cc:359] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.679049  6711 raft_consensus.cc:359] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.679558  6713 raft_consensus.cc:385] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.679625  6711 raft_consensus.cc:385] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.679869  6713 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.679951  6711 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.680325  6714 tablet_bootstrap.cc:654] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.680594  6711 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.680735  6713 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.683651  6740 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.684181  6740 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.686124  6740 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.687249  6713 ts_tablet_manager.cc:1434] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:16:14.688512  6713 tablet_bootstrap.cc:492] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:14.690101  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:14.690646  6843 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.694075  6713 tablet_bootstrap.cc:654] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.697587  6721 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.698191  6721 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.699731  6711 ts_tablet_manager.cc:1434] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:16:14.700717  6721 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.700702  6711 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.705084  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.706084  6594 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:14.708170  6711 tablet_bootstrap.cc:654] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.709506  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.709861  6522 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.715660  6713 tablet_bootstrap.cc:492] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.716188  6713 ts_tablet_manager.cc:1403] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.002s
I20260709 13:16:14.717039  6759 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.717527  6759 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.719527  6759 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.719372  6713 raft_consensus.cc:359] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.720059  6713 raft_consensus.cc:385] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.720319  6713 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.720710  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.720955  6713 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.721311  6521 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:14.722503  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:14.723268  6713 ts_tablet_manager.cc:1434] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:16:14.723505  6756 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.723827  6756 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.724181  6713 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:14.724164  6756 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.727133  6711 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.727279  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.728039  6711 ts_tablet_manager.cc:1403] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20260709 13:16:14.728283  6594 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:14.731029  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.730949  6711 raft_consensus.cc:359] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.731760  6711 raft_consensus.cc:385] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.731838  6596 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:14.732067  6711 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.732786  6711 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.732990  6756 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.734387  6488 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:14.734880  6711 ts_tablet_manager.cc:1434] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:14.734887  6756 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:14.735967  6711 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.742444  6713 tablet_bootstrap.cc:654] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.744460  6711 tablet_bootstrap.cc:654] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.730163  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:14.735530  6756 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.745406  6871 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.750854  6713 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.751499  6713 ts_tablet_manager.cc:1403] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.004s
I20260709 13:16:14.752791  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:14.753615  6521 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.754557  6713 raft_consensus.cc:359] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.755214  6713 raft_consensus.cc:385] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.755709  6711 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.755470  6713 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.756194  6711 ts_tablet_manager.cc:1403] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:16:14.756428  6713 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.758702  6713 ts_tablet_manager.cc:1434] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:14.758708  6711 raft_consensus.cc:359] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.759450  6711 raft_consensus.cc:385] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.759657  6713 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:14.759703  6711 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.761070  6711 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.763482  6711 ts_tablet_manager.cc:1434] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:14.764392  6711 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:14.765861  6843 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:14.768553  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:14.769099  6596 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.777115  6711 tablet_bootstrap.cc:654] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.780445  6713 tablet_bootstrap.cc:654] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:14.784123  6596 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.788777  6521 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:14.792320  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:14.795745  6740 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.796348  6740 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.806044  6740 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.806114  6813 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.795974  6756 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:14.799517  6871 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:14.819980  6756 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:14.830078  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:14.831482  6596 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:14.831740  6756 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.839134  6488 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:14.840716  6740 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:14.841177  6740 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.841574  6740 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:14.893926  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:14.901157  6740 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.927438  6915 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.928234  6915 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.933157  6711 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:14.936344  6711 ts_tablet_manager.cc:1403] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.172s	user 0.019s	sys 0.024s
I20260709 13:16:14.937459  6843 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.938794  6843 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.947348  6714 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:14.947839  6714 ts_tablet_manager.cc:1403] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.285s	user 0.033s	sys 0.040s
I20260709 13:16:14.951094  6714 raft_consensus.cc:359] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.951952  6714 raft_consensus.cc:385] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.952277  6714 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.954262  6714 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.955298  6845 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.964345  6713 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:14.964969  6713 ts_tablet_manager.cc:1403] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.206s	user 0.024s	sys 0.001s
I20260709 13:16:14.963339  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:14.967348  6915 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.988658  6711 raft_consensus.cc:359] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.989221  6711 raft_consensus.cc:385] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:14.989460  6711 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:14.989944  6711 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.969630  6845 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:14.996477  6843 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:14.998500  6845 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.000452  6845 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.000788  6845 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:15.002274  6486 leader_election.cc:343] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.003077  6711 ts_tablet_manager.cc:1434] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.066s	user 0.004s	sys 0.003s
I20260709 13:16:15.004071  6714 ts_tablet_manager.cc:1434] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.056s	user 0.015s	sys 0.016s
W20260709 13:16:15.007841  6563 leader_election.cc:343] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.009721  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:15.010396  6596 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.019657  6711 tablet_bootstrap.cc:492] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:15.019567  6845 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.024098  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:15.032862  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.035842  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.037077  6711 tablet_bootstrap.cc:654] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.037585  6740 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.038335  6759 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.038877  6759 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.039783  6714 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:15.041661  6596 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
W20260709 13:16:15.048942  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:15.050949  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.052250  6714 tablet_bootstrap.cc:654] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.053045  6711 tablet_bootstrap.cc:492] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:15.053500  6711 ts_tablet_manager.cc:1403] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.004s
I20260709 13:16:15.054215  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.054877  6664 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:15.056471  6711 raft_consensus.cc:359] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.057143  6711 raft_consensus.cc:385] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.057019  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.057454  6711 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.058607  6711 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.059484  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.060900  6960 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.062315  6711 ts_tablet_manager.cc:1434] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:16:15.063488  6711 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:15.064742  6714 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:15.065307  6714 ts_tablet_manager.cc:1403] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.000s
I20260709 13:16:15.041576  6759 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.042949  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:15.044236  6563 leader_election.cc:343] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.068328  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:15.068493  6563 leader_election.cc:343] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.069108  6596 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:15.045869  6756 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.069272  6711 tablet_bootstrap.cc:654] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.070124  6756 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.072713  6756 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:16:15.074332  6563 leader_election.cc:343] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:15.073843  6486 leader_election.cc:343] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.075426  6740 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.076987  6714 raft_consensus.cc:359] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.078715  6714 raft_consensus.cc:385] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.078521  6713 raft_consensus.cc:359] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.079118  6714 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.079293  6713 raft_consensus.cc:385] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.079627  6713 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.079934  6714 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.080250  6713 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.085439  6714 ts_tablet_manager.cc:1434] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:16:15.086180  6843 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.086740  6843 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.088811  6843 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.095078  6488 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:15.102696  6714 tablet_bootstrap.cc:492] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:15.103178  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.104682  6711 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
W20260709 13:16:15.104993  6563 leader_election.cc:343] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.105180  6711 ts_tablet_manager.cc:1403] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.013s
I20260709 13:16:15.107808  6711 raft_consensus.cc:359] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.109006  6664 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:15.111056  6561 leader_election.cc:343] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.111496  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:15.112831  6843 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.114867  6813 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.115340  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:15.116254  6714 tablet_bootstrap.cc:654] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.116328  6843 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.116856  6843 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.117213  6843 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.082455  6713 ts_tablet_manager.cc:1434] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.117s	user 0.005s	sys 0.003s
W20260709 13:16:15.123615  6561 leader_election.cc:343] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.124087  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:15.124871  6845 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.125631  6713 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:15.129355  6714 tablet_bootstrap.cc:492] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:15.129964  6714 ts_tablet_manager.cc:1403] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20260709 13:16:15.109102  6711 raft_consensus.cc:385] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.132135  6711 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
W20260709 13:16:15.132889  6561 leader_election.cc:343] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.133256  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:15.138182  6714 raft_consensus.cc:359] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.138876  6714 raft_consensus.cc:385] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.135411  6759 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:16:15.139132  6714 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.137284  6845 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.140548  6714 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.143432  6740 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.131879  6813 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:15.145610  6813 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.151654  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:15.152258  6595 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:15.149335  6711 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.157763  6714 ts_tablet_manager.cc:1434] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:16:15.156255  6968 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.158864  6711 ts_tablet_manager.cc:1434] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:16:15.159736  6711 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:15.160053  6714 tablet_bootstrap.cc:492] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:15.161559  6963 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:15.165956  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:15.166709  6963 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.167143  6963 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.167451  6963 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.170421  6714 tablet_bootstrap.cc:654] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.166462  6713 tablet_bootstrap.cc:654] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.172505  6963 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.164762  6968 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:16:15.177143  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.191831  6963 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.193466  6843 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.177925  6756 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:16:15.198050  6711 tablet_bootstrap.cc:654] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.203305  6969 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.205511  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:15.206249  6597 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.203763  6969 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:15.221506  6561 leader_election.cc:343] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.223361  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:15.221401  6843 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.224995  6776 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.247593  6969 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.226775  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:15.251598  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.253641  6714 tablet_bootstrap.cc:492] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:15.255640  6669 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.255299  6416 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.256258  6713 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:15.263259  6713 ts_tablet_manager.cc:1403] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.138s	user 0.009s	sys 0.023s
I20260709 13:16:15.256946  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:15.261293  6714 ts_tablet_manager.cc:1403] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.103s	user 0.022s	sys 0.024s
W20260709 13:16:15.265391  6563 leader_election.cc:343] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.272922  6756 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.273269  6713 raft_consensus.cc:359] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.274065  6713 raft_consensus.cc:385] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.274346  6713 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.273711  6756 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:15.279588  6970 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:16:15.282819  6963 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.286175  6522 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.287804  6714 raft_consensus.cc:359] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.288465  6714 raft_consensus.cc:385] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.289064  6714 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.290323  6714 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.292239  6714 ts_tablet_manager.cc:1434] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.000s
I20260709 13:16:15.293308  6714 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:15.297448  6713 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.301817  6669 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:15.285079  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.303601  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:15.305361  6845 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.305998  6845 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:15.307179  6845 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.312754  6713 ts_tablet_manager.cc:1434] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 13:16:15.313382  6759 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.313867  6759 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.315939  6759 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.316133  6714 tablet_bootstrap.cc:654] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.321697  6713 tablet_bootstrap.cc:492] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:15.323495  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:15.324144  6595 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:15.327553  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:15.318774  6597 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:15.316718  6756 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.333230  6759 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.334190  6759 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.334570  6759 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.334057  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:15.334690  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:15.336527  6593 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:15.336546  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.337445  6521 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:15.337884  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:15.339257  6909 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.339859  6909 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.340646  6909 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.338955  6756 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:16:15.347653  6976 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:16:15.344899  6756 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:15.354575  6711 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:15.355069  6711 ts_tablet_manager.cc:1403] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.196s	user 0.006s	sys 0.009s
I20260709 13:16:15.362057  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:15.362680  6520 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.365233  6711 raft_consensus.cc:359] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.366758  6711 raft_consensus.cc:385] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.367033  6711 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.369096  6756 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.374022  6711 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.374363  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.379886  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.383428  6517 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:15.384218  6520 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:15.367096  6909 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.390311  6845 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.390940  6845 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.419294  6711 ts_tablet_manager.cc:1434] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.064s	user 0.000s	sys 0.007s
I20260709 13:16:15.421365  6711 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:15.424224  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:15.420011  6909 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.424594  6713 tablet_bootstrap.cc:654] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.425498  6845 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.430366  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.427436  6759 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.430233  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:15.439057  6637 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:15.446018  6714 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:15.446477  6714 ts_tablet_manager.cc:1403] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.153s	user 0.008s	sys 0.007s
I20260709 13:16:15.438302  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.448788  6959 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:16:15.445358  6592 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.444590  6909 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.452569  6519 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:15.457691  6759 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.465493  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:15.465194  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:15.467121  6714 raft_consensus.cc:359] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.468413  6714 raft_consensus.cc:385] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.468655  6714 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.469358  6518 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.469463  6909 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.468528  6845 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.471117  6845 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.471201  6909 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.471495  6845 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:15.474002  6637 leader_election.cc:343] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.474545  6714 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.477026  6714 ts_tablet_manager.cc:1434] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:16:15.477505  6845 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.478008  6714 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:15.478058  6518 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:15.479331  6845 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.480131  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:15.480997  6519 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.484977  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:15.491425  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:15.492046  6593 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.492208  6714 tablet_bootstrap.cc:654] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:15.493947  6561 leader_election.cc:343] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.479337  6488 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:15.495253  6969 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:15.494684  6561 leader_election.cc:343] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.497952  6592 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:15.498241  6519 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:15.499423  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:15.500141  6845 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.500524  6711 tablet_bootstrap.cc:654] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.500916  6845 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:15.503368  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:15.504639  6963 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.505136  6963 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:15.505954  6963 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.507786  6845 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.512913  6713 tablet_bootstrap.cc:492] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:15.513509  6713 ts_tablet_manager.cc:1403] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.192s	user 0.008s	sys 0.013s
I20260709 13:16:15.517892  6713 raft_consensus.cc:359] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.518764  6713 raft_consensus.cc:385] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.519129  6713 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.525307  6417 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.529649  6417 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.534309  6713 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.537544  6713 ts_tablet_manager.cc:1434] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:16:15.539646  6713 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:15.536378  6593 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:15.551437  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:15.552075  6519 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.562552  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:15.563738  6909 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:16:15.565161  6963 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.584395  6714 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:15.585258  6714 ts_tablet_manager.cc:1403] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.107s	user 0.022s	sys 0.001s
I20260709 13:16:15.597664  6713 tablet_bootstrap.cc:654] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.604722  6519 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:15.617992  6714 raft_consensus.cc:359] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.621973  6714 raft_consensus.cc:385] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.622294  6714 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.589252  6963 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:15.624688  6711 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:15.627079  6711 ts_tablet_manager.cc:1403] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.206s	user 0.014s	sys 0.008s
I20260709 13:16:15.629544  6714 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.633688  6714 ts_tablet_manager.cc:1434] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.002s
I20260709 13:16:15.636281  6714 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:15.636626  6711 raft_consensus.cc:359] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.637925  6711 raft_consensus.cc:385] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.638278  6711 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.641052  6711 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.645591  6711 ts_tablet_manager.cc:1434] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 13:16:15.676193  6773 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.676795  6773 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.680366  6773 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.682554  6711 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:15.691381  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.691480  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.708375  6711 tablet_bootstrap.cc:654] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.703925  6417 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.709987  6417 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:15.711164  6713 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:15.711640  6713 ts_tablet_manager.cc:1403] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.172s	user 0.019s	sys 0.031s
I20260709 13:16:15.708564  6963 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.718238  6714 tablet_bootstrap.cc:654] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:15.718721  6561 leader_election.cc:343] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:15.719300  6563 leader_election.cc:343] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.719722  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:15.721166  6843 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:15.726914  6713 raft_consensus.cc:359] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.730458  6713 raft_consensus.cc:385] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.731359  6713 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.732566  6713 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.735994  6713 ts_tablet_manager.cc:1434] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:16:15.737967  6963 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.739238  6963 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.738320  6522 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.747701  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:15.749033  6637 leader_election.cc:343] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.751917  6963 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.752171  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:15.752862  6595 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:15.753715  6711 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:15.754175  6711 ts_tablet_manager.cc:1403] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.019s
I20260709 13:16:15.753623  6416 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.754601  6714 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:15.755371  6416 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.756757  6711 raft_consensus.cc:359] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.757871  6711 raft_consensus.cc:385] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.758069  6713 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:15.758127  6711 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.758293  6416 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:15.758878  6711 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.759099  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:15.760200  6963 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.760601  6963 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.760941  6963 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.755435  6714 ts_tablet_manager.cc:1403] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.119s	user 0.016s	sys 0.003s
I20260709 13:16:15.764360  6967 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.764997  6776 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.765633  6711 ts_tablet_manager.cc:1434] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:16:15.766502  6711 tablet_bootstrap.cc:492] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:15.774372  6714 raft_consensus.cc:359] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.775116  6714 raft_consensus.cc:385] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.775367  6714 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.776006  6714 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.776684  6963 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.781713  6713 tablet_bootstrap.cc:654] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.784449  6711 tablet_bootstrap.cc:654] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.797938  6714 ts_tablet_manager.cc:1434] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 13:16:15.805101  6714 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:15.791123  6669 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.786194  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:15.808152  6776 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.808490  6595 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.784549  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:15.808583  6776 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:15.809921  6637 leader_election.cc:343] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.810727  6776 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.806102  6995 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.811365  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:15.785482  6963 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:15.806833  6915 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.815610  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:15.815567  6740 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.817157  6740 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.819614  6740 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:15.823009  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:15.823079  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:15.817744  6561 leader_election.cc:343] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.830296  6714 tablet_bootstrap.cc:654] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:15.830451  6740 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.833971  6595 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
W20260709 13:16:15.834086  6486 leader_election.cc:343] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:15.839607  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:15.828449  6518 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:15.842437  6963 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:15.842834  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:15.842962  6963 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:15.823839  6592 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:15.845152  6963 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.846032  6915 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.850409  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:15.851977  6740 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:15.852469  6740 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.852851  6740 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.866518  6915 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.895037  6915 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:15.926720  6713 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:15.927407  6713 ts_tablet_manager.cc:1403] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.170s	user 0.012s	sys 0.020s
I20260709 13:16:15.932119  6959 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:15.928231  6909 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:15.938774  6713 raft_consensus.cc:359] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.939729  6713 raft_consensus.cc:385] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.940075  6713 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.940855  6713 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:15.949304  6561 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:15.953800  6714 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:15.973069  6711 tablet_bootstrap.cc:492] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:15.978120  6711 ts_tablet_manager.cc:1403] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.212s	user 0.049s	sys 0.053s
I20260709 13:16:15.977458  6714 ts_tablet_manager.cc:1403] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.173s	user 0.015s	sys 0.000s
I20260709 13:16:15.980741  6711 raft_consensus.cc:359] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.981433  6711 raft_consensus.cc:385] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.981700  6711 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.982414  6711 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.982700  6714 raft_consensus.cc:359] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.983387  6714 raft_consensus.cc:385] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:15.983673  6714 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:15.983979  6711 ts_tablet_manager.cc:1434] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:16:15.984941  6711 tablet_bootstrap.cc:492] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:15.957453  6713 ts_tablet_manager.cc:1434] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:16:15.989704  6713 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:15.989298  6915 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.955040  6740 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:15.995824  6963 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:15.998358  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:15.999382  6520 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:16.000828  6714 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.002708  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:16.003619  6595 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:16.008037  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:16:16.009380  6486 leader_election.cc:343] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.009830  6486 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:16.010954  6969 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:16.021077  6740 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.022540  6714 ts_tablet_manager.cc:1434] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.000s
I20260709 13:16:16.023659  6714 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:16.027017  6711 tablet_bootstrap.cc:654] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.027448  6713 tablet_bootstrap.cc:654] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.009488  6915 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.035693  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:16.042306  6713 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:16.042826  6713 ts_tablet_manager.cc:1403] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.004s
I20260709 13:16:16.042938  6714 tablet_bootstrap.cc:654] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.045740  6713 raft_consensus.cc:359] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.046272  6713 raft_consensus.cc:385] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.046500  6713 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.047238  6713 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.049229  6711 tablet_bootstrap.cc:492] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:16.049510  6714 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:16.049702  6713 ts_tablet_manager.cc:1434] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:16.050045  6711 ts_tablet_manager.cc:1403] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.012s
I20260709 13:16:16.050145  6714 ts_tablet_manager.cc:1403] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.004s
I20260709 13:16:16.050640  6713 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
W20260709 13:16:16.092521  6561 leader_election.cc:343] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.093467  6561 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:16.114359  6773 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:16.133352  6773 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.134256  6773 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.145545  6773 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.150265  6711 raft_consensus.cc:359] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.151284  6711 raft_consensus.cc:385] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.151667  6711 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.151412  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:16.152539  6711 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.150305  6714 raft_consensus.cc:359] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.157915  6714 raft_consensus.cc:385] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.162259  6714 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.162987  6714 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:16.174324  6561 leader_election.cc:343] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.182548  6713 tablet_bootstrap.cc:654] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.182703  6714 ts_tablet_manager.cc:1434] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.132s	user 0.008s	sys 0.003s
I20260709 13:16:16.184422  6714 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:16.190739  6711 ts_tablet_manager.cc:1434] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.140s	user 0.012s	sys 0.000s
I20260709 13:16:16.192384  6711 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:16.221081  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:16.224339  6711 tablet_bootstrap.cc:654] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.227306  6420 catalog_manager.cc:5697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:16.231721  6714 tablet_bootstrap.cc:654] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.233950  6713 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:16.235080  6713 ts_tablet_manager.cc:1403] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.185s	user 0.031s	sys 0.005s
I20260709 13:16:16.238574  6711 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:16.259938  6711 ts_tablet_manager.cc:1403] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.013s
I20260709 13:16:16.274452  6773 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.275310  6773 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.283726  6813 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.284158  6773 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.284296  6813 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.281803  6843 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:16:16.287492  6813 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.292296  6963 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.292878  6963 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.294811  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:16.296983  6963 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:16.298822  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:16.301209  6563 leader_election.cc:343] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.301769  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:16.302821  6845 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:16.303820  6486 leader_election.cc:343] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.306773  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:16.307389  6561 leader_election.cc:343] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.267311  7026 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 13:16:16.316315  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:16.317009  6520 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:16.317559  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:16.318269  6595 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:16.318737  6713 raft_consensus.cc:359] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.319553  6713 raft_consensus.cc:385] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.319831  6713 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.320581  6713 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.329066  6713 ts_tablet_manager.cc:1434] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.013s
I20260709 13:16:16.330168  6711 raft_consensus.cc:359] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.330869  6711 raft_consensus.cc:385] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.331147  6711 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.331950  6711 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:16.340399  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:16.344734  6711 ts_tablet_manager.cc:1434] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.080s	user 0.005s	sys 0.001s
I20260709 13:16:16.327087  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:16.345515  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:16.345920  6711 tablet_bootstrap.cc:492] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:16.346338  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:16.347966  7026 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:16.349767  7026 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.350131  7026 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:16.344164  6563 leader_election.cc:343] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.355090  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:16.356912  6909 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:16:16.538131  6714 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:16.591823  6714 ts_tablet_manager.cc:1403] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.408s	user 0.027s	sys 0.009s
I20260709 13:16:16.399228  6713 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:16.363821  6845 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:16.419117  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47632.3us
I20260709 13:16:16.494392  7025 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.602264  6711 tablet_bootstrap.cc:654] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.516203  7026 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.371562  6909 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.236s
I20260709 13:16:16.610872  6714 raft_consensus.cc:359] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.611584  6714 raft_consensus.cc:385] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.611857  6714 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.372112  6488 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:16.508000  6963 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:16.616562  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 245527us
I20260709 13:16:16.617129  6714 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.623522  6845 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.624370  6845 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.626099  6845 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.607851  7025 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.626773  6740 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:16.627117  6740 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.627385  6740 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:16.627646  7025 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.618981  6963 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.634984  6963 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:16.639038  6714 ts_tablet_manager.cc:1434] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.004s
I20260709 13:16:16.640226  6714 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:16.640730  6713 tablet_bootstrap.cc:654] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.655545  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:16.653496  7026 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:16.670881  6740 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.680117  6740 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.681720  6711 tablet_bootstrap.cc:492] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:16.682422  6711 ts_tablet_manager.cc:1403] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.337s	user 0.014s	sys 0.009s
I20260709 13:16:16.681813  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:16.681860  7032 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.689777  7032 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.692142  7032 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:16:16.698827  6563 leader_election.cc:343] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:16.703069  6561 leader_election.cc:343] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.704877  6561 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:16.705579  7025 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:16.706534  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:16.711478  6593 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:16.705610  6711 raft_consensus.cc:359] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.712656  6711 raft_consensus.cc:385] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.713042  6711 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.712656  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:16.715739  6592 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:16.717363  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:16.717988  6589 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:16.714349  6711 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.722572  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:16.725682  7033 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:16.726605  7033 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.725705  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:16.727211  6517 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:16.728611  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:16.737546  7033 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:16.750671  6714 tablet_bootstrap.cc:654] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.737902  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:16.751837  6713 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:16.752497  6713 ts_tablet_manager.cc:1403] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.354s	user 0.021s	sys 0.026s
I20260709 13:16:16.750413  6589 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:16.761227  6711 ts_tablet_manager.cc:1434] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.000s
I20260709 13:16:16.764148  6711 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:16.770726  6713 raft_consensus.cc:359] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.771489  6713 raft_consensus.cc:385] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.771760  6713 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.772333  6514 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:16.774725  6713 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.778760  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:16.779088  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:16.782153  6713 ts_tablet_manager.cc:1434] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:16:16.783308  6713 tablet_bootstrap.cc:492] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:16.784335  6517 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
W20260709 13:16:16.783210  6563 leader_election.cc:343] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:16.785871  6561 leader_election.cc:343] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.786299  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:16.791596  7024 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:16.790329  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:16.788884  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:16.839596  6711 tablet_bootstrap.cc:654] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.844321  6714 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:16.844821  6714 ts_tablet_manager.cc:1403] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.205s	user 0.021s	sys 0.013s
I20260709 13:16:16.848551  6714 raft_consensus.cc:359] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.849246  6714 raft_consensus.cc:385] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.849511  6714 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.838450  6522 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:16.850126  6714 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.860668  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:16.861354  6592 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:16.869928  7033 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.876150  6669 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:16.878727  6961 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:16.879508  6961 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:16:16.881392  6637 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:16.887830  6714 ts_tablet_manager.cc:1434] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:16:16.888895  6714 tablet_bootstrap.cc:492] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:16.886452  6713 tablet_bootstrap.cc:654] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.892851  7033 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:16.895685  6637 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:16.896522  7033 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:16.898000  6592 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:16.880427  6961 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.904733  6967 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.905236  6967 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.907778  6967 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:16.908156  6714 tablet_bootstrap.cc:654] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.908677  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:16.910087  6486 leader_election.cc:343] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:16.911471  6711 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:16.911936  6711 ts_tablet_manager.cc:1403] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.148s	user 0.018s	sys 0.007s
I20260709 13:16:16.912134  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:16.912979  6589 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:16.914724  6711 raft_consensus.cc:359] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.914932  6488 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:16.915400  6711 raft_consensus.cc:385] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.915762  6711 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.915895  6967 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:16.916805  6967 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:16.917930  6967 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.914427  7033 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:16.925200  6711 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.927029  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:16.927850  6589 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:16.933682  6711 ts_tablet_manager.cc:1434] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:16:16.934782  6711 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:16.941079  6714 tablet_bootstrap.cc:492] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:16.941624  6714 ts_tablet_manager.cc:1403] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.001s
I20260709 13:16:16.944929  6714 raft_consensus.cc:359] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.945642  6714 raft_consensus.cc:385] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.945902  6714 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.946511  6714 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.950043  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:16.953655  6714 ts_tablet_manager.cc:1434] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:16:16.959528  6713 tablet_bootstrap.cc:492] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:16.960407  6713 ts_tablet_manager.cc:1403] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.177s	user 0.031s	sys 0.045s
I20260709 13:16:16.961040  6711 tablet_bootstrap.cc:654] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:16.962679  6714 tablet_bootstrap.cc:492] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:16.963238  6713 raft_consensus.cc:359] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.963955  6713 raft_consensus.cc:385] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:16.964340  6713 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:16.965158  6713 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.967836  6713 ts_tablet_manager.cc:1434] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:16.970000  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:16.972909  7026 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:16.974217  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:16.969385  6713 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:16.979391  6967 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:16.979774  6967 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.980111  6967 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:16.983016  6417 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:16.989471  6420 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:16.996238  6740 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.996861  6740 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:16.999158  6740 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.005165  6713 tablet_bootstrap.cc:654] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.021073  6740 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.023780  6740 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.024917  6714 tablet_bootstrap.cc:654] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.027370  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.041913  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.044042  6740 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.042526  6669 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:17.011344  6967 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.048333  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:17.050158  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:17.052059  6915 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.051620  6740 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:17.052428  6915 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.054185  6915 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.055117  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:17.048388  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:17.056597  6563 leader_election.cc:343] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.056947  6711 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:17.056934  6593 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:17.057530  6711 ts_tablet_manager.cc:1403] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.123s	user 0.020s	sys 0.000s
I20260709 13:16:17.051222  6486 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.060563  6740 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:17.060839  6740 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.061023  6740 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.063290  6967 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.061542  6711 raft_consensus.cc:359] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.066617  6711 raft_consensus.cc:385] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.066936  6711 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.067788  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:17.067586  6711 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:17.069069  6486 leader_election.cc:343] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.063844  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:17.069691  6593 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.074975  6711 ts_tablet_manager.cc:1434] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:16:17.055323  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.079388  6711 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
W20260709 13:16:17.086819  6561 leader_election.cc:343] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.087282  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:17.088243  6915 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:17.101063  6740 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.104410  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:17.105204  6669 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.106128  6740 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.108095  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:17.108687  6592 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.104868  6593 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:17.119237  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:17.121249  6740 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:17.121655  6740 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:17.120744  6592 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:17.143178  6713 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:17.143689  6713 ts_tablet_manager.cc:1403] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.175s	user 0.016s	sys 0.016s
I20260709 13:16:17.143788  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.143579  6740 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.146862  6755 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:17.147356  6755 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:17.149794  6669 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:17.164992  6714 tablet_bootstrap.cc:492] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:17.165665  6714 ts_tablet_manager.cc:1403] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.211s	user 0.015s	sys 0.006s
I20260709 13:16:17.166631  6713 raft_consensus.cc:359] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.168489  6713 raft_consensus.cc:385] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.168771  6713 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.169480  6713 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.167284  6755 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.185319  6714 raft_consensus.cc:359] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.190747  6714 raft_consensus.cc:385] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.191146  6714 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.191792  6714 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.194305  6714 ts_tablet_manager.cc:1434] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.028s	user 0.002s	sys 0.005s
I20260709 13:16:17.195297  6714 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:17.198951  6711 tablet_bootstrap.cc:654] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.205410  6713 ts_tablet_manager.cc:1434] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.003s
I20260709 13:16:17.213860  6713 tablet_bootstrap.cc:492] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:17.221840  6711 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:17.222362  6711 ts_tablet_manager.cc:1403] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.143s	user 0.018s	sys 0.010s
I20260709 13:16:17.223904  6714 tablet_bootstrap.cc:654] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.225943  6711 raft_consensus.cc:359] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.226665  6711 raft_consensus.cc:385] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.226949  6711 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.229664  6711 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.231160  6711 ts_tablet_manager.cc:1434] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:16:17.232064  6711 tablet_bootstrap.cc:492] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:17.233865  6416 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:17.235746  6713 tablet_bootstrap.cc:654] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.235437  6416 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:17.251667  6711 tablet_bootstrap.cc:654] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.254094  6714 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:17.254674  6714 ts_tablet_manager.cc:1403] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.003s
I20260709 13:16:17.257436  6714 raft_consensus.cc:359] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.258145  6714 raft_consensus.cc:385] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.258459  6714 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.259140  6714 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.261124  6714 ts_tablet_manager.cc:1434] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:17.262072  6714 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:17.263769  6711 tablet_bootstrap.cc:492] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:17.264219  6711 ts_tablet_manager.cc:1403] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.000s
I20260709 13:16:17.267184  6711 raft_consensus.cc:359] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.267838  6711 raft_consensus.cc:385] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.268127  6714 tablet_bootstrap.cc:654] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.268105  6711 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.269380  6711 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.271103  6713 tablet_bootstrap.cc:492] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:17.271589  6713 ts_tablet_manager.cc:1403] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 13:16:17.276929  6711 ts_tablet_manager.cc:1434] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:16:17.274121  6713 raft_consensus.cc:359] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.277854  6713 raft_consensus.cc:385] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.278002  6711 tablet_bootstrap.cc:492] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:17.278162  6713 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.278825  6713 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.283352  6963 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.283929  6963 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.286574  6963 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:17.288252  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:17.289611  6711 tablet_bootstrap.cc:654] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.293831  6593 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:17.292047  6714 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:17.295054  6714 ts_tablet_manager.cc:1403] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.007s
I20260709 13:16:17.289759  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:17.297801  6714 raft_consensus.cc:359] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.298457  6714 raft_consensus.cc:385] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.298705  6714 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
W20260709 13:16:17.299109  6637 leader_election.cc:343] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.299527  6711 tablet_bootstrap.cc:492] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:17.299381  6714 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.299736  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:17.299950  6711 ts_tablet_manager.cc:1403] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 13:16:17.301402  6714 ts_tablet_manager.cc:1434] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:16:17.302457  6961 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:17.302503  6711 raft_consensus.cc:359] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.303259  6714 tablet_bootstrap.cc:492] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:17.303399  6711 raft_consensus.cc:385] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.303709  6711 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.304437  6711 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.309921  6714 tablet_bootstrap.cc:654] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.303103  6961 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.311411  6961 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.316272  6711 ts_tablet_manager.cc:1434] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:16:17.317306  6711 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:17.321501  6714 tablet_bootstrap.cc:492] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:17.322045  6714 ts_tablet_manager.cc:1403] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.010s
I20260709 13:16:17.322847  6711 tablet_bootstrap.cc:654] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.324820  6714 raft_consensus.cc:359] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.325518  6714 raft_consensus.cc:385] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.325804  6714 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.326452  6714 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.328392  6714 ts_tablet_manager.cc:1434] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:16:17.329252  6714 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:17.331156  6961 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.333351  6713 ts_tablet_manager.cc:1434] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.061s	user 0.002s	sys 0.004s
I20260709 13:16:17.335080  6714 tablet_bootstrap.cc:654] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.339771  6711 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:17.340205  6711 ts_tablet_manager.cc:1403] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.003s
I20260709 13:16:17.342876  6711 raft_consensus.cc:359] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.343524  6711 raft_consensus.cc:385] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.343787  6711 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.344501  6711 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.351769  6711 ts_tablet_manager.cc:1434] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:16:17.352762  6711 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:17.355469  6915 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.355957  6915 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.357805  6915 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.359669  6963 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.359462  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:17.361229  6563 leader_election.cc:343] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.362015  6711 tablet_bootstrap.cc:654] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.361691  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:17.363526  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:17.365442  6561 leader_election.cc:343] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.365965  6561 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:17.367518  6915 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:17.368029  6963 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:17.369007  6637 leader_election.cc:343] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.371049  6518 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.371697  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:17.371115  6961 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:17.372788  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:17.370604  6963 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:17.374179  6713 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:17.376094  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:17.376936  6593 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.383385  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:17.405530  6967 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.439687  6967 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.441164  6961 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:17.443431  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:17.444221  6967 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.444391  6592 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:17.446197  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.446341  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.447433  6967 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:17.448001  6967 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:17.448251  6486 leader_election.cc:343] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.402201  6963 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:17.449483  6967 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.421792  6915 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.450335  6915 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.453828  6915 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.399663  6713 tablet_bootstrap.cc:654] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.463289  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:17.464511  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.464793  7026 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:17.467706  6915 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.471832  6561 leader_election.cc:343] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:17.473135  6563 leader_election.cc:343] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.473519  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:17.476020  6845 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:17.468200  6915 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.484056  6714 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:17.484557  6714 ts_tablet_manager.cc:1403] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.156s	user 0.049s	sys 0.069s
I20260709 13:16:17.406224  6593 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:17.485630  6961 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.485450  6915 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.493600  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:17.494280  6516 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:17.494743  6967 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.494833  6961 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.494738  6711 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:17.497022  6711 ts_tablet_manager.cc:1403] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.144s	user 0.041s	sys 0.062s
I20260709 13:16:17.502661  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.508816  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:17.509492  6592 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.509841  7025 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:16:17.510099  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:17.510250  7025 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.510632  7025 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.506126  6714 raft_consensus.cc:359] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.514119  6714 raft_consensus.cc:385] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.514500  6714 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.515688  6755 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.516177  6755 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.516536  6967 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.519199  6961 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:17.520016  6755 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.520831  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:17.521493  6593 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:17.523082  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.522578  6714 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.520457  6592 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:17.524526  6755 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:17.541395  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:17.541458  6755 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.542101  6755 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.541842  7025 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.544700  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:17.545298  6518 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.534049  6740 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:17.535205  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.536183  6669 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.537444  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:17.538194  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:17.540421  6714 ts_tablet_manager.cc:1434] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 13:16:17.551963  6714 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:17.553261  6597 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:17.554811  7026 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:17.555327  7026 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:17.559069  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:17.563463  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.567091  7026 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:17.570504  6561 leader_election.cc:343] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.572926  7070 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:17.573472  7070 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:17.574446  6759 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:17.574877  6759 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.575171  6759 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.574357  7070 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.576062  6755 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.582201  6522 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:17.590777  6967 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.591488  6967 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.593700  6967 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.606067  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:17.626149  6713 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:17.626662  6713 ts_tablet_manager.cc:1403] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.253s	user 0.017s	sys 0.024s
I20260709 13:16:17.606402  6755 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.634532  6713 raft_consensus.cc:359] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.635286  6713 raft_consensus.cc:385] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.635569  6713 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.636231  6713 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.636776  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:17.638399  6488 leader_election.cc:343] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.606695  6518 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:17.604913  7025 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.625197  6915 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.660878  6711 raft_consensus.cc:359] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.661626  6711 raft_consensus.cc:385] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.661953  6711 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.651914  6915 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.679108  6711 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.674703  6755 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.075s
I20260709 13:16:17.708815  6714 tablet_bootstrap.cc:654] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.750483  6711 ts_tablet_manager.cc:1434] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.253s	user 0.005s	sys 0.004s
I20260709 13:16:17.773604  6713 ts_tablet_manager.cc:1434] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.147s	user 0.029s	sys 0.021s
W20260709 13:16:17.709933  7071 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:16:17.675908  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:17.794997  6594 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.798556  6711 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:17.803089  6915 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:17.807235  6713 tablet_bootstrap.cc:492] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
W20260709 13:16:17.811131  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 185429us
I20260709 13:16:17.821342  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.834978  7025 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:17.851543  6759 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.869376  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:17.874015  6594 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:17.875154  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.876789  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:17.877816  6670 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:17.887766  6759 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:17.902144  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:17.902971  6517 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:17.864358  7025 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:17.907331  7025 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.912119  6713 tablet_bootstrap.cc:654] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.914304  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:17.915870  7083 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:17.916344  7083 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:17.916744  6967 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:17.923202  7083 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:17.980717  6714 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
W20260709 13:16:18.037528  6486 leader_election.cc:343] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:18.002594  7080 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:18.004703  6561 leader_election.cc:343] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:18.053380  6714 ts_tablet_manager.cc:1403] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.502s	user 0.028s	sys 0.059s
W20260709 13:16:18.003710  7077 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.222s	user 0.007s	sys 0.000s
W20260709 13:16:18.055549  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50900.5us
W20260709 13:16:18.056327  6486 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:18.057011  6561 leader_election.cc:343] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:17.980715  6416 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:18.057792  6714 raft_consensus.cc:359] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.058494  6714 raft_consensus.cc:385] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.058776  6714 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.059557  6714 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.053901  7080 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.022869  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:18.061666  6670 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:18.061810  7080 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:18.001708  7082 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.062675  7082 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.008929  6420 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:18.013309  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:18.065966  6711 tablet_bootstrap.cc:700] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Time spent opening tablet: real 0.149s	user 0.016s	sys 0.024s
I20260709 13:16:18.066637  6711 tablet_bootstrap.cc:654] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.924371  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:18.069003  6520 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:17.981132  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:18.069628  7091 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:18.070654  6967 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:18.079489  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:18.080155  6589 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:18.082685  6714 ts_tablet_manager.cc:1434] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:16:18.083801  6714 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:18.087152  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:18.088459  6759 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:18.088888  6759 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.089263  6759 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.099699  6714 tablet_bootstrap.cc:654] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.069201  7082 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.143288  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:18.144136  6759 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.151859  6713 tablet_bootstrap.cc:492] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:18.101130  6593 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:16:18.152648  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49129.8us
I20260709 13:16:18.152781  6759 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:18.156026  6713 ts_tablet_manager.cc:1403] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.362s	user 0.022s	sys 0.035s
I20260709 13:16:18.163072  6563 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:16:18.160456  6561 leader_election.cc:343] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:18.159379  6967 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.171974  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:18.168105  7082 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:18.175026  7082 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.176367  7082 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.176864  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:18.177600  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:18.181082  7033 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:18.171257  6967 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.207003  6967 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.201629  6517 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:18.208070  6713 raft_consensus.cc:359] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.208994  6713 raft_consensus.cc:385] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.215340  6713 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.220624  6713 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:18.228228  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56741.7us
I20260709 13:16:18.211503  6589 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.231299  7099 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.231775  7099 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.251613  7099 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.256068  6563 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:16:18.257125  6486 leader_election.cc:343] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:18.263358  6486 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:18.263687  6711 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:18.264422  6711 ts_tablet_manager.cc:1403] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.485s	user 0.028s	sys 0.056s
W20260709 13:16:18.267616  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 101458us
I20260709 13:16:18.271549  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:18.272199  6520 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:18.275046  6713 ts_tablet_manager.cc:1434] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.119s	user 0.004s	sys 0.004s
I20260709 13:16:18.281145  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:18.281899  6522 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.279803  6915 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:18.321866  6915 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.322271  6915 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.318168  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:18.323319  6664 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:18.327602  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:18.330571  7100 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:18.361495  6713 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:18.393926  6662 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.404536  6589 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:18.406394  6711 raft_consensus.cc:359] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.416200  6711 raft_consensus.cc:385] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.416508  6711 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.417184  6711 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.423213  6915 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.426815  6967 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.419735  6666 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:18.432930  7082 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.437162  6711 ts_tablet_manager.cc:1434] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.172s	user 0.010s	sys 0.004s
I20260709 13:16:18.449452  6714 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:18.452484  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:18.459151  6714 ts_tablet_manager.cc:1403] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.376s	user 0.017s	sys 0.037s
W20260709 13:16:18.462282  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 188379us
I20260709 13:16:18.470897  6561 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:18.434790  6967 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.474787  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:18.475085  7108 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:18.475533  7108 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.475874  7108 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:18.480983  6713 tablet_bootstrap.cc:654] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.478562  6711 tablet_bootstrap.cc:492] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:18.482085  6518 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:18.490383  6963 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:18.505882  6915 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.521934  6522 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:18.499572  7082 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.531757  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:18.534399  6963 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:18.539768  6963 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.544973  6714 raft_consensus.cc:359] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.545568  6714 raft_consensus.cc:385] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.545794  6714 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.554683  7078 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.580711  6755 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:18.581343  6755 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.555413  7078 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.591897  6755 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.596179  7078 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:18.600497  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:18.601290  6515 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.608534  6967 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.592566  7108 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.616827  6714 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:18.619366  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 110238us
W20260709 13:16:18.620971  6561 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:18.623904  6561 leader_election.cc:343] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:18.712606  6714 ts_tablet_manager.cc:1434] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.252s	user 0.005s	sys 0.008s
I20260709 13:16:18.716161  6521 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.633877  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:18.718189  6594 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:18.642599  6417 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:18.719918  6711 tablet_bootstrap.cc:654] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.721746  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:18.722393  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:18.722637  6595 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:18.725598  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:18.727344  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:18.726830  6670 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:18.731184  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:18.731820  6593 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:18.733492  6713 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:18.733965  6713 ts_tablet_manager.cc:1403] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.415s	user 0.012s	sys 0.014s
I20260709 13:16:18.736011  7108 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:16:18.743664  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:18.744580  6967 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:18.746032  6486 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:18.747717  6714 tablet_bootstrap.cc:492] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:18.749593  6740 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:18.749965  6740 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.750288  6740 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:18.750464  7106 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:16:18.756453  6713 raft_consensus.cc:359] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.758853  6713 raft_consensus.cc:385] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.759325  6713 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.760241  6713 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.763450  6713 ts_tablet_manager.cc:1434] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.002s
I20260709 13:16:18.757880  6740 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.766589  7099 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:18.773478  6713 tablet_bootstrap.cc:492] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:18.774176  6740 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.776197  6515 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:18.779711  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:18.780508  6520 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 2: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:18.776990  6759 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:18.793443  6755 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:18.794458  7119 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.797565  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:18.799082  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:18.799660  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:18.799822  6521 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.805155  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:18.805946  6595 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:18.795011  7119 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.818920  6521 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:18.820067  6488 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:18.821136  6740 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:18.821455  6420 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:18.825006  7070 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:18.826550  6563 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:18.830307  6563 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:18.834892  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:18.843137  7113 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:18.832471  7108 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:18.832196  7099 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:18.834827  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:18.849215  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:16:18.869526  6548 tablet.cc:2406] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:18.858637  7119 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:18.872814  6714 tablet_bootstrap.cc:654] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.857722  7118 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:18.862792  7108 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:18.863145  7099 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:18.886477  7108 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.874070  6663 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:18.867440  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:18.896236  7119 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.896414  7103 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.901563  6711 tablet_bootstrap.cc:492] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:18.902735  6711 ts_tablet_manager.cc:1403] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.445s	user 0.015s	sys 0.006s
I20260709 13:16:18.909682  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:18.914386  7099 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.914673  6711 raft_consensus.cc:359] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.915377  6711 raft_consensus.cc:385] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.915679  6711 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.917647  7118 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:18.921156  7118 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:18.922048  6711 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:18.928472  6561 leader_election.cc:343] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:18.930765  6561 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:18.940223  6711 ts_tablet_manager.cc:1434] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:16:18.899420  7103 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.944494  6416 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:18.906958  7119 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:18.954365  6711 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Bootstrap starting.
I20260709 13:16:18.954808  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:18.961166  6713 tablet_bootstrap.cc:654] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.948249  7079 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:16:18.971395  7076 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:18.980860  7119 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.008452  7103 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.025352  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.028447  6711 tablet_bootstrap.cc:654] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.033771  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:19.035629  6563 leader_election.cc:343] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:19.038542  6563 leader_election.cc:343] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:19.039497  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 108817us
W20260709 13:16:19.040175  6561 leader_election.cc:343] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.040562  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.042163  7082 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.058898  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:19.063912  6637 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.041744  7118 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.075090  6713 tablet_bootstrap.cc:492] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.075582  6713 ts_tablet_manager.cc:1403] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.302s	user 0.037s	sys 0.023s
I20260709 13:16:19.075901  7102 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.071727  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:19.085033  6671 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:19.100714  6663 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
W20260709 13:16:19.103734  6561 leader_election.cc:343] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.104156  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.105556  6486 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:19.111264  6755 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:19.112277  6755 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:19.112282  6713 raft_consensus.cc:359] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.112638  7118 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.113070  6713 raft_consensus.cc:385] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.113409  6713 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.113247  6755 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.114017  6713 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.115563  6713 ts_tablet_manager.cc:1434] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.040s	user 0.002s	sys 0.004s
I20260709 13:16:19.120987  7086 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.116988  7080 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:16:19.124049  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:19.124644  6845 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.125135  6845 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.126605  7083 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:16:19.129352  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:19.127153  6845 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:16:19.131877  7129 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:16:19.133661  6711 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: No bootstrap required, opened a new log
I20260709 13:16:19.134208  6711 ts_tablet_manager.cc:1403] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent bootstrapping tablet: real 0.185s	user 0.017s	sys 0.004s
I20260709 13:16:19.135483  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.139501  7087 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.141427  6714 tablet_bootstrap.cc:492] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:19.141887  6714 ts_tablet_manager.cc:1403] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.415s	user 0.030s	sys 0.018s
I20260709 13:16:19.142956  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:16:19.144269  6563 leader_election.cc:343] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.130868  7114 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:16:19.147286  7116 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:19.154881  6416 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.156708  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:19.156806  6416 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.140009  7087 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:16:19.164147  6561 leader_election.cc:343] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.164713  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.167407  6713 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.169412  7082 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.174036  7070 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:19.174439  7070 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:19.205593  6662 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.230419  7087 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.215160  7070 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.284578  7025 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.285794  7025 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.311239  6714 raft_consensus.cc:359] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.311985  6714 raft_consensus.cc:385] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.312247  6714 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.356170  6714 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.354576  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:19.359812  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:19.324254  7070 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:19.361841  6714 ts_tablet_manager.cc:1434] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.220s	user 0.009s	sys 0.002s
I20260709 13:16:19.324518  6711 raft_consensus.cc:359] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.325362  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.354117  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.315263  7025 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.366096  6711 raft_consensus.cc:385] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.366441  6711 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.368563  6711 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.372080  6714 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Bootstrap starting.
W20260709 13:16:19.373879  6561 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:19.379626  6713 tablet_bootstrap.cc:654] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:19.380291  6561 leader_election.cc:343] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.386039  6714 tablet_bootstrap.cc:654] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.387748  6845 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.388389  6845 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.398037  6711 ts_tablet_manager.cc:1434] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent starting tablet: real 0.263s	user 0.009s	sys 0.001s
W20260709 13:16:19.406173  6563 leader_election.cc:343] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.406653  6563 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.408558  6845 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:16:19.409626  6561 leader_election.cc:343] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.415601  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:19.417234  6713 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.417721  6713 ts_tablet_manager.cc:1403] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.251s	user 0.001s	sys 0.016s
I20260709 13:16:19.419517  6417 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 2, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.420454  6713 raft_consensus.cc:359] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.421097  6713 raft_consensus.cc:385] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.422219  6714 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:19.422848  7133 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.424084  6714 ts_tablet_manager.cc:1403] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.001s
W20260709 13:16:19.425719  6563 leader_election.cc:343] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.426128  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.426831  6714 raft_consensus.cc:359] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.427729  6714 raft_consensus.cc:385] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.428052  6714 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.428072  7133 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.428859  6714 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.431824  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.432206  6714 ts_tablet_manager.cc:1434] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
W20260709 13:16:19.433053  6561 consensus_peers.cc:597] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:19.433267  6714 tablet_bootstrap.cc:492] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
W20260709 13:16:19.434113  6561 leader_election.cc:343] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.421365  6713 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.435931  6713 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.437956  6713 ts_tablet_manager.cc:1434] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:16:19.438942  6713 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.443099  7114 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.443583  7114 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.445384  6713 tablet_bootstrap.cc:654] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.447566  7114 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:16:19.450596  6563 leader_election.cc:343] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.450789  6845 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:19.451066  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.452446  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:19.455880  6637 leader_election.cc:343] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.458940  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:19.459591  6588 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:19.461743  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:19.462836  6714 tablet_bootstrap.cc:654] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.463492  7116 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:19.463884  7116 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.464203  7116 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.472061  7116 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.473287  7087 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.475296  6845 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:19.478578  6713 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.479082  6713 ts_tablet_manager.cc:1403] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.017s
I20260709 13:16:19.482126  6713 raft_consensus.cc:359] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.482868  6713 raft_consensus.cc:385] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.483105  6713 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.483656  6713 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.484265  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:16:19.485644  6637 leader_election.cc:343] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.486001  7116 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.487084  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:19.487615  6595 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.489838  6713 ts_tablet_manager.cc:1434] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:16:19.492673  6713 tablet_bootstrap.cc:492] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.497020  6595 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:19.498680  6714 tablet_bootstrap.cc:492] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:19.499583  6714 ts_tablet_manager.cc:1403] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.005s
I20260709 13:16:19.500872  6713 tablet_bootstrap.cc:654] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.502250  6714 raft_consensus.cc:359] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.503054  6714 raft_consensus.cc:385] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.503355  6714 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.504025  6714 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.504936  6967 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.505409  6967 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.507843  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:16:19.507838  6967 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.509553  7101 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:19.510016  7101 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:19.510862  7101 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.511976  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:19.512552  6595 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:19.514374  6714 ts_tablet_manager.cc:1434] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:16:19.515173  6488 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:19.517302  6755 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:19.517730  6755 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.518123  6755 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.519621  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:19.521863  6486 leader_election.cc:343] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.522266  6713 tablet_bootstrap.cc:492] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.522734  6713 ts_tablet_manager.cc:1403] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.003s
I20260709 13:16:19.524302  6755 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.525195  6713 raft_consensus.cc:359] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.525812  6713 raft_consensus.cc:385] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.526079  6713 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.526386  6669 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:19.527153  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:19.527633  6755 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.527750  7070 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:19.528513  6714 tablet_bootstrap.cc:492] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
W20260709 13:16:19.528393  6486 leader_election.cc:343] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.526786  6713 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.530668  6713 ts_tablet_manager.cc:1434] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:19.530774  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:19.531697  6713 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.531613  6588 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.540407  6713 tablet_bootstrap.cc:654] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.552744  6713 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.553208  6713 ts_tablet_manager.cc:1403] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 13:16:19.555879  6713 raft_consensus.cc:359] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.556505  6713 raft_consensus.cc:385] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.556771  6713 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.568789  6714 tablet_bootstrap.cc:654] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.568954  6713 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.572971  6713 ts_tablet_manager.cc:1434] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.002s
I20260709 13:16:19.576071  6595 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:19.577888  6713 tablet_bootstrap.cc:492] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.577994  6967 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:19.556655  6588 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:19.587399  6713 tablet_bootstrap.cc:654] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.587743  6488 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:19.589754  6967 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:19.590279  6967 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:19.591317  6967 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.591852  6420 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:19.602255  7116 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.602821  7116 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.604704  7116 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.606102  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:19.613127  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.613869  6515 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:19.606719  6595 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:19.617990  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:19.620492  7116 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:19.623823  7116 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.645767  6714 tablet_bootstrap.cc:492] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:19.646271  6714 ts_tablet_manager.cc:1403] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.119s	user 0.021s	sys 0.005s
I20260709 13:16:19.679725  6713 tablet_bootstrap.cc:492] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.683235  6713 ts_tablet_manager.cc:1403] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.106s	user 0.014s	sys 0.015s
I20260709 13:16:19.639004  7116 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.709007  6714 raft_consensus.cc:359] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.710321  6714 raft_consensus.cc:385] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.710813  6714 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.711925  6714 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.715132  6420 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.717809  6713 raft_consensus.cc:359] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.719416  6713 raft_consensus.cc:385] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.719934  6713 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.732937  7116 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.735646  6714 ts_tablet_manager.cc:1434] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.089s	user 0.014s	sys 0.000s
I20260709 13:16:19.748870  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:19.749332  6714 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:19.749508  6522 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.739037  6967 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.754099  6967 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.755247  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:19.755688  6714 tablet_bootstrap.cc:654] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.755900  6588 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.756901  6522 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:19.759850  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:19.760907  7118 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:19.761415  7118 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:19.763568  6588 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:19.744673  7116 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.731904  6713 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.766074  6713 ts_tablet_manager.cc:1434] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.002s
I20260709 13:16:19.767040  6713 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.762452  7118 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.779247  6967 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.780695  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:19.784070  6714 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:19.784576  6714 ts_tablet_manager.cc:1403] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.036s	user 0.022s	sys 0.008s
I20260709 13:16:19.780575  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:19.786262  6595 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 1: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:19.780025  6713 tablet_bootstrap.cc:654] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.789709  7116 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.790210  7116 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.791976  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:19.792413  7116 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.794081  6714 raft_consensus.cc:359] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.794327  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:19.794814  6714 raft_consensus.cc:385] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.795092  6714 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.795706  6714 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.797387  6755 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.797827  6714 ts_tablet_manager.cc:1434] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:16:19.798790  6714 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:19.805279  6714 tablet_bootstrap.cc:654] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.822961  6713 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.823450  6713 ts_tablet_manager.cc:1403] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.007s
I20260709 13:16:19.826210  6755 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.823568  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.827955  6637 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:19.828666  7116 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:19.831642  6468 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:19.831984  6714 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:19.832451  6714 ts_tablet_manager.cc:1403] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.014s
I20260709 13:16:19.827754  6713 raft_consensus.cc:359] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.834806  6713 raft_consensus.cc:385] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.835098  6714 raft_consensus.cc:359] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.835817  6714 raft_consensus.cc:385] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.836077  6714 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.836711  6714 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.835100  6713 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.838761  6714 ts_tablet_manager.cc:1434] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:19.839363  6713 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.840322  6714 tablet_bootstrap.cc:492] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Bootstrap starting.
I20260709 13:16:19.841837  6713 ts_tablet_manager.cc:1434] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:16:19.842854  6713 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.846633  6714 tablet_bootstrap.cc:654] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.847602  6420 catalog_manager.cc:5697] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:19.849918  6845 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.850410  6845 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.852177  6845 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.853097  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:19.853730  6515 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 0.
I20260709 13:16:19.856180  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:19.857084  7025 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:19.857583  7025 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.857995  7025 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.861505  6713 tablet_bootstrap.cc:654] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.861174  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:16:19.864431  6561 leader_election.cc:343] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.867302  6669 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.871279  6714 tablet_bootstrap.cc:492] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: No bootstrap required, opened a new log
I20260709 13:16:19.871762  6714 ts_tablet_manager.cc:1403] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.010s
I20260709 13:16:19.874176  6714 raft_consensus.cc:359] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.874866  6714 raft_consensus.cc:385] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.875643  6714 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.876252  6714 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.878315  6714 ts_tablet_manager.cc:1434] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:19.881871  6713 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.882381  6713 ts_tablet_manager.cc:1403] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.001s
I20260709 13:16:19.887140  7025 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.889827  6713 raft_consensus.cc:359] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.890405  7025 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.890662  6713 raft_consensus.cc:385] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.891113  6713 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.891281  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:19.891903  6515 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.891980  6713 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.897519  6713 ts_tablet_manager.cc:1434] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:16:19.898530  6713 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.899501  6515 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:16:19.900450  7118 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:19.900767  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:19.901983  7086 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:19.902593  7086 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:19.903481  7086 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.907130  6666 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.901752  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:16:19.916990  6561 leader_election.cc:343] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.923617  6713 tablet_bootstrap.cc:654] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.924939  6420 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:19.937147  6713 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:19.937647  6713 ts_tablet_manager.cc:1403] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.000s
I20260709 13:16:19.940435  6713 raft_consensus.cc:359] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.941115  6713 raft_consensus.cc:385] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.941380  6713 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.947436  6713 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.949625  6713 ts_tablet_manager.cc:1434] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:16:19.950697  6713 tablet_bootstrap.cc:492] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Bootstrap starting.
I20260709 13:16:19.957134  6713 tablet_bootstrap.cc:654] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.961526  7156 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:19.973874  7101 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.974380  7101 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.976377  7101 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.986356  6961 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.986887  6961 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.988755  6961 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:19.989710  6967 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.990180  6967 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:19.993546  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:19.994130  6669 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:19.995111  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:19.995852  7070 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:19.996201  7070 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.996313  6755 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:19.996575  7070 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.996976  6967 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:19.998034  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:19.998950  6588 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:20.006562  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:20.007712  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:20.008944  6592 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:16:20.011144  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:20.011926  6520 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 1: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:16:20.016247  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:20.003935  7070 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.031631  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:20.032111  6669 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.035149  7070 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:20.036603  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:20.037163  6595 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.032238  6666 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.042539  7091 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.043659  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:20.044392  7156 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.045946  6637 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:20.046808  7156 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.047232  6666 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:20.048357  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:20.049314  7091 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.049796  7091 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:20.050585  7091 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.052265  6713 tablet_bootstrap.cc:492] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: No bootstrap required, opened a new log
I20260709 13:16:20.052712  6713 ts_tablet_manager.cc:1403] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Time spent bootstrapping tablet: real 0.102s	user 0.028s	sys 0.038s
I20260709 13:16:20.056088  6595 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:20.055325  6713 raft_consensus.cc:359] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.062692  6713 raft_consensus.cc:385] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.063052  6713 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.063859  6713 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.066519  6713 ts_tablet_manager.cc:1434] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:16:20.068922  7156 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.072664  6967 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.073269  6967 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.075942  6967 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:20.077924  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:20.078588  6662 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:20.080670  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:20.081725  7091 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.082156  7091 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.082530  7091 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.083104  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:20.085105  6595 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 0.
I20260709 13:16:20.087925  6420 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.089059  7091 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.091528  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:20.091717  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:20.092221  6595 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.092271  6669 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.091620  7091 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:20.098073  6669 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:20.099045  6595 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:16:20.099251  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:20.100060  7091 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.101637  7091 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:20.102471  7091 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.117832  6417 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 0 to 1, leader changed from <none> to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.125010  6961 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.125471  6961 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.127174  6961 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:20.127772  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:20.127976  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:20.128337  6520 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:20.128497  6595 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 0.
I20260709 13:16:20.129249  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:20.129881  6961 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.130227  6961 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.130548  6961 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.134968  6961 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.136771  6961 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:20.137712  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:20.137887  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:20.138253  6520 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.138381  6595 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.142463  6520 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:20.142601  6595 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:16:20.143330  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:20.144634  6961 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.145267  6961 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:20.146123  6961 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.155426  6417 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 0 to 1, leader changed from <none> to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.167007  6961 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.179199  7116 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.294587  7025 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.304229  7025 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:16:20.307533  7101 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.308017  7101 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.309887  7101 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:20.311885  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:20.313068  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:20.315419  6637 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:20.316159  7116 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.462224  6662 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.522493  6967 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.532284  7070 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.587100  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:20.600630  7183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:20.602039  7116 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:20.603399  7184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:20.606575  7186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:20.606777  5410 server_base.cc:1061] running on GCE node
I20260709 13:16:20.613276  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:20.613633  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:20.613933  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783602980613913 us; error 0 us; skew 500 ppm
I20260709 13:16:20.614816  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:20.614589  7101 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.635545  6845 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:20.636051  6845 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.638098  6845 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:20.638801  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:20.639420  6522 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:20.639864  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:20.640591  6669 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 2: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:20.640511  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:20.638855  5410 webserver.cc:533] Webserver started at http://127.5.72.132:40785/ using document root <none> and password file <none>
I20260709 13:16:20.641690  7086 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:20.642123  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:20.642095  7086 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:20.642664  7086 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:20.642638  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:20.671283  7182 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.674875  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:20.676931  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-3-root/instance:
uuid: "64b95375974b48d18c596a23c9cac75c"
format_stamp: "Formatted at 2026-07-09 13:16:20 on dist-test-slave-n5kz"
I20260709 13:16:20.681649  7091 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:20.696699  7086 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.700172  7086 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:20.701164  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:20.702036  6662 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 2: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:20.710350  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:20.712105  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.034s	user 0.006s	sys 0.011s
I20260709 13:16:20.716792  6520 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:20.717464  7195 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:20.717885  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:16:20.718709  7086 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:20.718835  5410 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:16:20.719254  7086 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:20.719272  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-3-root
uuid: "64b95375974b48d18c596a23c9cac75c"
format_stamp: "Formatted at 2026-07-09 13:16:20 on dist-test-slave-n5kz"
I20260709 13:16:20.719852  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:20.720155  7086 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:20.733568  6420 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.743072  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:20.744989  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:20.746588  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:20.748839  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:20.749053  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:20.749320  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:20.749490  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:20.824939  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.132:36575
I20260709 13:16:20.825031  7261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.132:36575 every 8 connection(s)
I20260709 13:16:20.855927  5410 auto_leader_rebalancer-test.cc:384] add a tserver: 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:20.885948  7262 heartbeater.cc:344] Connected to a master server at 127.5.72.190:46511
I20260709 13:16:20.886368  7262 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:20.887149  7262 heartbeater.cc:507] Master 127.5.72.190:46511 requested a full tablet report, sending...
I20260709 13:16:20.888919  6420 ts_manager.cc:194] Registered new tserver with Master: 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:20.890292  6420 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56974
I20260709 13:16:21.181715  6666 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:21.182997  7025 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:21.196343  6520 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:21.202343  7025 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:21.892619  7262 heartbeater.cc:499] Master 127.5.72.190:46511 was elected leader, sending a full tablet report...
I20260709 13:16:22.002107  6669 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.009210  6591 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.013586  6961 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.014039  6522 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.018177  7274 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.027769  7274 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.029642  6591 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.033895  6522 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:22.035961  6636 consensus_peers.cc:597] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9bbf06685e047fdaa1b6bc881dc7a93. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:22.051607  6420 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.065958  6591 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.074204  6522 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.075073  6669 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.075907  7025 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.076932  6845 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.102892  6845 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:22.107167  6562 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6be297bb363c43c99d04cccfc55bec8f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:22.106721  6669 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.113065  6520 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.121183  6416 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.138232  6520 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.161688  6669 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.163341  7091 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.168567  6591 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.170183  7270 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:22.177613  7270 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.179610  6669 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:22.186228  6487 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2cbaf7daabfe40028228e412b8f9214b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:22.201572  6669 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.201910  6416 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.203733  6591 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:22.226518  6636 consensus_peers.cc:597] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5a761916a9242a99d2d5b2e54ffda61. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:22.228698  6588 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.235626  6515 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:22.239516  7290 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.241463  7274 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:22.257762  6961 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.266485  6515 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.279019  6588 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.291112  6416 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:22.534643  7297 ts_tablet_manager.cc:933] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:22.539036  7297 tablet_copy_client.cc:323] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:22.551735  6681 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet c9bbf06685e047fdaa1b6bc881dc7a93 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:22.552385  6681 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet c9bbf06685e047fdaa1b6bc881dc7a93 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-c9bbf06685e047fdaa1b6bc881dc7a93
I20260709 13:16:22.558447  6681 tablet_copy_source_session.cc:215] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:22.562668  7297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9bbf06685e047fdaa1b6bc881dc7a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:22.575816  7297 tablet_copy_client.cc:806] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:22.576457  7297 tablet_copy_client.cc:670] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:22.580149  7297 tablet_copy_client.cc:538] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:22.587911  7297 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:22.606938  7297 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:22.607733  7297 tablet_bootstrap.cc:492] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:22.608273  7297 ts_tablet_manager.cc:1403] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.004s
I20260709 13:16:22.610865  7297 raft_consensus.cc:359] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.611474  7297 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:22.611992  7297 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.614300  7297 ts_tablet_manager.cc:1434] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:16:22.616194  6681 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-c9bbf06685e047fdaa1b6bc881dc7a93 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:22.616670  6681 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-c9bbf06685e047fdaa1b6bc881dc7a93 on tablet c9bbf06685e047fdaa1b6bc881dc7a93 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:22.621906  7297 ts_tablet_manager.cc:933] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:22.626830  7297 tablet_copy_client.cc:323] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:22.628473  6681 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet e5a761916a9242a99d2d5b2e54ffda61 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:22.629019  6681 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet e5a761916a9242a99d2d5b2e54ffda61 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-e5a761916a9242a99d2d5b2e54ffda61
I20260709 13:16:22.643762  7304 ts_tablet_manager.cc:933] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:22.647367  7304 tablet_copy_client.cc:323] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:22.650804  6681 tablet_copy_source_session.cc:215] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:22.654405  7297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a761916a9242a99d2d5b2e54ffda61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:22.663487  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 6be297bb363c43c99d04cccfc55bec8f from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:22.664112  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 6be297bb363c43c99d04cccfc55bec8f from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-6be297bb363c43c99d04cccfc55bec8f
I20260709 13:16:22.671867  6607 tablet_copy_source_session.cc:215] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:22.673254  7297 tablet_copy_client.cc:806] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:22.674234  7297 tablet_copy_client.cc:670] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:22.678344  7304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6be297bb363c43c99d04cccfc55bec8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:22.685715  7297 tablet_copy_client.cc:538] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:22.694541  7297 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:22.700146  7304 tablet_copy_client.cc:806] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:22.700851  7304 tablet_copy_client.cc:670] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:22.709714  7304 tablet_copy_client.cc:538] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:22.723305  7304 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:22.737134  7307 ts_tablet_manager.cc:933] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:22.748025  7297 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:22.749362  7297 tablet_bootstrap.cc:492] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:22.754858  7297 ts_tablet_manager.cc:1403] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.006s
I20260709 13:16:22.761318  7307 tablet_copy_client.cc:323] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:22.763335  7297 raft_consensus.cc:359] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.764361  7297 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:22.765149  7297 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.778643  7297 ts_tablet_manager.cc:1434] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:16:22.788987  7304 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:22.789960  7304 tablet_bootstrap.cc:492] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:22.790329  6681 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-e5a761916a9242a99d2d5b2e54ffda61 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:22.790854  6681 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-e5a761916a9242a99d2d5b2e54ffda61 on tablet e5a761916a9242a99d2d5b2e54ffda61 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:22.793982  7304 ts_tablet_manager.cc:1403] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.071s	user 0.029s	sys 0.005s
I20260709 13:16:22.797875  7304 raft_consensus.cc:359] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.800254  7304 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:22.800997  7304 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.807014  7304 ts_tablet_manager.cc:1434] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 13:16:22.807091  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 2cbaf7daabfe40028228e412b8f9214b from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:22.807714  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 2cbaf7daabfe40028228e412b8f9214b from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-2cbaf7daabfe40028228e412b8f9214b
I20260709 13:16:22.808727  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-6be297bb363c43c99d04cccfc55bec8f received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:22.809162  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-6be297bb363c43c99d04cccfc55bec8f on tablet 6be297bb363c43c99d04cccfc55bec8f with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:22.814879  6532 tablet_copy_source_session.cc:215] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:22.817874  7307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbaf7daabfe40028228e412b8f9214b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:22.832100  7307 tablet_copy_client.cc:806] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:22.832643  7307 tablet_copy_client.cc:670] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:22.836015  7307 tablet_copy_client.cc:538] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:22.845595  7307 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:22.872080  7307 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:22.872898  7307 tablet_bootstrap.cc:492] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:22.873644  7307 ts_tablet_manager.cc:1403] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.028s	user 0.029s	sys 0.001s
I20260709 13:16:22.876804  7307 raft_consensus.cc:359] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.877403  7307 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:22.877914  7307 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:22.879988  7307 ts_tablet_manager.cc:1434] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:16:22.881875  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-2cbaf7daabfe40028228e412b8f9214b received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:22.882382  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-2cbaf7daabfe40028228e412b8f9214b on tablet 2cbaf7daabfe40028228e412b8f9214b with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:22.958935  7237 raft_consensus.cc:1217] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:23.024734  7237 raft_consensus.cc:1217] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:23.092952  7237 raft_consensus.cc:1217] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:23.201463  7237 raft_consensus.cc:1217] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:23.524593  6961 raft_consensus.cc:1064] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:23.526993  6961 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:23.532737  7237 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.535188  6522 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.535213  7315 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:23.542843  6592 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.546109  7274 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:23.547996  7315 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:23.557335  7315 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:23.559803  7314 raft_consensus.cc:1064] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:23.563789  7314 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:23.593396  6961 raft_consensus.cc:1064] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:23.597108  6515 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:23.616227  7182 raft_consensus.cc:1064] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:23.624392  7182 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:23.648206  7237 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.650053  6592 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:23.654024  6845 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:23.661163  6522 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.662834  7086 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:23.680827  6669 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.680012  6417 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.684612  7086 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:23.692744  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 113970us
I20260709 13:16:23.703016  6961 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:23.702015  7236 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:23.889353  7235 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.903448  7331 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:23.917186  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 145780us
I20260709 13:16:23.925629  6664 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.931077  7337 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:16:23.950510  7237 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:23.975422  7351 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:23.978569  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49284.7us
W20260709 13:16:23.984263  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 61092.4us
I20260709 13:16:23.993132  7328 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:23.995060  6592 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:24.001999  7237 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.029919  7338 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:24.029811  6664 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:24.052111  7237 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.047384  6416 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.056099  6961 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.056022  7343 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.065589  7324 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.066021  6666 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.073022  6591 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:24.089318  6522 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:24.090536  6592 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.091178  6522 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.095538  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e5a761916a9242a99d2d5b2e54ffda61 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:24.101711  6595 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.108455  7235 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.078357  7237 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.116997  7320 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:24.124341  6595 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:24.132007  7320 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.139431  7236 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.141006  7352 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.141919  6416 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.144716  6522 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.141693  6420 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.147171  7352 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.152820  7327 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:24.157688  7328 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.120085  6671 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.186305  7235 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.191181  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet e5a761916a9242a99d2d5b2e54ffda61 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:24.171880  6595 raft_consensus.cc:2955] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.193810  6515 raft_consensus.cc:2955] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.204336  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6be297bb363c43c99d04cccfc55bec8f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:24.236846  7362 tablet_replica.cc:333] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:24.240723  6961 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.243520  7362 raft_consensus.cc:2243] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:24.244309  7362 raft_consensus.cc:2272] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:24.243748  6521 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.248066  6420 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.247411  6591 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.264322  7362 ts_tablet_manager.cc:1916] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:24.251663  6515 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:24.269858  7235 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.274849  7237 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.274847  6671 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.277843  7338 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.278821  7332 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.288015  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 6be297bb363c43c99d04cccfc55bec8f with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:24.289373  7362 ts_tablet_manager.cc:1929] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:24.289925  7362 log.cc:1199] T e5a761916a9242a99d2d5b2e54ffda61 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/e5a761916a9242a99d2d5b2e54ffda61
I20260709 13:16:24.292372  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet e5a761916a9242a99d2d5b2e54ffda61 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:24.291582  7338 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.293805  7364 tablet_replica.cc:333] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:24.317656  7237 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.321688  6671 raft_consensus.cc:2955] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.330770  7364 raft_consensus.cc:2243] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:24.333910  7364 raft_consensus.cc:2272] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:24.337452  7364 ts_tablet_manager.cc:1916] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:24.342197  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2cbaf7daabfe40028228e412b8f9214b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:24.335649  6420 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.357796  7364 ts_tablet_manager.cc:1929] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:24.358280  7364 log.cc:1199] T 6be297bb363c43c99d04cccfc55bec8f P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/6be297bb363c43c99d04cccfc55bec8f
I20260709 13:16:24.362179  6671 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:24.368067  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 6be297bb363c43c99d04cccfc55bec8f (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:24.378952  7237 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.378362  6419 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.379865  6591 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:24.380398  6961 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.382263  7320 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:24.406939  6585 tablet_service.cc:1558] Processing DeleteTablet for tablet 2cbaf7daabfe40028228e412b8f9214b with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:24.416666  7320 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.424750  6595 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.430012  7237 raft_consensus.cc:2955] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.449456  7365 tablet_replica.cc:333] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:24.455274  6419 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:24.469429  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet c9bbf06685e047fdaa1b6bc881dc7a93 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:24.470448  7362 tablet_replica.cc:333] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:24.470922  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c9bbf06685e047fdaa1b6bc881dc7a93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:24.471578  7362 raft_consensus.cc:2243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:24.472415  7362 raft_consensus.cc:2272] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:24.474579  7365 raft_consensus.cc:2243] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:24.475309  7365 raft_consensus.cc:2272] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:24.482667  7365 ts_tablet_manager.cc:1916] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:24.483636  7362 ts_tablet_manager.cc:1916] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:24.499104  7365 ts_tablet_manager.cc:1929] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:24.499564  7365 log.cc:1199] T 2cbaf7daabfe40028228e412b8f9214b P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/2cbaf7daabfe40028228e412b8f9214b
I20260709 13:16:24.501869  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet 2cbaf7daabfe40028228e412b8f9214b (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:24.507319  7362 ts_tablet_manager.cc:1929] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:24.507786  7362 log.cc:1199] T c9bbf06685e047fdaa1b6bc881dc7a93 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/c9bbf06685e047fdaa1b6bc881dc7a93
I20260709 13:16:24.509049  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet c9bbf06685e047fdaa1b6bc881dc7a93 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:25.549010  6671 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:25.554836  6636 consensus_peers.cc:597] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34cabfbde16f4e10affcaec358aa0dd4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:25.554879  6595 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.555317  6515 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.556120  7366 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.556982  7326 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.571441  6595 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.569814  7326 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.582263  6515 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.584249  6588 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:25.594609  6562 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7365731c56504a678dc1a5ceac3f8da4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:25.596931  6671 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.597757  6417 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.599543  6515 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.600876  7328 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.602646  7328 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.612535  7352 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.615394  6521 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.618358  6669 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.643893  6515 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:25.658293  6419 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:25.669750  6487 consensus_peers.cc:597] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d9d0fb5a0864567ac4cc6ee615911b6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:25.672433  6595 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.672617  6662 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.673870  7338 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.675827  7349 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:25.694551  7349 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.696821  6595 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.705186  6662 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.715101  6588 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:25.721494  6562 consensus_peers.cc:597] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 627436404a154d80a882f4900dff2948. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:25.724236  6515 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.722147  6416 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.728839  7352 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.733263  6662 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:25.737227  7377 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:25.748467  7328 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.761660  6662 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.763638  6521 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:25.783471  6417 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:26.004043  7389 ts_tablet_manager.cc:933] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:26.006192  7389 tablet_copy_client.cc:323] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:26.007754  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 7365731c56504a678dc1a5ceac3f8da4 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:26.008298  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 7365731c56504a678dc1a5ceac3f8da4 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-7365731c56504a678dc1a5ceac3f8da4
I20260709 13:16:26.015277  6607 tablet_copy_source_session.cc:215] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:26.020851  7389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7365731c56504a678dc1a5ceac3f8da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:26.037736  7389 tablet_copy_client.cc:806] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:26.038282  7389 tablet_copy_client.cc:670] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:26.041661  7389 tablet_copy_client.cc:538] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:26.047941  7389 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:26.063359  7389 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:26.064059  7389 tablet_bootstrap.cc:492] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:26.064584  7389 ts_tablet_manager.cc:1403] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 13:16:26.066784  7389 raft_consensus.cc:359] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.067389  7389 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:26.067962  7389 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.071718  7389 ts_tablet_manager.cc:1434] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:16:26.073246  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-7365731c56504a678dc1a5ceac3f8da4 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:26.073683  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-7365731c56504a678dc1a5ceac3f8da4 on tablet 7365731c56504a678dc1a5ceac3f8da4 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:26.085044  7389 ts_tablet_manager.cc:933] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:26.086593  7389 tablet_copy_client.cc:323] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:26.088030  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 5d9d0fb5a0864567ac4cc6ee615911b6 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:26.088461  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 5d9d0fb5a0864567ac4cc6ee615911b6 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-5d9d0fb5a0864567ac4cc6ee615911b6
I20260709 13:16:26.093290  6532 tablet_copy_source_session.cc:215] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:26.095757  7389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9d0fb5a0864567ac4cc6ee615911b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:26.108605  7389 tablet_copy_client.cc:806] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:26.109122  7389 tablet_copy_client.cc:670] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:26.112449  7389 tablet_copy_client.cc:538] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:26.119033  7389 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:26.135197  7395 ts_tablet_manager.cc:933] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:26.139398  7389 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:26.139853  7395 tablet_copy_client.cc:323] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:26.141479  6681 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet 34cabfbde16f4e10affcaec358aa0dd4 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:26.141992  6681 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet 34cabfbde16f4e10affcaec358aa0dd4 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-34cabfbde16f4e10affcaec358aa0dd4
I20260709 13:16:26.140506  7389 tablet_bootstrap.cc:492] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:26.143208  7389 ts_tablet_manager.cc:1403] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
I20260709 13:16:26.146430  7389 raft_consensus.cc:359] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.147220  7389 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:26.147791  7389 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.149935  7389 ts_tablet_manager.cc:1434] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:26.151638  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-5d9d0fb5a0864567ac4cc6ee615911b6 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:26.152020  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-5d9d0fb5a0864567ac4cc6ee615911b6 on tablet 5d9d0fb5a0864567ac4cc6ee615911b6 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:26.157013  6681 tablet_copy_source_session.cc:215] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:26.169968  7395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34cabfbde16f4e10affcaec358aa0dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:26.194254  7395 tablet_copy_client.cc:806] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:26.194828  7395 tablet_copy_client.cc:670] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:26.265851  7389 ts_tablet_manager.cc:933] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:26.273209  7395 tablet_copy_client.cc:538] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:26.294234  7389 tablet_copy_client.cc:323] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:26.305054  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 627436404a154d80a882f4900dff2948 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:26.306242  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 627436404a154d80a882f4900dff2948 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-627436404a154d80a882f4900dff2948
I20260709 13:16:26.348225  7395 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:26.406538  6607 tablet_copy_source_session.cc:215] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:26.428793  7389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 627436404a154d80a882f4900dff2948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:26.467039  7389 tablet_copy_client.cc:806] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:26.467711  7389 tablet_copy_client.cc:670] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:26.491359  7389 tablet_copy_client.cc:538] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:26.500227  7389 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:26.506666  7395 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:26.507632  7395 tablet_bootstrap.cc:492] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:26.508314  7395 ts_tablet_manager.cc:1403] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.160s	user 0.020s	sys 0.015s
I20260709 13:16:26.512053  7395 raft_consensus.cc:359] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.513332  7395 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:26.514091  7395 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.521449  7395 ts_tablet_manager.cc:1434] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:16:26.523554  6681 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-34cabfbde16f4e10affcaec358aa0dd4 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:26.524240  6681 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-34cabfbde16f4e10affcaec358aa0dd4 on tablet 34cabfbde16f4e10affcaec358aa0dd4 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:26.532655  7389 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:26.533740  7389 tablet_bootstrap.cc:492] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:26.534425  7389 ts_tablet_manager.cc:1403] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.012s
I20260709 13:16:26.539337  7389 raft_consensus.cc:359] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.541494  7389 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:26.542943  7389 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:26.547374  7389 ts_tablet_manager.cc:1434] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:16:26.554430  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-627436404a154d80a882f4900dff2948 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:26.555399  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-627436404a154d80a882f4900dff2948 on tablet 627436404a154d80a882f4900dff2948 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:26.582273  7237 raft_consensus.cc:1217] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:26.652825  7237 raft_consensus.cc:1217] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:26.706766  7237 raft_consensus.cc:1217] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:26.874017  7237 raft_consensus.cc:1217] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:27.067611  7380 raft_consensus.cc:1064] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:27.070603  7380 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.076623  7403 raft_consensus.cc:1064] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:27.080683  7403 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.090749  7237 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.091542  7236 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.093472  7377 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.093834  6521 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.096561  7403 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.098897  6521 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.103206  7328 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:27.108402  6666 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.108760  6595 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.110385  7400 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.111526  7404 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.111997  7380 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.129958  6522 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.129666  7328 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.159651  6416 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.170922  7400 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.198283  7237 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.222489  7236 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:16:27.247133  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61386.9us
I20260709 13:16:27.253163  6595 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.258519  6515 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.265158  6666 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.296399  7386 raft_consensus.cc:1064] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:27.299718  7386 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.300427  6419 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.324755  7413 raft_consensus.cc:1064] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:27.328332  7413 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.339164  7236 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.341470  7430 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.348106  6666 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.351722  7386 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.358636  6669 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.361045  7236 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.361979  6595 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.363592  7414 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:16:27.368817  6521 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:27.368643  7414 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:16:27.370253  7401 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:27.372725  7328 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:27.390048  7429 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.417997  6595 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.425103  6592 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.427095  6521 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.428390  7236 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.428376  7414 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:27.430150  7413 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:27.436010  7413 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.441349  6515 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.446926  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7365731c56504a678dc1a5ceac3f8da4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:27.455677  6669 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.468626  6419 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.483022  7236 raft_consensus.cc:2955] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.494149  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 7365731c56504a678dc1a5ceac3f8da4 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:27.500716  7413 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.518548  7440 tablet_replica.cc:333] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:27.520139  7440 raft_consensus.cc:2243] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:27.527436  7440 raft_consensus.cc:2272] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:27.532436  7440 ts_tablet_manager.cc:1916] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:27.541325  6521 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.550114  7235 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.554606  7400 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:27.556146  6666 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.567459  7418 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.570175  7440 ts_tablet_manager.cc:1929] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:27.570974  7440 log.cc:1199] T 7365731c56504a678dc1a5ceac3f8da4 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/7365731c56504a678dc1a5ceac3f8da4
I20260709 13:16:27.573498  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 7365731c56504a678dc1a5ceac3f8da4 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:27.576334  6669 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.592875  7234 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.615398  6419 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:16:27.624068  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 70924.2us
I20260709 13:16:27.629944  6418 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.635558  6515 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.644690  6585 tablet_service.cc:1558] Processing DeleteTablet for tablet 34cabfbde16f4e10affcaec358aa0dd4 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:27.647781  7234 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.640094  7236 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.655314  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 34cabfbde16f4e10affcaec358aa0dd4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:27.657707  7403 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:27.664093  6521 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.676538  6588 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.678063  7401 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:27.679844  6521 raft_consensus.cc:2955] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.699220  7237 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.687070  7429 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.690227  6588 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.691836  6420 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.703145  7401 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:27.708328  7236 raft_consensus.cc:2955] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.722872  7442 tablet_replica.cc:333] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:27.724232  7442 raft_consensus.cc:2243] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:27.724948  7442 raft_consensus.cc:2272] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:27.726456  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5d9d0fb5a0864567ac4cc6ee615911b6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:27.736230  6419 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.744421  7442 ts_tablet_manager.cc:1916] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:27.760304  7442 ts_tablet_manager.cc:1929] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:27.760878  7442 log.cc:1199] T 34cabfbde16f4e10affcaec358aa0dd4 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/34cabfbde16f4e10affcaec358aa0dd4
I20260709 13:16:27.761790  6592 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:27.763494  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet 34cabfbde16f4e10affcaec358aa0dd4 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:27.764175  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 5d9d0fb5a0864567ac4cc6ee615911b6 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:27.765086  7440 tablet_replica.cc:333] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:27.766254  7440 raft_consensus.cc:2243] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:27.766907  7440 raft_consensus.cc:2272] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:27.768627  7236 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.770234  7380 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:27.770308  7440 ts_tablet_manager.cc:1916] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:27.777738  7328 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.779434  7236 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.785605  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 627436404a154d80a882f4900dff2948 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:27.789119  6666 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:27.796092  7380 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:27.794778  6420 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.803031  6666 raft_consensus.cc:2955] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:27.806608  7440 ts_tablet_manager.cc:1929] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:27.807071  7440 log.cc:1199] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/5d9d0fb5a0864567ac4cc6ee615911b6
I20260709 13:16:27.808761  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 5d9d0fb5a0864567ac4cc6ee615911b6 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:27.815518  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 627436404a154d80a882f4900dff2948 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:27.819602  7443 tablet_replica.cc:333] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:27.820987  7443 raft_consensus.cc:2243] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:27.821708  7443 raft_consensus.cc:2272] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:27.829285  7443 ts_tablet_manager.cc:1916] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:27.842464  7443 ts_tablet_manager.cc:1929] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:27.842888  7443 log.cc:1199] T 627436404a154d80a882f4900dff2948 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/627436404a154d80a882f4900dff2948
I20260709 13:16:27.845263  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 627436404a154d80a882f4900dff2948 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
W20260709 13:16:28.859127  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.003s	sys 0.018s
W20260709 13:16:28.859393  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.003s	sys 0.018s
I20260709 13:16:28.909593  7413 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:28.910149  7413 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:28.912220  7413 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:28.947680  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:28.978312  6513 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:28.979915  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:28.992080  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73788.4us
W20260709 13:16:28.994804  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76348.8us
I20260709 13:16:29.011888  6662 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:29.012012  6487 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f67b820295ac47a4b54e21400f8b4201. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:29.013399  7401 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:29.018360  6591 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:29.021217  7427 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:29.023667  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08001d3a20 after lost signal to thread 5413
W20260709 13:16:29.024710  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08000c70c0 after lost signal to thread 6454
W20260709 13:16:29.025923  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7ca0 after lost signal to thread 6546
W20260709 13:16:29.026985  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080011aaa0 after lost signal to thread 6621
W20260709 13:16:29.028129  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08002dafe0 after lost signal to thread 6695
W20260709 13:16:29.029206  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080031a100 after lost signal to thread 7261
I20260709 13:16:29.028306  6662 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.026207  7427 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.059615  6561 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:29.061141  7380 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:29.089670  6588 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:29.097290  6419 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:29.107869  6562 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 542a321aae654f82803540bffcf7968f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:29.108048  6662 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:29.109535  7380 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:29.110060  6513 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:29.113171  7377 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:29.116549  6588 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.136713  7380 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.147871  6662 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.150249  6513 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.162864  6522 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:29.189791  6487 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c870451bc2945e8b0a88e8fedf73ce6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:29.194070  6662 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:29.194073  6588 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:29.196437  7445 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:29.198356  7427 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:29.228868  6662 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.225162  7445 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.333405  6588 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.339757  6668 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:29.364876  6420 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:29.381279  6636 consensus_peers.cc:597] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c0a546cb4d749bc83ffa9cbd0f40297. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:29.382337  6595 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:29.384290  6513 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:29.386358  7454 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:29.366451  6420 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.418958  7422 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.421547  7403 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:29.426323  6591 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.435245  6522 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.446807  6420 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:29.524715  7475 ts_tablet_manager.cc:933] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:29.526657  7475 tablet_copy_client.cc:323] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:29.531368  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 542a321aae654f82803540bffcf7968f from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:29.531960  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 542a321aae654f82803540bffcf7968f from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-542a321aae654f82803540bffcf7968f
I20260709 13:16:29.540392  6607 tablet_copy_source_session.cc:215] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:29.545426  7475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542a321aae654f82803540bffcf7968f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.561153  7475 tablet_copy_client.cc:806] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:29.561718  7475 tablet_copy_client.cc:670] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:29.593065  7475 tablet_copy_client.cc:538] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:29.601442  7475 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:29.608596  7477 ts_tablet_manager.cc:933] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:29.630203  7475 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:29.622036  7477 tablet_copy_client.cc:323] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:29.631399  7475 tablet_bootstrap.cc:492] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:29.632202  7475 ts_tablet_manager.cc:1403] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.007s
I20260709 13:16:29.632792  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet f67b820295ac47a4b54e21400f8b4201 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:29.633376  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet f67b820295ac47a4b54e21400f8b4201 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-f67b820295ac47a4b54e21400f8b4201
I20260709 13:16:29.640439  7475 raft_consensus.cc:359] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:29.641115  6532 tablet_copy_source_session.cc:215] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:29.644656  7477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f67b820295ac47a4b54e21400f8b4201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.641337  7475 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:29.645833  7475 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:29.662060  7475 ts_tablet_manager.cc:1434] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.008s
I20260709 13:16:29.663934  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-542a321aae654f82803540bffcf7968f received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:29.664381  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-542a321aae654f82803540bffcf7968f on tablet 542a321aae654f82803540bffcf7968f with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:29.680385  7477 tablet_copy_client.cc:806] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:29.681239  7477 tablet_copy_client.cc:670] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:29.695973  7477 tablet_copy_client.cc:538] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:29.704715  7477 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:29.737342  7475 ts_tablet_manager.cc:933] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:29.781545  7477 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:29.792169  7477 tablet_bootstrap.cc:492] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:29.792917  7477 ts_tablet_manager.cc:1403] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.015s
I20260709 13:16:29.804607  7475 tablet_copy_client.cc:323] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:29.808403  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 4c870451bc2945e8b0a88e8fedf73ce6 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:29.808995  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 4c870451bc2945e8b0a88e8fedf73ce6 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-4c870451bc2945e8b0a88e8fedf73ce6
I20260709 13:16:29.816708  7477 raft_consensus.cc:359] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:29.818022  7477 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:29.818763  7477 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:29.825469  7477 ts_tablet_manager.cc:1434] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.032s	user 0.002s	sys 0.006s
I20260709 13:16:29.828706  6531 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-f67b820295ac47a4b54e21400f8b4201 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:29.829129  6531 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-f67b820295ac47a4b54e21400f8b4201 on tablet f67b820295ac47a4b54e21400f8b4201 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:29.858268  6532 tablet_copy_source_session.cc:215] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:29.868187  7475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c870451bc2945e8b0a88e8fedf73ce6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.886790  7475 tablet_copy_client.cc:806] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:29.887674  7477 ts_tablet_manager.cc:933] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:29.894940  7475 tablet_copy_client.cc:670] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:29.899312  7475 tablet_copy_client.cc:538] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:29.911214  7475 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:29.913564  7477 tablet_copy_client.cc:323] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:29.915354  6681 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet 2c0a546cb4d749bc83ffa9cbd0f40297 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:29.915971  6681 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet 2c0a546cb4d749bc83ffa9cbd0f40297 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-2c0a546cb4d749bc83ffa9cbd0f40297
I20260709 13:16:29.930024  6681 tablet_copy_source_session.cc:215] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:29.934381  7477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c0a546cb4d749bc83ffa9cbd0f40297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:29.982980  7237 raft_consensus.cc:1217] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:30.031489  7477 tablet_copy_client.cc:806] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:30.032655  7475 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:30.032096  7477 tablet_copy_client.cc:670] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:30.033579  7475 tablet_bootstrap.cc:492] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:30.034291  7475 ts_tablet_manager.cc:1403] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.123s	user 0.020s	sys 0.023s
I20260709 13:16:30.037591  7475 raft_consensus.cc:359] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:30.038235  7477 tablet_copy_client.cc:538] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:30.038455  7475 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:30.039213  7475 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:30.044149  7475 ts_tablet_manager.cc:1434] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:16:30.045951  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-4c870451bc2945e8b0a88e8fedf73ce6 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:30.046381  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-4c870451bc2945e8b0a88e8fedf73ce6 on tablet 4c870451bc2945e8b0a88e8fedf73ce6 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:30.058061  7237 raft_consensus.cc:1217] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:30.074223  7477 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:30.106365  7477 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:30.107266  7477 tablet_bootstrap.cc:492] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:30.107914  7477 ts_tablet_manager.cc:1403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.034s	user 0.030s	sys 0.004s
I20260709 13:16:30.110832  7477 raft_consensus.cc:359] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:30.111661  7477 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:30.112252  7477 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:30.120015  7477 ts_tablet_manager.cc:1434] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:16:30.123281  6681 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-2c0a546cb4d749bc83ffa9cbd0f40297 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:30.123697  6681 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-2c0a546cb4d749bc83ffa9cbd0f40297 on tablet 2c0a546cb4d749bc83ffa9cbd0f40297 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:30.293068  7237 raft_consensus.cc:1217] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:30.335726  7237 raft_consensus.cc:1217] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:30.604763  7377 raft_consensus.cc:1064] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:30.607199  7377 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:30.613447  7427 raft_consensus.cc:1064] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:30.615922  7427 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:30.634462  7237 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.637991  7427 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:30.639065  6664 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.643174  7445 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:30.650059  6595 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.664312  7445 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:30.669083  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 48284.1us
I20260709 13:16:30.695186  7237 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.701457  7485 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:30.720520  6513 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.722813  7485 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:30.731925  7446 raft_consensus.cc:1064] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:30.734575  7446 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:30.738289  6668 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.756723  7236 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.757319  6668 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.760722  7445 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:30.763571  7446 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:30.767629  6592 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.772958  7446 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:30.756274  7490 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:16:30.811138  7445 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.811326  7413 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.816157  6664 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.825727  6513 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.828352  7485 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:30.845456  7236 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.877315  6419 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.879705  7452 raft_consensus.cc:1064] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:30.882289  7452 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:30.891474  6664 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.904225  7236 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:30.888574  7445 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.911315  7454 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:30.938119  7236 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.952612  7237 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:30.988508  6418 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:31.030032  6522 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:31.032419  6591 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:31.036182  6590 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.029374  6592 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:31.040339  6594 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.050791  7236 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.053936  7505 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:31.056854  6513 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.065474  6664 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.072948  7485 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:31.081013  7499 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.080349  6418 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.096596  7236 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.107182  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 542a321aae654f82803540bffcf7968f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:31.111914  7506 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:31.119692  6513 raft_consensus.cc:2955] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.129168  7403 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:31.142966  6519 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:31.172194  7236 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.174731  7427 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:31.177296  6419 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.185812  6664 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.185400  6519 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:31.190740  7426 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:31.198851  7445 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.201821  7236 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.204748  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 542a321aae654f82803540bffcf7968f with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:31.204195  7236 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.206826  7427 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:31.216425  6669 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.216643  7426 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.221730  7427 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:31.224012  7237 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:16:31.225359  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 64209.1us
I20260709 13:16:31.236413  7403 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.248991  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f67b820295ac47a4b54e21400f8b4201 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:31.248538  6669 raft_consensus.cc:2955] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.254408  6513 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.259990  6664 raft_consensus.cc:2955] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.270015  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4c870451bc2945e8b0a88e8fedf73ce6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:31.277258  7236 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.281035  6594 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.282429  7516 tablet_replica.cc:333] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:31.271428  6416 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.291656  7516 raft_consensus.cc:2243] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:31.292353  7516 raft_consensus.cc:2272] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:31.299860  7516 ts_tablet_manager.cc:1916] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:31.307004  6419 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:31.308162  6420 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:31.313850  6585 tablet_service.cc:1558] Processing DeleteTablet for tablet f67b820295ac47a4b54e21400f8b4201 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:31.323709  7519 tablet_replica.cc:333] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:31.325116  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c870451bc2945e8b0a88e8fedf73ce6 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:31.326572  7519 raft_consensus.cc:2243] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:31.327351  7519 raft_consensus.cc:2272] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:31.329123  6664 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:31.333981  7236 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.336133  7515 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:31.339286  6594 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:31.342686  7519 ts_tablet_manager.cc:1916] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:31.344302  7515 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.348609  7236 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.360476  7506 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:31.364305  7516 ts_tablet_manager.cc:1929] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:31.373111  7516 log.cc:1199] T 542a321aae654f82803540bffcf7968f P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/542a321aae654f82803540bffcf7968f
I20260709 13:16:31.367067  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2c0a546cb4d749bc83ffa9cbd0f40297 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:31.375085  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 542a321aae654f82803540bffcf7968f (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:31.376405  7519 ts_tablet_manager.cc:1929] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:31.376880  7519 log.cc:1199] T f67b820295ac47a4b54e21400f8b4201 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/f67b820295ac47a4b54e21400f8b4201
I20260709 13:16:31.378552  7519 tablet_replica.cc:333] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:31.378471  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet f67b820295ac47a4b54e21400f8b4201 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:31.379859  7519 raft_consensus.cc:2243] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:31.380623  7519 raft_consensus.cc:2272] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:31.384502  7519 ts_tablet_manager.cc:1916] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:31.384580  6594 raft_consensus.cc:2955] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:31.400507  7519 ts_tablet_manager.cc:1929] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:31.400915  7519 log.cc:1199] T 4c870451bc2945e8b0a88e8fedf73ce6 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/4c870451bc2945e8b0a88e8fedf73ce6
I20260709 13:16:31.408102  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet 4c870451bc2945e8b0a88e8fedf73ce6 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:31.407820  6416 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:31.427554  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c0a546cb4d749bc83ffa9cbd0f40297 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:31.435839  7520 tablet_replica.cc:333] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:31.436755  7520 raft_consensus.cc:2243] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:31.437520  7520 raft_consensus.cc:2272] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:31.443867  7520 ts_tablet_manager.cc:1916] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:31.456933  7520 ts_tablet_manager.cc:1929] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:31.457309  7520 log.cc:1199] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/2c0a546cb4d749bc83ffa9cbd0f40297
I20260709 13:16:31.459360  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 2c0a546cb4d749bc83ffa9cbd0f40297 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:32.512235  6519 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:32.523268  6664 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:32.524610  6487 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffa871a3f5334bd49b9c75539f218332. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.524637  6592 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:32.525105  7445 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.527895  7445 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.535743  7517 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.537623  6592 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.561838  6420 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.563710  6664 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:32.566308  6669 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:32.577316  6636 consensus_peers.cc:597] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60fb01da2cc5490ebbbca7ec57b64a20. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.584256  6592 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:32.586022  7521 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.590706  6519 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:32.598299  7515 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.616010  7529 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.635957  6592 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.662328  6515 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.673659  6592 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:32.705412  6416 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:32.718809  6562 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56ba1578b97b4faa966168899769e067. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.725816  6669 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:32.727514  7485 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.731530  6519 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:32.733981  7485 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.739410  7499 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.750144  6669 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.768930  6416 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.781987  6669 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:32.787498  6515 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:32.796475  6636 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f8a48eeccdac45838ef8456e5e9332b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.799847  6519 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:32.800437  6594 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:32.801383  7529 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:32.802568  7531 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:32.815595  7521 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.817130  6519 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.845963  6416 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.887239  6594 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:32.951963  7546 ts_tablet_manager.cc:933] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:32.954553  7546 tablet_copy_client.cc:323] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:32.956357  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet ffa871a3f5334bd49b9c75539f218332 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:32.956923  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet ffa871a3f5334bd49b9c75539f218332 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-ffa871a3f5334bd49b9c75539f218332
I20260709 13:16:32.965785  6532 tablet_copy_source_session.cc:215] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:32.969611  7546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffa871a3f5334bd49b9c75539f218332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:32.993496  7546 tablet_copy_client.cc:806] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:32.994300  7546 tablet_copy_client.cc:670] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:32.998981  7546 tablet_copy_client.cc:538] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:33.013600  7546 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:33.046079  7546 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:33.047195  7546 tablet_bootstrap.cc:492] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:33.047991  7546 ts_tablet_manager.cc:1403] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.009s
I20260709 13:16:33.051226  7546 raft_consensus.cc:359] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.052171  7546 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:33.053119  7546 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.063515  7546 ts_tablet_manager.cc:1434] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:16:33.068578  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-ffa871a3f5334bd49b9c75539f218332 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:33.069047  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-ffa871a3f5334bd49b9c75539f218332 on tablet ffa871a3f5334bd49b9c75539f218332 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:33.108707  7546 ts_tablet_manager.cc:933] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:33.122156  7546 tablet_copy_client.cc:323] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:33.124363  6681 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet 60fb01da2cc5490ebbbca7ec57b64a20 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:33.124977  6681 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet 60fb01da2cc5490ebbbca7ec57b64a20 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-60fb01da2cc5490ebbbca7ec57b64a20
I20260709 13:16:33.145960  6681 tablet_copy_source_session.cc:215] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:33.149591  7546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fb01da2cc5490ebbbca7ec57b64a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:33.188992  7546 tablet_copy_client.cc:806] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:33.189673  7546 tablet_copy_client.cc:670] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:33.193799  7546 tablet_copy_client.cc:538] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:33.204515  7546 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:33.235073  7546 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:33.236054  7546 tablet_bootstrap.cc:492] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:33.236728  7546 ts_tablet_manager.cc:1403] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.012s
I20260709 13:16:33.240305  7546 raft_consensus.cc:359] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.242219  7546 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:33.243131  7546 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.250144  7546 ts_tablet_manager.cc:1434] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:16:33.252226  6681 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-60fb01da2cc5490ebbbca7ec57b64a20 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:33.252676  6681 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-60fb01da2cc5490ebbbca7ec57b64a20 on tablet 60fb01da2cc5490ebbbca7ec57b64a20 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:33.297159  7546 ts_tablet_manager.cc:933] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:33.299115  7546 tablet_copy_client.cc:323] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:33.300923  6681 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet f8a48eeccdac45838ef8456e5e9332b5 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:33.301489  6681 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet f8a48eeccdac45838ef8456e5e9332b5 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-f8a48eeccdac45838ef8456e5e9332b5
I20260709 13:16:33.323350  6681 tablet_copy_source_session.cc:215] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:33.326584  7546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8a48eeccdac45838ef8456e5e9332b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:33.338302  7553 ts_tablet_manager.cc:933] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:33.363108  7553 tablet_copy_client.cc:323] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:33.365463  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 56ba1578b97b4faa966168899769e067 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:33.366048  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 56ba1578b97b4faa966168899769e067 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-56ba1578b97b4faa966168899769e067
I20260709 13:16:33.374598  7236 raft_consensus.cc:1217] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:33.378201  6607 tablet_copy_source_session.cc:215] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:33.385983  7546 tablet_copy_client.cc:806] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:33.386611  7546 tablet_copy_client.cc:670] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:33.393882  7553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ba1578b97b4faa966168899769e067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:33.417140  7553 tablet_copy_client.cc:806] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:33.417790  7553 tablet_copy_client.cc:670] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:33.425671  7553 tablet_copy_client.cc:538] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:33.428071  7546 tablet_copy_client.cc:538] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:33.437332  7546 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:33.447923  7553 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:33.506556  7553 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:33.507548  7553 tablet_bootstrap.cc:492] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:33.508423  7553 ts_tablet_manager.cc:1403] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.004s
I20260709 13:16:33.511507  7553 raft_consensus.cc:359] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.512365  7553 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:33.512995  7553 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.527392  7553 ts_tablet_manager.cc:1434] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:16:33.533313  7546 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:33.536360  7546 tablet_bootstrap.cc:492] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:33.537030  7546 ts_tablet_manager.cc:1403] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.100s	user 0.018s	sys 0.011s
I20260709 13:16:33.539778  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-56ba1578b97b4faa966168899769e067 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:33.539657  7546 raft_consensus.cc:359] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.540261  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-56ba1578b97b4faa966168899769e067 on tablet 56ba1578b97b4faa966168899769e067 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:33.540437  7546 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:33.541052  7546 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:33.545526  7546 ts_tablet_manager.cc:1434] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:16:33.548583  6681 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-f8a48eeccdac45838ef8456e5e9332b5 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:33.549006  6681 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-f8a48eeccdac45838ef8456e5e9332b5 on tablet f8a48eeccdac45838ef8456e5e9332b5 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:33.593722  7237 raft_consensus.cc:1217] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:33.677173  7237 raft_consensus.cc:1217] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:33.873324  7236 raft_consensus.cc:1217] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:33.985502  7526 raft_consensus.cc:1064] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:33.987462  7526 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:33.997035  7236 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.006026  7517 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.006383  6664 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.007972  6594 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.013717  7533 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:34.017815  7559 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.039592  6594 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.040011  7517 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.045101  6664 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.055222  7521 raft_consensus.cc:1064] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:34.062093  7521 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:34.072814  6416 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.074723  7236 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.098280  6594 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.100188  7236 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.101392  6515 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.101496  7529 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.102428  7563 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.106714  7564 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.132834  7521 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.134081  6590 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.138847  6515 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.153702  6420 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.169163  6515 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:34.174440  7236 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.186040  7237 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.186664  6669 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.187507  7517 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.190215  7558 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.200706  7558 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.207207  6669 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.204814  7237 raft_consensus.cc:2955] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.246466  6664 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:34.252514  7234 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.258909  6594 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.267876  7563 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.268129  6418 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.272365  7563 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.277428  7521 raft_consensus.cc:1064] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:34.281414  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet ffa871a3f5334bd49b9c75539f218332 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:34.282788  7521 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:34.282653  7403 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.286954  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ffa871a3f5334bd49b9c75539f218332 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:34.288820  7574 tablet_replica.cc:333] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:34.289790  7574 raft_consensus.cc:2243] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:34.290378  7574 raft_consensus.cc:2272] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:34.294775  6594 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.291561  7234 raft_consensus.cc:2955] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.301528  7574 ts_tablet_manager.cc:1916] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:34.313161  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60fb01da2cc5490ebbbca7ec57b64a20 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:34.325341  7574 ts_tablet_manager.cc:1929] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:34.325881  7574 log.cc:1199] T ffa871a3f5334bd49b9c75539f218332 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/ffa871a3f5334bd49b9c75539f218332
I20260709 13:16:34.328917  7237 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.329401  6418 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:34.333709  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet ffa871a3f5334bd49b9c75539f218332 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:34.336021  6592 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.338274  6515 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.350194  7564 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:34.351266  7403 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.356998  7521 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.387281  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 60fb01da2cc5490ebbbca7ec57b64a20 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:34.394896  7579 tablet_replica.cc:333] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:34.395920  7579 raft_consensus.cc:2243] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:34.398543  7579 raft_consensus.cc:2272] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:34.397342  7563 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.408527  7579 ts_tablet_manager.cc:1916] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:34.407791  6515 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.419092  6592 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.423993  6418 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:34.436196  7237 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.445297  7579 ts_tablet_manager.cc:1929] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:34.445861  7579 log.cc:1199] T 60fb01da2cc5490ebbbca7ec57b64a20 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/60fb01da2cc5490ebbbca7ec57b64a20
I20260709 13:16:34.447618  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 60fb01da2cc5490ebbbca7ec57b64a20 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:34.456306  7573 raft_consensus.cc:1064] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:34.459290  7573 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:34.461422  6669 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:34.473408  6519 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.474803  7563 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.476200  7237 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.477623  7234 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.478190  6515 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.481340  7542 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.482702  7521 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.483443  7542 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.484871  6519 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.481364  7403 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.488178  7237 raft_consensus.cc:2955] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.478878  6669 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:34.495047  7485 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:34.511898  6420 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.514066  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f8a48eeccdac45838ef8456e5e9332b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:34.543639  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet f8a48eeccdac45838ef8456e5e9332b5 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:34.544579  7574 tablet_replica.cc:333] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:34.545557  7574 raft_consensus.cc:2243] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:34.546187  7574 raft_consensus.cc:2272] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:34.551730  7574 ts_tablet_manager.cc:1916] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:34.554697  7573 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.574265  7574 ts_tablet_manager.cc:1929] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:34.576241  7574 log.cc:1199] T f8a48eeccdac45838ef8456e5e9332b5 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/f8a48eeccdac45838ef8456e5e9332b5
I20260709 13:16:34.580365  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet f8a48eeccdac45838ef8456e5e9332b5 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:34.584137  7234 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.588613  6515 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.592736  6669 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.627658  6420 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.656922  6594 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:34.663940  7234 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.664280  6515 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:34.666191  7485 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.667037  7584 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:34.674528  7542 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.675545  6515 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.676755  7234 raft_consensus.cc:2955] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.692518  6420 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:34.698693  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 56ba1578b97b4faa966168899769e067 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:34.704656  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 56ba1578b97b4faa966168899769e067 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:34.707026  7585 tablet_replica.cc:333] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:34.707999  7585 raft_consensus.cc:2243] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:34.708961  7585 raft_consensus.cc:2272] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:34.711786  7585 ts_tablet_manager.cc:1916] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:34.727509  7585 ts_tablet_manager.cc:1929] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:34.728034  7585 log.cc:1199] T 56ba1578b97b4faa966168899769e067 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/56ba1578b97b4faa966168899769e067
I20260709 13:16:34.730955  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 56ba1578b97b4faa966168899769e067 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:35.792405  6590 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:35.799297  6562 consensus_peers.cc:597] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d598d50f158e42a6b8d5e297f271951d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.799503  6519 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.800170  6669 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.801054  7587 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.801970  7542 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.809317  7587 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.811044  6519 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.823845  6590 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:35.843025  6418 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:35.844462  6562 consensus_peers.cc:597] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17b598f21a79476cabdf3f5191b4f9a8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.844704  6519 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.846169  7593 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.853835  6669 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.860955  7542 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.870198  6513 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.875327  6664 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.886240  7587 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.896931  6662 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:35.900075  6664 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:35.922158  6636 consensus_peers.cc:597] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8885ae897ec84eca940e2aa65f7311b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.922425  6594 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.923611  6513 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.923926  7586 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.925624  7529 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.927481  6418 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.941773  7586 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.943679  6594 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.945152  6513 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.972466  6418 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:35.975607  6664 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:35.998590  6513 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:36.001224  6594 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:36.002717  6636 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 004d58f566fd4b6d954bc425675471fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:36.009846  7564 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.010805  7604 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.025141  7564 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:36.026458  6594 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:36.030284  6515 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:36.044252  6420 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:36.216634  7609 ts_tablet_manager.cc:933] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:36.226046  7609 tablet_copy_client.cc:323] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:36.227660  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet d598d50f158e42a6b8d5e297f271951d from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:36.228190  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet d598d50f158e42a6b8d5e297f271951d from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-d598d50f158e42a6b8d5e297f271951d
I20260709 13:16:36.236531  6607 tablet_copy_source_session.cc:215] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:36.239820  7609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d598d50f158e42a6b8d5e297f271951d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:36.255781  7609 tablet_copy_client.cc:806] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:36.256330  7609 tablet_copy_client.cc:670] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:36.261528  7609 tablet_copy_client.cc:538] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:36.270447  7609 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:36.290763  7609 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.291755  7609 tablet_bootstrap.cc:492] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:36.292434  7609 ts_tablet_manager.cc:1403] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.013s
I20260709 13:16:36.295305  7609 raft_consensus.cc:359] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.296195  7609 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:36.296881  7609 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.299628  7612 ts_tablet_manager.cc:933] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:36.308559  7609 ts_tablet_manager.cc:1434] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.006s
I20260709 13:16:36.310271  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-d598d50f158e42a6b8d5e297f271951d received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:36.310703  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-d598d50f158e42a6b8d5e297f271951d on tablet d598d50f158e42a6b8d5e297f271951d with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:36.315685  7612 tablet_copy_client.cc:323] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:36.318248  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 17b598f21a79476cabdf3f5191b4f9a8 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:36.318754  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 17b598f21a79476cabdf3f5191b4f9a8 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-17b598f21a79476cabdf3f5191b4f9a8
I20260709 13:16:36.326313  6607 tablet_copy_source_session.cc:215] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:36.330017  7612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b598f21a79476cabdf3f5191b4f9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:36.344918  7612 tablet_copy_client.cc:806] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:36.345505  7612 tablet_copy_client.cc:670] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:36.419274  7612 tablet_copy_client.cc:538] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:36.446312  7612 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:36.464843  7609 ts_tablet_manager.cc:933] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:36.469257  7609 tablet_copy_client.cc:323] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:36.472007  6681 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet 8885ae897ec84eca940e2aa65f7311b5 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:36.472741  6681 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet 8885ae897ec84eca940e2aa65f7311b5 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-8885ae897ec84eca940e2aa65f7311b5
I20260709 13:16:36.480302  7612 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.481475  7612 tablet_bootstrap.cc:492] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:36.482295  7612 ts_tablet_manager.cc:1403] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.015s
I20260709 13:16:36.484704  6681 tablet_copy_source_session.cc:215] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:36.490732  7609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8885ae897ec84eca940e2aa65f7311b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:36.499145  7612 raft_consensus.cc:359] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.500708  7612 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:36.501564  7612 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.522336  7612 ts_tablet_manager.cc:1434] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 13:16:36.530153  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-17b598f21a79476cabdf3f5191b4f9a8 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:36.530833  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-17b598f21a79476cabdf3f5191b4f9a8 on tablet 17b598f21a79476cabdf3f5191b4f9a8 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:36.536378  7609 tablet_copy_client.cc:806] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:36.537153  7609 tablet_copy_client.cc:670] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:36.541994  7609 tablet_copy_client.cc:538] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:36.568157  7609 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:36.601853  7609 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.602852  7609 tablet_bootstrap.cc:492] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:36.603682  7609 ts_tablet_manager.cc:1403] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.007s
I20260709 13:16:36.605149  7612 ts_tablet_manager.cc:933] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:36.606985  7609 raft_consensus.cc:359] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.608024  7609 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:36.608705  7609 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.610679  7609 ts_tablet_manager.cc:1434] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:16:36.614457  7612 tablet_copy_client.cc:323] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:36.615840  6681 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-8885ae897ec84eca940e2aa65f7311b5 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:36.616343  6681 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-8885ae897ec84eca940e2aa65f7311b5 on tablet 8885ae897ec84eca940e2aa65f7311b5 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:36.616290  6680 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet 004d58f566fd4b6d954bc425675471fa from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:36.617249  6680 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet 004d58f566fd4b6d954bc425675471fa from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-004d58f566fd4b6d954bc425675471fa
I20260709 13:16:36.668126  6680 tablet_copy_source_session.cc:215] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:36.671458  7612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 004d58f566fd4b6d954bc425675471fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:36.692731  7237 raft_consensus.cc:1217] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:36.691274  7612 tablet_copy_client.cc:806] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:36.706653  7612 tablet_copy_client.cc:670] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:36.711727  7612 tablet_copy_client.cc:538] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:36.720909  7612 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:36.730742  7237 raft_consensus.cc:1217] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:36.757954  7612 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.758893  7612 tablet_bootstrap.cc:492] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:36.759578  7612 ts_tablet_manager.cc:1403] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.039s	user 0.024s	sys 0.007s
I20260709 13:16:36.763064  7612 raft_consensus.cc:359] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.763962  7612 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:36.764602  7612 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:36.767580  7612 ts_tablet_manager.cc:1434] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:16:36.770793  6680 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-004d58f566fd4b6d954bc425675471fa received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:36.771165  6680 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-004d58f566fd4b6d954bc425675471fa on tablet 004d58f566fd4b6d954bc425675471fa with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:37.001878  7237 raft_consensus.cc:1217] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:37.070541  7234 raft_consensus.cc:1217] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:37.103715  7592 raft_consensus.cc:1064] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:37.106608  7592 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:37.123854  6662 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.124027  7234 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.125202  6513 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.125401  7592 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.126533  7623 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:37.127468  7542 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.154284  7542 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.157222  6669 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.172710  6515 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.172605  7234 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.174341  6419 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.236171  6592 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:37.240875  6669 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.240965  7234 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.242281  7542 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:37.243968  7593 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:37.251736  7623 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.253340  7234 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.257017  6669 raft_consensus.cc:2955] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.269191  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d598d50f158e42a6b8d5e297f271951d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:37.274390  6420 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.291954  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet d598d50f158e42a6b8d5e297f271951d with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:37.293891  7632 tablet_replica.cc:333] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:37.296056  7632 raft_consensus.cc:2243] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:37.296911  7632 raft_consensus.cc:2272] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:37.300274  7632 ts_tablet_manager.cc:1916] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:37.319026  7542 raft_consensus.cc:1064] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:37.319602  7632 ts_tablet_manager.cc:1929] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:37.320092  7632 log.cc:1199] T d598d50f158e42a6b8d5e297f271951d P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/d598d50f158e42a6b8d5e297f271951d
I20260709 13:16:37.321547  7542 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:37.324937  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet d598d50f158e42a6b8d5e297f271951d (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:37.331418  7234 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.332048  6515 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.333949  7593 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.334955  7592 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.337291  6662 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.338660  7593 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.348304  7592 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.363572  6513 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.366268  6669 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.371672  7234 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.386533  6420 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.414772  6592 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:37.422556  6513 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.423265  7234 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.424424  7542 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:37.425225  7592 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:37.433099  7593 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.433347  6515 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.443216  7234 raft_consensus.cc:2955] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.454561  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 17b598f21a79476cabdf3f5191b4f9a8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:37.459370  6420 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.471138  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 17b598f21a79476cabdf3f5191b4f9a8 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:37.473219  7637 tablet_replica.cc:333] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:37.474035  7637 raft_consensus.cc:2243] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:37.474713  7637 raft_consensus.cc:2272] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:37.479435  7637 ts_tablet_manager.cc:1916] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:37.493341  7637 ts_tablet_manager.cc:1929] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:37.493708  7637 log.cc:1199] T 17b598f21a79476cabdf3f5191b4f9a8 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/17b598f21a79476cabdf3f5191b4f9a8
I20260709 13:16:37.495347  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 17b598f21a79476cabdf3f5191b4f9a8 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:37.602823  7604 raft_consensus.cc:1064] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:37.605093  7604 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:37.611414  7237 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.613344  7586 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:37.615787  6594 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.616050  6515 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.617161  7564 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.618153  7586 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.638631  7586 raft_consensus.cc:1064] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:37.643777  7586 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:37.668228  7646 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.688668  6418 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:37.708118  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 66571.6us
W20260709 13:16:37.715160  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 67358.3us
I20260709 13:16:37.719986  7236 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.721505  6519 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.723039  6522 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.724146  6588 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:37.728152  7645 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:37.731352  6588 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.741892  7564 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:37.753357  7586 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:16:37.790917  7564 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.804584  6513 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.809886  6588 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.814165  7236 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.817588  6418 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:37.843708  6662 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:37.860621  7236 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.861771  6592 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.861984  7564 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:37.863417  7649 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:37.868486  7649 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.870110  7236 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.880703  6592 raft_consensus.cc:2955] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.884351  6418 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:37.887133  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8885ae897ec84eca940e2aa65f7311b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:37.896561  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 8885ae897ec84eca940e2aa65f7311b5 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:37.906185  7655 tablet_replica.cc:333] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:37.907279  7655 raft_consensus.cc:2243] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:37.913733  7655 raft_consensus.cc:2272] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:37.918047  7655 ts_tablet_manager.cc:1916] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:37.931115  7655 ts_tablet_manager.cc:1929] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:37.931471  7655 log.cc:1199] T 8885ae897ec84eca940e2aa65f7311b5 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/8885ae897ec84eca940e2aa65f7311b5
I20260709 13:16:37.932830  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 8885ae897ec84eca940e2aa65f7311b5 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:38.217864  7236 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:38.229001  6662 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:38.238173  7234 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:38.243510  7643 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:38.253394  6513 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:38.254165  7643 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:38.257745  7644 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:38.262244  7236 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:38.265256  6513 raft_consensus.cc:2955] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:38.277537  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 004d58f566fd4b6d954bc425675471fa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:38.280622  6420 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:38.293030  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet 004d58f566fd4b6d954bc425675471fa with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:38.299945  7657 tablet_replica.cc:333] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:38.300933  7657 raft_consensus.cc:2243] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:38.301596  7657 raft_consensus.cc:2272] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:38.304922  7657 ts_tablet_manager.cc:1916] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:38.321261  7657 ts_tablet_manager.cc:1929] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:38.321630  7657 log.cc:1199] T 004d58f566fd4b6d954bc425675471fa P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/004d58f566fd4b6d954bc425675471fa
I20260709 13:16:38.323138  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet 004d58f566fd4b6d954bc425675471fa (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:39.358098  6592 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:39.368265  6562 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85490b85caca40d19a25af108b1da46a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:39.369939  6519 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.371145  7656 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:39.372495  6669 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.373885  7656 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:39.379711  7592 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.381178  6519 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.382951  6669 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.393066  6513 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:39.400815  6418 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:39.413079  6487 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53f3e5f2d81c4952b6ae972bdb3a18c1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:39.413919  6669 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.416278  6588 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.417449  7654 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:39.418495  7517 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:39.434717  6588 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.433319  7654 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.454751  6420 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.468819  6662 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.469519  6669 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:39.476193  6636 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4f0dad199224b8182ae036d97281b72. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:39.476356  6513 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.478385  6588 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.478188  7586 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:39.482038  7643 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:39.494817  7643 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.496338  6513 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.503849  6592 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.515801  6420 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:39.520881  6662 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:39.535419  6636 consensus_peers.cc:597] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8fc61b2fa4ee43b7a7a91b2eb2ecd874. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:39.535899  6592 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.536213  6513 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:39.537461  7586 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:39.538276  7676 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:39.544996  7643 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.546605  6592 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.546859  6513 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.567497  6420 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:39.793782  7680 ts_tablet_manager.cc:933] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:39.795625  7680 tablet_copy_client.cc:323] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:39.797291  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:39.797804  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-53f3e5f2d81c4952b6ae972bdb3a18c1
I20260709 13:16:39.805334  6532 tablet_copy_source_session.cc:215] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:39.808388  7680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:39.827405  7680 tablet_copy_client.cc:806] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:39.828147  7680 tablet_copy_client.cc:670] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:39.838925  7680 tablet_copy_client.cc:538] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:39.851119  7680 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:39.883615  7680 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:39.884359  7680 tablet_bootstrap.cc:492] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:39.884985  7680 ts_tablet_manager.cc:1403] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.008s
I20260709 13:16:39.887797  7683 ts_tablet_manager.cc:933] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:39.888090  7680 raft_consensus.cc:359] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:39.888952  7680 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:39.889627  7680 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:39.890713  7683 tablet_copy_client.cc:323] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:39.892722  7680 ts_tablet_manager.cc:1434] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 13:16:39.893636  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 85490b85caca40d19a25af108b1da46a from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:39.894188  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 85490b85caca40d19a25af108b1da46a from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-85490b85caca40d19a25af108b1da46a
I20260709 13:16:39.898228  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-53f3e5f2d81c4952b6ae972bdb3a18c1 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:39.898689  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-53f3e5f2d81c4952b6ae972bdb3a18c1 on tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:39.903200  6607 tablet_copy_source_session.cc:215] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:39.910820  7683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85490b85caca40d19a25af108b1da46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:39.929996  7683 tablet_copy_client.cc:806] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:39.930544  7683 tablet_copy_client.cc:670] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:39.933897  7683 tablet_copy_client.cc:538] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:39.940414  7683 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:39.960837  7683 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:39.962014  7683 tablet_bootstrap.cc:492] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:39.962699  7683 ts_tablet_manager.cc:1403] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.002s
I20260709 13:16:39.965664  7683 raft_consensus.cc:359] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:39.966706  7683 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:39.967345  7683 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:39.971928  7683 ts_tablet_manager.cc:1434] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:16:39.973569  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-85490b85caca40d19a25af108b1da46a received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:39.973985  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-85490b85caca40d19a25af108b1da46a on tablet 85490b85caca40d19a25af108b1da46a with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:40.044344  7683 ts_tablet_manager.cc:933] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:40.048904  7683 tablet_copy_client.cc:323] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:40.050326  6680 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet f4f0dad199224b8182ae036d97281b72 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:40.050869  6680 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet f4f0dad199224b8182ae036d97281b72 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-f4f0dad199224b8182ae036d97281b72
I20260709 13:16:40.057587  6680 tablet_copy_source_session.cc:215] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:40.060382  7683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4f0dad199224b8182ae036d97281b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:40.082149  7683 tablet_copy_client.cc:806] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:40.082713  7683 tablet_copy_client.cc:670] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:40.085943  7683 tablet_copy_client.cc:538] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:40.092434  7683 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:40.118474  7683 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:40.119671  7683 tablet_bootstrap.cc:492] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:40.120523  7683 ts_tablet_manager.cc:1403] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 13:16:40.130237  7683 raft_consensus.cc:359] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:40.131127  7683 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:40.131757  7683 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:40.134332  7683 ts_tablet_manager.cc:1434] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:16:40.137758  6680 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-f4f0dad199224b8182ae036d97281b72 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:40.138207  6680 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-f4f0dad199224b8182ae036d97281b72 on tablet f4f0dad199224b8182ae036d97281b72 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:40.152204  7683 ts_tablet_manager.cc:933] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:40.154220  7683 tablet_copy_client.cc:323] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:40.158205  6680 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:40.159003  6680 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-8fc61b2fa4ee43b7a7a91b2eb2ecd874
I20260709 13:16:40.184300  6680 tablet_copy_source_session.cc:215] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:40.190749  7683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:40.213409  7683 tablet_copy_client.cc:806] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:40.213984  7683 tablet_copy_client.cc:670] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:40.219808  7683 tablet_copy_client.cc:538] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:40.231722  7683 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:40.246765  7683 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:40.247794  7683 tablet_bootstrap.cc:492] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:40.248387  7683 ts_tablet_manager.cc:1403] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 13:16:40.251101  7683 raft_consensus.cc:359] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:40.251849  7683 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:40.253186  7683 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:40.255774  7683 ts_tablet_manager.cc:1434] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:40.258140  6680 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-8fc61b2fa4ee43b7a7a91b2eb2ecd874 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:40.258585  6680 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-8fc61b2fa4ee43b7a7a91b2eb2ecd874 on tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:40.302671  7236 raft_consensus.cc:1217] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:40.509302  7234 raft_consensus.cc:1217] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:40.640223  7234 raft_consensus.cc:1217] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:40.682413  7234 raft_consensus.cc:1217] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:40.875384  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:40.893757  7692 raft_consensus.cc:1064] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:40.896440  7692 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:40.904486  7236 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:40.906025  7592 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:40.911157  6522 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:40.912446  7692 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:40.921440  7517 raft_consensus.cc:1064] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:40.923712  7517 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:40.955039  6662 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:40.959899  6662 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:40.964984  7592 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:40.966107  7654 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:40.984566  6592 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:40.990521  7517 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:40.991147  7234 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:41.015864  6662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:41.018330  6662 raft_consensus.cc:606] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:41.028349  7692 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.032437  6669 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.049717  7689 raft_consensus.cc:1064] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:41.051331  7654 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:41.052352  7689 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.064441  7699 raft_consensus.cc:993] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.065207  7699 raft_consensus.cc:1081] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.075558  6594 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:41.076171  6594 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:41.076573  6594 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.078423  6519 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.084239  6594 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.086089  6594 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:41.099153  7234 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:41.100734  7689 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:41.102648  6588 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:41.105902  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:41.106520  6669 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.106822  6669 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:41.107604  6669 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.110954  6669 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.120198  6419 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.127372  6669 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.130987  6519 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:41.136453  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:41.137243  6522 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.131788  7695 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:16:41.150123  6522 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.158460  6563 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:41.160226  7234 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.161111  7517 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.174098  7236 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.179843  7695 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:16:41.184233  7664 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:41.224287  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:41.224893  6521 raft_consensus.cc:606] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:41.232317  7664 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:41.251766  7664 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.324478  6669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a72647203b143708b26072192a9c102"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:41.325069  6669 raft_consensus.cc:606] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:16:41.326297  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48389.7us
W20260709 13:16:41.442988  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62020.6us
I20260709 13:16:41.445209  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:41.445894  6668 raft_consensus.cc:606] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:41.467583  6670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:41.468209  6670 raft_consensus.cc:606] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:41.467300  6671 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.476716  7730 raft_consensus.cc:1064] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
W20260709 13:16:41.480859  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 56868.5us
I20260709 13:16:41.498996  6664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:41.499557  6664 raft_consensus.cc:606] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:41.500778  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:41.502388  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:41.507444  7643 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.503129  7730 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.518286  7517 raft_consensus.cc:993] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:41.518759  7517 raft_consensus.cc:1081] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:41.573508  6519 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.579480  7233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46684
I20260709 13:16:41.580127  7233 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:41.580519  7233 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:41.584853  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 78144.9us
I20260709 13:16:41.589255  6593 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.607473  7699 raft_consensus.cc:993] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.608193  7699 raft_consensus.cc:1081] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.617384  7690 raft_consensus.cc:993] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.617985  7699 raft_consensus.cc:1081] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.623241  6588 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:41.623805  6588 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:41.624202  6588 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.624792  6591 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.629684  7237 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.644815  6588 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.645515  6590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:41.647522  6588 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:41.656893  6590 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:41.660894  6590 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.661229  6416 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:41.648077  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:41.668021  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:41.669518  6521 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.669606  6670 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.669999  6670 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:41.670737  6670 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.676020  6521 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.676268  6670 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.680964  7233 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.683655  7233 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:41.684546  6670 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.684928  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:41.698447  6515 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:41.697674  6519 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.699496  6519 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:41.700317  6519 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.701546  6519 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.704464  7696 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:41.691169  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:41.696384  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:41.705483  6593 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.706760  6594 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:41.707943  7722 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:41.708425  7722 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:41.668488  6590 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.709208  7722 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.709517  6592 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.712828  6590 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:41.688927  6419 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:41.714761  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:41.715379  6515 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.717518  6593 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:41.717563  7236 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:41.721436  6521 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:41.723414  6515 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.725082  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:41.725669  7237 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.725586  6419 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.730302  7690 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:41.732257  7730 raft_consensus.cc:993] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.732721  7699 raft_consensus.cc:1081] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.732975  7731 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:41.734840  7731 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:41.742053  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:41.743191  7728 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:41.743661  7728 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:41.742274  7592 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.744511  7728 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.746495  7234 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.753294  6515 raft_consensus.cc:2955] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.754693  7236 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:16:41.755668  6405 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leader transfer in progress [suppressed 2 similar messages]
I20260709 13:16:41.759400  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 85490b85caca40d19a25af108b1da46a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:41.767383  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:41.801556  6670 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.803540  6670 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:41.804725  6670 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.822479  6670 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.823949  6515 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.806078  6519 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
W20260709 13:16:41.819219  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53608.1us
I20260709 13:16:41.828424  6419 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:41.819221  7237 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.815392  7697 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
W20260709 13:16:41.829185  6405 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73542.5us
I20260709 13:16:41.851811  6593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:41.864714  6593 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:41.867475  7234 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:41.876048  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:41.868327  6593 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.884271  6670 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.888296  7697 raft_consensus.cc:993] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.896937  7664 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:41.897809  6593 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.910745  7664 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:41.920120  7654 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:41.921797  7689 raft_consensus.cc:1081] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:41.916741  7664 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:41.941424  7699 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:41.945468  6593 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:41.948834  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:41.950532  6662 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.951228  6662 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:41.952951  6662 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:41.955741  6662 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.965250  6593 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:41.969446  6662 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:41.985473  7233 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:42.050397  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:42.051062  6518 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.051419  6662 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:42.059624  6591 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a72647203b143708b26072192a9c102"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:42.060927  6591 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:42.061578  6591 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.055367  7726 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:42.074896  6518 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:42.066210  6561 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:42.086886  7664 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.088457  7664 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:42.090178  7664 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:42.107203  6521 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:42.114681  6519 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:42.118436  6405 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 53fee9894731461a821a131be24dbbca is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:16:42.779870  6405 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 661471us
I20260709 13:16:42.092761  7233 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:42.105048  7692 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:42.163851  6592 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:42.848271  6520 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:42.459244  6591 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:42.851963  6591 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:42.677804  7725 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.543s
I20260709 13:16:42.704968  6420 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:42.852207  6419 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.104107  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:16:42.860567  6407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 756403us
I20260709 13:16:42.725525  6522 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:42.731998  6515 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:42.742975  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:42.864854  7692 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.760623  7230 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:42.866528  7743 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:42.768090  7730 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.781231  6669 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:42.825736  6665 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:42.867528  7664 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:42.867537  7730 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:42.870952  7730 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:42.837404  6416 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:42.854920  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:42.856117  7751 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:42.874126  6520 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.880367  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:42.884958  7692 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:42.889588  6520 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:42.891947  6563 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:42.896893  6590 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.913054  6518 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.913812  6588 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:42.921769  7754 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.922883  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:42.923508  7730 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.909657  7664 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.938366  7664 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:42.940042  7664 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:42.912325  7743 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:16:42.943387  7751 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:42.924023  7730 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:42.951879  7730 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:42.923954  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:42.909513  7714 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:16:42.960644  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:42.954457  7730 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.000242  6591 raft_consensus.cc:2955] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:16:42.986035  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45582.9us
W20260709 13:16:43.037848  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58097.4us
I20260709 13:16:43.143812  7698 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:16:43.144407  7698 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:16:43.146445  6405 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 53fee9894731461a821a131be24dbbca is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:43.160970  7699 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:43.141237  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 173) took 1050 ms. Trace:
W20260709 13:16:43.180601  7762 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:16:43.182761  7698 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:43.207733  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:43.218111  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:43.176694  7233 rpcz_store.cc:276] 0709 13:16:42.082269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.089791 (+  7522us) service_pool.cc:224] Handling call
0709 13:16:42.090380 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.092616 (+  2236us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:42.999279 (+906663us) log.cc:844] Serialized 14 byte log entry
0709 13:16:43.035928 (+ 36649us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:43.037548 (+  1620us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:43.039193 (+  1645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.040730 (+  1537us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.132807 (+ 92077us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":306,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":3632,"spinlock_wait_cycles":82523520,"thread_start_us":22424,"threads_started":1,"wal-append.queue_time_us":147891}
W20260709 13:16:43.233486  7764 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.220s	user 0.003s	sys 0.000s
W20260709 13:16:43.234484  7765 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:16:43.237210  6420 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 201) took 1385 ms. Trace:
W20260709 13:16:43.186887  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 130666us
I20260709 13:16:43.257911  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 85490b85caca40d19a25af108b1da46a with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
W20260709 13:16:43.267616  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 255984us
I20260709 13:16:43.239403  6420 rpcz_store.cc:276] 0709 13:16:41.853840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.854595 (+   755us) service_pool.cc:224] Handling call
0709 13:16:41.883804 (+ 29209us) spinlock_profiling.cc:243] Waited 7.93 ms on lock 0x7b5400002490. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5d0b2 00007f04beed6a89 00007f04bee6eda6 00007f04bee40088 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc
0709 13:16:41.915501 (+ 31697us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8feb090 00007f04b9089f3a 00007f04bee401cc 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc 00007f04bd3fe392 00007f04b73bc48c 00007f04b73bb8c6
0709 13:16:43.237099 (+1321598us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:42.878197 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:42.878762 (+   565us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:42.878795 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:16:42.878814 (+    19us) tablet.cc:662] Decoding operations
0709 13:16:42.879122 (+   308us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:42.879199 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:42.879228 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:42.879402 (+   174us) tablet.cc:701] Row locks acquired
0709 13:16:42.879420 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:16:42.879556 (+   136us) write_op.cc:270] Start()
0709 13:16:42.879657 (+   101us) write_op.cc:276] Timestamp: P: 1783603002879515 usec, L: 0
0709 13:16:42.879676 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:16:42.880242 (+   566us) log.cc:844] Serialized 532 byte log entry
0709 13:16:43.002766 (+122524us) op_driver.cc:464] REPLICATION: finished
0709 13:16:43.213665 (+210899us) write_op.cc:301] APPLY: starting
0709 13:16:43.216765 (+  3100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:43.222690 (+  5925us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:43.222813 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:43.223055 (+   242us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:43.223227 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:43.223271 (+    44us) write_op.cc:312] APPLY: finished
0709 13:16:43.224034 (+   763us) log.cc:844] Serialized 22 byte log entry
0709 13:16:43.224207 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:43.224281 (+    74us) write_op.cc:454] Released schema lock
0709 13:16:43.224314 (+    33us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:43.224620 (+   306us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:43.224645 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":786106,"reported_tablets":1,"spinlock_wait_cycles":54288512,"child_traces":[["op",{"apply.queue_time_us":202224,"apply.run_cpu_time_us":1759,"apply.run_wall_time_us":10969,"num_ops":1,"prepare.queue_time_us":23757,"prepare.run_cpu_time_us":2408,"prepare.run_wall_time_us":31039,"replication_time_us":99471,"thread_start_us":149468,"threads_started":2}]]}
W20260709 13:16:43.283074  7757 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.265s	user 0.000s	sys 0.002s
I20260709 13:16:43.302542  6665 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
W20260709 13:16:43.305279  7769 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.115s	user 0.000s	sys 0.004s
W20260709 13:16:43.322438  7761 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.307s	user 0.003s	sys 0.000s
W20260709 13:16:43.341473  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 174702us
I20260709 13:16:43.471591  7692 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.472115  7692 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:43.472890  7753 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:43.473444  7753 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:43.474144  7692 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:43.476115  7753 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:43.482896  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:16:43.479655  7719 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.065s	user 0.002s	sys 0.003s
I20260709 13:16:43.491544  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:43.541651  6518 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.541807  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:43.547140  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080012f3c0 after lost signal to thread 5413
W20260709 13:16:43.548202  6618 debug-util.cc:398] Leaking SignalData structure 0x7b08001a4680 after lost signal to thread 6454
W20260709 13:16:43.549291  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080034ff00 after lost signal to thread 6546
W20260709 13:16:43.549552  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080034fea0 after lost signal to thread 6547
W20260709 13:16:43.550462  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080031f5c0 after lost signal to thread 6621
W20260709 13:16:43.551466  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080034cd40 after lost signal to thread 6695
W20260709 13:16:43.551726  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080034cbe0 after lost signal to thread 6696
W20260709 13:16:43.552702  6618 debug-util.cc:398] Leaking SignalData structure 0x7b0800344a00 after lost signal to thread 7261
I20260709 13:16:43.585053  6637 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:16:43.591764  6416 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 204) took 1510 ms. Trace:
I20260709 13:16:43.597864  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:16:43.630069  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:43.631062  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:43.631878  6522 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.587250  7695 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:43.638995  6670 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:43.642863  7724 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:43.643456  7724 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:43.646915  7779 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.647437  7779 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:43.651463  7779 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:43.666004  7780 tablet_replica.cc:333] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:43.600821  7722 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:16:43.679188  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:43.713536  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:43.714370  6668 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:43.719399  6419 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.592267  6416 rpcz_store.cc:276] 0709 13:16:42.081735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.088388 (+  6653us) service_pool.cc:224] Handling call
0709 13:16:42.776736 (+688348us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8feb090 00007f04b9089f3a 00007f04bee401cc 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc 00007f04bd3fe392 00007f04b73bc48c 00007f04b73bb8c6
0709 13:16:43.591695 (+814959us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:43.295635 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:43.299977 (+  4342us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:43.300011 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:16:43.300031 (+    20us) tablet.cc:662] Decoding operations
0709 13:16:43.300354 (+   323us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:43.300393 (+    39us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:43.300418 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:43.300598 (+   180us) tablet.cc:701] Row locks acquired
0709 13:16:43.300616 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:16:43.300739 (+   123us) write_op.cc:270] Start()
0709 13:16:43.300835 (+    96us) write_op.cc:276] Timestamp: P: 1783603003300699 usec, L: 0
0709 13:16:43.300856 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:16:43.323076 (+ 22220us) log.cc:844] Serialized 590 byte log entry
0709 13:16:43.484183 (+161107us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b907ce28 00007f04bcab7581 00007f04bcab6d58 00007f04bcabb9f1 00007f04bcabb9a9 00007f04bcabb939 00007f04bcabb901 00007f04bcabab2d 000000000038f354 000000000038f199
0709 13:16:43.539352 (+ 55169us) op_driver.cc:464] REPLICATION: finished
0709 13:16:43.539731 (+   379us) write_op.cc:301] APPLY: starting
0709 13:16:43.539808 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:43.539902 (+    94us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:43.539923 (+    21us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:43.540106 (+   183us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:43.540239 (+   133us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:43.540279 (+    40us) write_op.cc:312] APPLY: finished
0709 13:16:43.540450 (+   171us) log.cc:844] Serialized 22 byte log entry
0709 13:16:43.540595 (+   145us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:43.540664 (+    69us) write_op.cc:454] Released schema lock
0709 13:16:43.540697 (+    33us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:43.541011 (+   314us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:43.541036 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":24360704,"child_traces":[["op",{"apply.queue_time_us":212,"apply.run_cpu_time_us":1225,"apply.run_wall_time_us":1233,"num_ops":1,"prepare.queue_time_us":421681,"prepare.run_cpu_time_us":4394,"prepare.run_wall_time_us":188700,"replication_time_us":238386,"spinlock_wait_cycles":13485952}]]}
I20260709 13:16:43.722150  6419 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.721774  6486 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:43.737251  7779 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.744650  7779 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.745029  7779 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.728693  7724 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:43.749828  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:43.750550  7236 raft_consensus.cc:1813] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Handling vote request from an unknown peer 53fee9894731461a821a131be24dbbca
I20260709 13:16:43.751454  7236 raft_consensus.cc:2410] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:16:43.754887  7780 raft_consensus.cc:2243] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:43.724372  6419 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:43.760848  7760 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:43.767073  6419 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.781394  7780 raft_consensus.cc:2272] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:43.796512  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 160602us
I20260709 13:16:43.822252  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:43.793481  6665 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.762132  7760 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:43.828212  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:43.828598  7692 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:16:43.938567  7592 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:43.989418  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 354422us
W20260709 13:16:44.002416  7790 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:16:44.016340  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.018992  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:44.036206  7760 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:16:44.053822  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 62377.6us
I20260709 13:16:44.061736  7779 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:44.063938  7780 ts_tablet_manager.cc:1916] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:44.096014  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:16:44.098114  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56555.8us
I20260709 13:16:44.101015  6515 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:44.109723  7233 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:44.095232  7798 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:44.121915  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:44.130507  7779 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:44.129711  7654 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:44.139963  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 291712us
I20260709 13:16:44.150084  6636 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:16:44.121256  7798 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:44.156154  7696 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:44.157917  7798 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:44.170995  6521 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:44.280967  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:44.281584  6669 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.287429  7697 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:16:44.288024  7697 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:44.196556  6668 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:44.314591  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 193971us
I20260709 13:16:44.312510  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:16:44.316994  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 197667us
I20260709 13:16:44.278435  7695 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:16:44.326296  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:44.326567  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:44.316241  7697 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:44.336899  7517 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:44.340489  7517 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:44.339591  6669 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:16:44.314407  7664 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:16:44.370796  7517 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:44.352459  7780 ts_tablet_manager.cc:1929] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:44.380609  7780 log.cc:1199] T 85490b85caca40d19a25af108b1da46a P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/85490b85caca40d19a25af108b1da46a
I20260709 13:16:44.375404  7739 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:16:44.392344  7775 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:44.392839  7775 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:44.380381  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:44.393839  7775 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:44.400887  7806 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:16:44.416877  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:44.407528  7760 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:44.420437  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:44.425284  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:44.442951  7766 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:16:44.422998  7780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45428 (request call id 78) took 1166 ms. Trace:
I20260709 13:16:44.464301  6486 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:44.465132  7517 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:44.469612  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:44.471342  7517 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:44.477542  7517 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:44.486989  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:44.538375  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:44.515882  7724 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:44.562984  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:44.451277  7780 rpcz_store.cc:276] 0709 13:16:43.255009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.256578 (+  1569us) service_pool.cc:224] Handling call
0709 13:16:43.580840 (+324262us) ts_tablet_manager.cc:1151] Deleting tablet 85490b85caca40d19a25af108b1da46a
0709 13:16:43.580915 (+    75us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:16:44.160113 (+579198us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:44.342071 (+181958us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:44.420238 (+ 78167us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":322325,"thread_start_us":288451,"threads_started":1}
I20260709 13:16:44.579963  6667 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:16:44.587901  7813 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:16:44.710992  7775 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:44.751598  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.752602  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:44.753715  7806 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:44.759624  6591 raft_consensus.cc:2410] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:44.769115  7749 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:44.765017  6596 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:44.772771  7236 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:44.781913  7749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45428 (request call id 80) took 1146 ms. Trace:
I20260709 13:16:44.782445  7749 rpcz_store.cc:276] 0709 13:16:43.635745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.636143 (+   398us) service_pool.cc:224] Handling call
0709 13:16:43.659732 (+ 23589us) log.cc:844] Serialized 494 byte log entry
0709 13:16:44.018692 (+358960us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b18000687a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:16:44.139274 (+120582us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18000687a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b8fdab0b 00007f04b905fb63 00007f04b9087752 00007f04b907f9fa
0709 13:16:44.781823 (+642549us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189128,"raft.queue_time_us":306380,"raft.run_cpu_time_us":9143,"raft.run_wall_time_us":163880,"spinlock_wait_cycles":10560512,"thread_start_us":175610,"threads_started":1,"wal-append.queue_time_us":331424}
I20260709 13:16:44.793962  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:44.806938  7755 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:44.580011  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 85490b85caca40d19a25af108b1da46a (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
W20260709 13:16:44.809165  6405 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 229160us
I20260709 13:16:44.809904  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f4f0dad199224b8182ae036d97281b72 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:44.810127  7237 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:44.813334  7777 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:44.815354  6416 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 121) took 1043 ms. Trace:
I20260709 13:16:44.816726  6416 rpcz_store.cc:276] 0709 13:16:43.772303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.772681 (+   378us) service_pool.cc:224] Handling call
0709 13:16:43.851530 (+ 78849us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b5400002490. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5d0b2 00007f04beed6a89 00007f04bee6eda6 00007f04bee40088 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc
0709 13:16:44.815298 (+963768us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31158,"reported_tablets":2,"spinlock_wait_cycles":9870080}
I20260709 13:16:44.832299  6419 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 200) took 1304 ms. Trace:
I20260709 13:16:44.832365  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:44.838093  7780 tablet_replica.cc:333] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:44.849414  7780 raft_consensus.cc:2243] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:44.833160  6419 rpcz_store.cc:276] 0709 13:16:43.528211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.531230 (+  3019us) service_pool.cc:224] Handling call
0709 13:16:44.832238 (+1301008us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:43.825912 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:43.995361 (+169449us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:43.997711 (+  2350us) write_op.cc:435] Acquired schema lock
0709 13:16:43.997731 (+    20us) tablet.cc:662] Decoding operations
0709 13:16:44.033979 (+ 36248us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:44.034064 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:44.034107 (+    43us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:16:44.034645 (+   538us) tablet.cc:701] Row locks acquired
0709 13:16:44.034679 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:16:44.034860 (+   181us) write_op.cc:270] Start()
0709 13:16:44.035017 (+   157us) write_op.cc:276] Timestamp: P: 1783603004034802 usec, L: 0
0709 13:16:44.035045 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:16:44.035826 (+   781us) log.cc:844] Serialized 1746 byte log entry
0709 13:16:44.131389 (+ 95563us) op_driver.cc:464] REPLICATION: finished
0709 13:16:44.424108 (+292719us) write_op.cc:301] APPLY: starting
0709 13:16:44.424190 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:44.424325 (+   135us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:44.424456 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:44.424976 (+   520us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:44.425150 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:16:44.425194 (+    44us) write_op.cc:312] APPLY: finished
0709 13:16:44.452677 (+ 27483us) log.cc:844] Serialized 41 byte log entry
0709 13:16:44.452864 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:44.452954 (+    90us) write_op.cc:454] Released schema lock
0709 13:16:44.452992 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:44.453327 (+   335us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:44.453354 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":189647,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":291595,"apply.run_cpu_time_us":2341,"apply.run_wall_time_us":29203,"num_ops":4,"prepare.queue_time_us":7265,"prepare.run_cpu_time_us":4384,"prepare.run_wall_time_us":227792,"replication_time_us":87495,"thread_start_us":216408,"threads_started":1}]]}
I20260709 13:16:44.850195  7780 raft_consensus.cc:2272] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:44.853215  7780 ts_tablet_manager.cc:1916] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:44.892616  6666 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:44.929940  6415 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:44.942181  7775 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:44.968464  7714 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:44.970904  7233 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:44.998847  6515 raft_consensus.cc:2955] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:45.009158  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 7)
I20260709 13:16:45.004910  7780 ts_tablet_manager.cc:1929] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:45.015014  7780 log.cc:1199] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/53f3e5f2d81c4952b6ae972bdb3a18c1
I20260709 13:16:45.022261  6588 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:16:45.022734  6588 pending_rounds.cc:85] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 1
I20260709 13:16:45.023016  6588 pending_rounds.cc:107] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:45.023288  6588 raft_consensus.cc:2889] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:45.025364  7517 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:45.027293  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 53f3e5f2d81c4952b6ae972bdb3a18c1 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:45.033118  6417 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:45.034260  6666 raft_consensus.cc:2955] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:45.053943  6666 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:45.055334  7517 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:45.082684  6420 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:45.084946  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet f4f0dad199224b8182ae036d97281b72 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:45.096046  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "627436404a154d80a882f4900dff2948"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:45.096729  6588 raft_consensus.cc:606] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:45.116380  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:45.117142  6588 raft_consensus.cc:606] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:45.118219  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:45.122316  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85490b85caca40d19a25af108b1da46a"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:45.137516  6588 raft_consensus.cc:606] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:45.136677  7714 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:45.139077  7714 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.139899  7722 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:45.145731  7722 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.148989  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:45.149248  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:45.152715  7722 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:45.154886  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:45.155274  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:45.153508  7714 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:45.157909  6562 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:45.141429  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:45.158970  7807 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:45.159319  6588 raft_consensus.cc:606] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:16:45.163033  6517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:45.163641  6517 raft_consensus.cc:606] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:45.164464  7835 tablet_replica.cc:333] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:45.168012  7834 tablet_replica.cc:333] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:45.169607  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:45.170209  6668 raft_consensus.cc:606] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:45.175583  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:45.176203  6589 raft_consensus.cc:606] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:16:45.176555  6589 raft_consensus.cc:614] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:45.177531  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:45.179121  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:16:45.180797  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:45.194693  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:45.199046  7722 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:45.195080  7777 raft_consensus.cc:993] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:45.200276  7777 raft_consensus.cc:1081] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:45.200847  7835 raft_consensus.cc:2243] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:45.201715  7835 raft_consensus.cc:2272] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:45.206382  6515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:60540
I20260709 13:16:45.207047  6515 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.207392  6515 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.211207  7834 raft_consensus.cc:2243] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:45.212030  7834 raft_consensus.cc:2272] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:45.217908  7835 ts_tablet_manager.cc:1916] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:45.218091  7834 ts_tablet_manager.cc:1916] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:45.238607  6515 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.258942  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:45.259497  6588 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:45.259863  6588 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:45.261994  6515 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:45.260965  6588 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.263304  6588 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.256764  7835 ts_tablet_manager.cc:1929] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:16:45.268890  7517 raft_consensus.cc:993] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:45.269382  7517 raft_consensus.cc:1081] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:45.270395  7749 raft_consensus.cc:993] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:45.268878  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:45.270955  7749 raft_consensus.cc:1081] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:45.271222  6668 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.273468  7835 log.cc:1199] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/8fc61b2fa4ee43b7a7a91b2eb2ecd874
I20260709 13:16:45.273470  6594 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:45.274163  6594 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.274559  6594 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.276064  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 8fc61b2fa4ee43b7a7a91b2eb2ecd874 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:45.277647  7834 ts_tablet_manager.cc:1929] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:45.278108  7834 log.cc:1199] T f4f0dad199224b8182ae036d97281b72 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/f4f0dad199224b8182ae036d97281b72
I20260709 13:16:45.280966  6594 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.286145  6588 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:16:45.287351  6589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:16:45.287787  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet f4f0dad199224b8182ae036d97281b72 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:45.292311  6589 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.292804  6589 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.302397  6594 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:45.291574  6668 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:16:45.297351  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:45.339452  7703 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:45.346966  7722 raft_consensus.cc:993] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:45.346452  6589 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.347306  7703 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:45.347491  7722 raft_consensus.cc:1081] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:45.348235  7703 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.349284  7236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "627436404a154d80a882f4900dff2948"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:16:45.349828  7236 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.350160  7236 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.350752  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:45.351311  6515 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.351576  6515 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:45.352288  6515 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.352991  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:45.353333  6515 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.353574  6517 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.360245  6515 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:45.361930  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:45.362879  7777 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.363364  7777 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:45.364243  7777 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.357999  7236 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.367156  6589 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:45.375172  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:45.376940  6594 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.377564  7236 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:45.379529  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:45.383488  6594 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:45.385722  6594 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.389887  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:45.390725  6669 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.391130  6669 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:45.392099  6669 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.401346  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:45.401813  6669 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:45.401440  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.402228  6666 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.407146  6594 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.386664  6668 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.418084  6666 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:45.420166  6517 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:45.425050  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:45.426321  7722 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.426896  7722 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:45.427871  7722 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:45.428017  6420 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.429211  7807 raft_consensus.cc:993] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:45.436084  7807 raft_consensus.cc:1081] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:45.438179  7233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:16:45.438889  7233 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.439574  7233 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.434676  6594 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:45.441982  6669 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
W20260709 13:16:45.446735  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 43598.5us
I20260709 13:16:45.448544  6668 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:45.449780  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:45.450634  7751 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.456432  6415 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.473690  7751 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:45.500213  7751 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.501783  7233 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.505630  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:45.506163  6515 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.512720  6515 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:45.514771  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:45.516340  7233 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:45.532547  6415 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.537021  7843 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.543939  7843 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:45.549037  7843 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.553355  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:45.554101  6589 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.554503  6589 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:45.557430  6515 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:45.555439  6589 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.559666  7844 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:45.572358  6589 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.580154  6589 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:45.580754  6415 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:45.600704  6415 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:45.647213  7722 raft_consensus.cc:993] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:45.647732  7722 raft_consensus.cc:1081] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:45.649276  7233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85490b85caca40d19a25af108b1da46a"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:16:45.649974  7233 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.650406  7233 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.658638  7233 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.660851  7233 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:45.661793  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:45.661887  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:45.662644  6515 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.662748  6589 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.663162  6589 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:45.664044  6589 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.665516  6589 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.669488  6515 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:45.674780  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:45.676010  6589 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:45.677649  7843 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.679188  7843 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:45.684585  7843 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:45.695154  6415 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:45.801192  6669 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:45.802501  7703 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:45.836203  6589 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:45.837728  7654 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:45.897439  6669 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:45.898984  7844 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:45.907699  6668 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:45.909507  7844 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:45.925833  6517 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:45.926178  6521 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:45.929406  7845 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:45.929428  7807 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:16:45.948638  6588 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:45.952721  7843 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:45.979709  6521 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:45.986532  7843 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:46.003935  6666 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:46.006567  6588 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:46.007167  7846 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:46.015163  7751 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:46.183089  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:46.228717  6521 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:46.232594  6515 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:46.234462  7751 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:46.253206  6588 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:46.254774  7751 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:46.271842  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:46.316191  6487 consensus_peers.cc:597] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bab5f01bd4774a168d3c01940a615cce. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:46.361207  6588 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:46.374915  7517 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.394416  6669 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:46.400558  7779 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:46.406384  7703 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.413080  6589 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:46.429636  6588 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:46.438318  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:46.439085  6668 raft_consensus.cc:606] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:46.441700  6669 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:46.448381  6417 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:46.452963  6562 consensus_peers.cc:597] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35df31b6c7b2475cac55c1e114ba24dc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:46.453819  6668 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:46.456882  7777 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.508059  6515 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:46.519871  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56ba1578b97b4faa966168899769e067"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:46.530211  6589 raft_consensus.cc:606] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:46.552560  7845 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:46.584757  7880 raft_consensus.cc:993] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:46.586470  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:46.590374  6594 raft_consensus.cc:606] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:46.621704  7722 raft_consensus.cc:1081] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:46.626343  7237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56ba1578b97b4faa966168899769e067"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:16:46.627213  7237 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:46.627696  7237 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.629465  7845 raft_consensus.cc:993] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:46.630059  7845 raft_consensus.cc:1081] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
W20260709 13:16:46.632524  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52592.1us
I20260709 13:16:46.633102  7233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:16:46.633827  7233 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:46.634233  7233 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.638520  7785 raft_consensus.cc:993] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:46.639307  7785 raft_consensus.cc:1081] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:46.647634  7236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46666
I20260709 13:16:46.648218  7236 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:46.648667  7236 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.584617  7807 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:46.660404  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 95760.6us
W20260709 13:16:46.671064  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 82068.3us
I20260709 13:16:46.675216  7236 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.679253  6667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:46.679963  6667 raft_consensus.cc:606] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:46.685899  6667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:46.686810  6667 raft_consensus.cc:606] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:46.688972  7237 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.691344  7236 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:46.692337  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:46.691689  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:46.693264  6517 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.693784  6521 raft_consensus.cc:606] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:46.696923  7237 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:46.697014  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:46.701268  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:46.701958  6669 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:46.702291  6669 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:46.703598  6669 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.704862  6669 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.708812  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:46.709519  6588 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.709697  6663 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:46.718397  7883 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.057s	user 0.003s	sys 0.000s
I20260709 13:16:46.720108  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:46.722965  6588 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:46.725502  6594 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:46.724539  6669 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:46.725773  6594 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:46.727572  6594 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.728435  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:46.728919  6594 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.731451  6517 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:46.732226  6521 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:46.732961  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:46.690421  7233 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.733937  7875 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:46.737202  7875 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:46.738350  7875 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.739511  6589 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:46.735620  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:46.743652  7751 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:46.744184  7751 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:46.745038  7751 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.749593  6594 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:46.752126  7233 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:46.748430  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:46.753582  6517 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.776371  6515 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:46.782083  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:46.782802  6594 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:46.783108  6594 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:46.783929  6594 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.785373  6594 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.788137  7878 raft_consensus.cc:993] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:46.788650  7878 raft_consensus.cc:1081] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:46.805419  6594 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:46.806484  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:46.807461  7843 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:46.808735  7843 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:46.809777  7843 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:46.811479  6517 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:46.835279  6667 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.835781  6589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:46.836604  6589 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:46.837090  6589 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.847935  7517 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:46.853870  6487 consensus_peers.cc:597] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd28ad0fdaad48d586176f4a8e3f1aa3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:46.869292  6415 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:46.888401  6517 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:46.901041  7891 ts_tablet_manager.cc:933] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:46.902088  7840 raft_consensus.cc:993] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:46.902726  7889 raft_consensus.cc:1081] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:46.906605  6596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:46.907164  6596 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:46.907480  6596 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.896585  6417 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:46.919286  6417 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:46.920993  6589 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:46.924391  6594 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.933861  6589 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:46.935050  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:46.935706  6668 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:46.936002  6668 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:46.936908  6668 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:46.938235  6668 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.939975  6417 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:46.945142  6596 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:46.948318  7517 raft_consensus.cc:993] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:46.948969  7654 raft_consensus.cc:1081] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:46.927250  7891 tablet_copy_client.cc:323] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:46.950372  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:46.950994  6663 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:46.948489  6596 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:46.946347  6668 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:46.951355  6663 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:46.954811  6667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:60486
I20260709 13:16:46.955236  7517 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:46.955413  6667 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:46.955309  6663 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:46.956583  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet bab5f01bd4774a168d3c01940a615cce from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:46.957177  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet bab5f01bd4774a168d3c01940a615cce from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-bab5f01bd4774a168d3c01940a615cce
I20260709 13:16:46.953402  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:46.958868  7882 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:46.958981  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:46.959398  7882 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:46.959565  6515 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.960342  7882 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:46.955938  6667 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.964097  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:46.956597  6663 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.964664  6521 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.975219  6532 tablet_copy_source_session.cc:215] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:46.991869  6663 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:46.997570  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:46.999703  7891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab5f01bd4774a168d3c01940a615cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:46.978441  6667 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:47.004854  7807 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.005370  6515 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:47.005409  7807 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:47.010169  6521 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:47.011813  6667 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:47.012619  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:47.012332  7807 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:47.013173  6596 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.023546  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:47.024291  6516 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:47.024652  6516 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:47.025511  6516 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:47.038559  6596 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:47.041894  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:47.052034  7889 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.052670  7889 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:47.053615  7889 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:47.044238  6516 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.090855  7890 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:47.093829  6667 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:47.163753  6667 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:47.165313  7751 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:47.182552  7898 ts_tablet_manager.cc:933] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:47.185606  7898 tablet_copy_client.cc:323] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:47.197464  7891 tablet_copy_client.cc:806] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:47.198123  7891 tablet_copy_client.cc:670] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:47.214166  6516 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:47.259341  6594 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:47.264917  6516 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:47.279166  7843 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:47.293620  6417 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:47.306283  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 133775us
I20260709 13:16:47.329866  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 35df31b6c7b2475cac55c1e114ba24dc from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:47.330461  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 35df31b6c7b2475cac55c1e114ba24dc from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-35df31b6c7b2475cac55c1e114ba24dc
I20260709 13:16:47.331580  6589 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:47.377326  6607 tablet_copy_source_session.cc:215] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:47.586786  6594 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:47.611330  6521 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:47.667654  6516 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:47.750075  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 373685us
W20260709 13:16:47.861400  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 245135us
W20260709 13:16:47.878105  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.884426  6516 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:47.940085  7903 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:47.949568  6591 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:47.963340  7807 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:47.869477  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:47.971067  7913 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:48.058009  6671 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:48.108174  6668 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.118319  6671 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.121378  7517 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:48.164001  6592 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.166656  6515 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.172291  7890 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:48.174232  7884 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.176853  6671 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:48.180634  7884 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.190198  7928 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:16:48.191071  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 851734us
I20260709 13:16:48.191094  6589 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:48.220170  6420 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.230880  6420 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.222082  7785 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:16:48.271638  7929 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:48.272317  7929 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.291049  7929 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.281093  7884 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:48.308466  7807 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.308970  7807 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.303539  7884 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:48.316270  7807 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:48.316500  7884 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.322592  6417 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:48.331637  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.336009  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.336750  6515 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:48.344523  7877 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.302728  7929 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:48.362660  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.372299  7929 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.380714  7929 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.395467  7934 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:16:48.396090  7934 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.398313  7934 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.398762  7937 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:48.398844  7934 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:48.399381  7937 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.399890  7934 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:16:48.420133  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800050ac0 after lost signal to thread 5413
W20260709 13:16:48.421221  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800170640 after lost signal to thread 6454
W20260709 13:16:48.422428  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800355c80 after lost signal to thread 6546
I20260709 13:16:48.428150  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:48.430639  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:16:48.433182  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002caf00 after lost signal to thread 6621
W20260709 13:16:48.434361  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800320540 after lost signal to thread 6695
W20260709 13:16:48.435488  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec960 after lost signal to thread 7261
W20260709 13:16:48.436069  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080033c2e0 after lost signal to thread 7875
W20260709 13:16:48.436537  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080025b460 after lost signal to thread 7892
I20260709 13:16:48.429759  7929 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:16:48.440521  7929 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.442288  7937 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:48.447261  7891 tablet_copy_client.cc:538] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:48.432338  7807 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.455842  7884 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:48.456286  7884 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.456621  7884 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:48.456650  7934 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.442564  7929 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.459092  7938 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:16:48.470820  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.473975  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.475088  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.465174  7934 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:48.471426  7938 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.486016  7779 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:16:48.486739  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.486649  7779 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.489542  7947 ts_tablet_manager.cc:933] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:48.490453  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.489809  7779 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:48.488363  7934 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.515648  7876 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:16:48.523924  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:48.516243  7876 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.553529  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:48.553929  7876 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:16:48.554761  6487 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23ba4e226865438d85ee0c061786f3db. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:48.555845  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:48.556108  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:48.557444  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:48.562554  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.569213  7935 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:16:48.571769  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.595217  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.615278  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.606606  7898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35df31b6c7b2475cac55c1e114ba24dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:48.616799  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:48.621043  7938 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:48.627396  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:48.630128  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.634819  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:48.635185  7938 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.666303  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:48.670287  6522 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:48.655025  7947 tablet_copy_client.cc:323] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:48.711181  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:48.711961  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet fd28ad0fdaad48d586176f4a8e3f1aa3 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:48.712538  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet fd28ad0fdaad48d586176f4a8e3f1aa3 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-fd28ad0fdaad48d586176f4a8e3f1aa3
I20260709 13:16:48.672789  7934 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.665040  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:16:48.718312  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 45020.5us
W20260709 13:16:48.724025  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 89330.6us
I20260709 13:16:48.724957  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:48.736963  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:48.730233  7949 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.745935  6532 tablet_copy_source_session.cc:215] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:48.660876  7944 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 13:16:48.748795  7777 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.749699  7777 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.710254  7889 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:16:48.750958  7889 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.730362  7928 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:16:48.760838  7891 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:48.762269  7930 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.771342  7949 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.777978  6522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:48.778573  6522 raft_consensus.cc:606] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:48.776003  7884 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.782572  6589 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:48.787138  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:48.785342  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:48.754570  7889 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:48.794973  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.793223  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:48.798363  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:48.798586  6671 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:48.783238  7884 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:48.800770  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:48.804561  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:48.804705  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.805037  7938 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.800239  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:48.807164  6522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:48.807716  6522 raft_consensus.cc:606] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:48.798009  7949 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.795317  7930 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:48.817296  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:48.818002  7957 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:48.788837  7777 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:48.838963  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:48.845644  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:48.846326  6590 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:48.849031  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:48.851253  7880 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:48.858788  6665 raft_consensus.cc:606] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:48.870229  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:48.870875  6518 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:48.878307  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:48.879058  6665 raft_consensus.cc:606] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:48.881469  7891 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:48.883481  7891 tablet_bootstrap.cc:492] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:48.884331  7891 ts_tablet_manager.cc:1403] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.124s	user 0.028s	sys 0.017s
I20260709 13:16:48.885514  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:48.887780  6665 raft_consensus.cc:606] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:48.888764  7841 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:16:48.889158  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:48.889479  7841 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:48.891237  7908 raft_consensus.cc:993] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:48.892088  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:48.892037  7916 raft_consensus.cc:1081] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:48.891711  7898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46680 (request call id 301) took 1795 ms. Trace:
I20260709 13:16:48.893118  7841 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:48.894821  7898 rpcz_store.cc:276] 0709 13:16:47.091214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.091330 (+   116us) service_pool.cc:224] Handling call
0709 13:16:47.182918 (+ 91588us) ts_tablet_manager.cc:934] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
0709 13:16:48.492800 (+1309882us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:48.796479 (+303679us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:48.800937 (+  4458us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:48.828832 (+ 27895us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:48.886097 (+ 57265us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:48.886475 (+   378us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:48.886546 (+    71us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":195,"tablet-copy.queue_time_us":87100,"thread_start_us":84428,"threads_started":2}
I20260709 13:16:48.898551  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:48.898033  7898 tablet_copy_client.cc:806] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:48.899963  7898 tablet_copy_client.cc:670] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:48.904976  7891 raft_consensus.cc:359] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:48.906055  7891 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:48.906839  7891 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:48.910913  7891 ts_tablet_manager.cc:1434] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 13:16:48.917543  7913 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:16:48.918150  7913 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:48.920392  7913 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:48.921325  7913 raft_consensus.cc:993] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:48.922835  7517 raft_consensus.cc:1081] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:48.929131  6667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:60486
W20260709 13:16:48.930358  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 126952us
I20260709 13:16:48.930733  6667 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.931613  6667 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.935879  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-bab5f01bd4774a168d3c01940a615cce received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:48.936398  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-bab5f01bd4774a168d3c01940a615cce on tablet bab5f01bd4774a168d3c01940a615cce with peer 64b95375974b48d18c596a23c9cac75c
W20260709 13:16:48.936319  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47569.8us
I20260709 13:16:48.937158  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:48.942090  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:16:48.943642  6667 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:48.947105  6667 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:48.947646  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:48.948310  7235 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.959442  6592 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:48.929500  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:48.984799  6518 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:16:48.994902  7234 raft_consensus.cc:1217] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:49.035171  6517 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:49.003723  7964 ts_tablet_manager.cc:933] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:49.017009  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:49.039399  7910 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.027721  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:49.027920  6596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:49.041183  6596 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:49.041566  6596 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.049810  6596 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:49.050392  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:49.051102  6516 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:49.003393  6666 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:49.051437  6516 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:49.052814  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:16:49.052433  6516 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.053762  7930 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:49.054205  7908 raft_consensus.cc:993] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:49.053958  7910 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:49.054741  7957 raft_consensus.cc:1081] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:49.056900  7947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd28ad0fdaad48d586176f4a8e3f1aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:49.057781  7234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46666
I20260709 13:16:49.058377  7234 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:49.058833  7234 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.054782  6596 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:49.068358  7235 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:49.068008  6516 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.070933  7785 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:49.075816  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:49.071854  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:16:49.081558  7234 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.086644  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:49.091223  7234 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:49.086078  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:49.099313  6522 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.100674  7232 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:49.078469  7841 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:49.083670  7722 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:49.112119  7785 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:16:49.083362  7898 tablet_copy_client.cc:538] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:49.112450  7722 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:49.115011  7722 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.119208  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:49.112910  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:49.125088  6668 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:49.125547  6668 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:49.122864  7959 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:16:49.129856  7930 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:49.130240  7930 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:49.126526  7785 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.166497  6522 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:49.169219  7964 tablet_copy_client.cc:323] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:49.175865  6417 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 217) took 1041 ms. Trace:
I20260709 13:16:49.136534  7957 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:16:49.179098  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 107254us
I20260709 13:16:49.182488  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:49.187773  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 23ba4e226865438d85ee0c061786f3db from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:49.188372  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 23ba4e226865438d85ee0c061786f3db from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-23ba4e226865438d85ee0c061786f3db
I20260709 13:16:49.131160  7930 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.191491  7930 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.191598  6415 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:49.202934  7957 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:49.204136  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:49.204797  6595 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.213630  6563 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:49.220920  7807 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:49.234227  7807 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:49.235204  7807 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:49.242493  6668 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.243906  6668 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.176273  6417 rpcz_store.cc:276] 0709 13:16:48.134450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.134724 (+   274us) service_pool.cc:224] Handling call
0709 13:16:49.175803 (+1041079us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:48.483277 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:48.555223 (+ 71946us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:48.555254 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:16:48.555271 (+    17us) tablet.cc:662] Decoding operations
0709 13:16:48.555603 (+   332us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:48.555645 (+    42us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:48.555673 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:48.555845 (+   172us) tablet.cc:701] Row locks acquired
0709 13:16:48.555863 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:16:48.555983 (+   120us) write_op.cc:270] Start()
0709 13:16:48.556081 (+    98us) write_op.cc:276] Timestamp: P: 1783603008555939 usec, L: 0
0709 13:16:48.556102 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:16:48.556764 (+   662us) log.cc:844] Serialized 595 byte log entry
0709 13:16:49.111885 (+555121us) op_driver.cc:464] REPLICATION: finished
0709 13:16:49.129877 (+ 17992us) write_op.cc:301] APPLY: starting
0709 13:16:49.129961 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:49.130072 (+   111us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:49.130165 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:49.130400 (+   235us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:49.130549 (+   149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:49.130594 (+    45us) write_op.cc:312] APPLY: finished
0709 13:16:49.130812 (+   218us) log.cc:844] Serialized 23 byte log entry
0709 13:16:49.130971 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:49.131893 (+   922us) write_op.cc:454] Released schema lock
0709 13:16:49.132780 (+   887us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:49.170946 (+ 38166us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:49.171003 (+    57us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41920,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":17738,"apply.run_cpu_time_us":1668,"apply.run_wall_time_us":14929,"mutex_wait_us":720,"num_ops":1,"prepare.queue_time_us":145375,"prepare.run_cpu_time_us":2727,"prepare.run_wall_time_us":79108,"replication_time_us":553558,"thread_start_us":31900,"threads_started":1}]]}
I20260709 13:16:49.204211  7957 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.252988  6488 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:16:49.258699  7910 raft_consensus.cc:993] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:49.261782  6668 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:49.258369  7785 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.271226  6595 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:49.255487  7779 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.276731  6671 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:49.277141  6671 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:49.277992  7914 raft_consensus.cc:1081] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:49.277819  6671 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:49.279404  6671 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.279906  7235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46684
I20260709 13:16:49.281966  7235 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:49.282414  7235 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.288853  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:49.293725  7949 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:49.300904  7949 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:49.302048  7947 tablet_copy_client.cc:806] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:49.301710  7949 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.312918  6593 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:49.309232  7947 tablet_copy_client.cc:670] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:49.327914  6532 tablet_copy_source_session.cc:215] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:49.333794  7877 raft_consensus.cc:993] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:49.335623  7898 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:49.396775  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:49.397281  7970 raft_consensus.cc:1081] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:49.399983  6671 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:49.420620  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:49.390398  7785 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:49.430675  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:49.398897  6516 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:49.438089  7908 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.438481  7235 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.442240  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:49.442874  6516 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:49.443176  6516 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:49.443941  6516 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.445005  6516 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:49.452564  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.457865  7964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23ba4e226865438d85ee0c061786f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:49.460819  6667 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:49.468650  6516 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:49.471632  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:49.473554  7929 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:49.474047  7929 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:49.474925  7929 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.481527  7235 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:16:49.192672  7935 consensus_queue.cc:1175] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ceee54e49d041be9f7f48a2052709c5" responder_term: 3 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: [2.5-2.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:49.495632  7947 tablet_copy_client.cc:538] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:49.498066  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:49.498739  6667 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.520578  7517 raft_consensus.cc:1064] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
W20260709 13:16:49.544687  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 134653us
W20260709 13:16:49.545135  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 118361us
I20260709 13:16:49.516637  7935 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:16:49.547681  7517 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.547087  7947 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:49.557605  6595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:49.558243  6595 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:49.558677  6595 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.555155  7949 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:49.567313  6595 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.562561  7230 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:49.590571  6595 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:49.592710  6667 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:49.603894  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:49.605597  7232 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.583945  7949 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:49.619088  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:49.621894  6666 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:49.622221  6666 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:49.623062  6666 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.623935  6666 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.627658  7949 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.629091  7949 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.630061  6666 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:49.634503  7964 tablet_copy_client.cc:806] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:49.635186  7964 tablet_copy_client.cc:670] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:49.639488  6415 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 221) took 1187 ms. Trace:
I20260709 13:16:49.642449  6415 rpcz_store.cc:276] 0709 13:16:48.451600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.451773 (+   173us) service_pool.cc:224] Handling call
0709 13:16:49.639404 (+1187631us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:49.552198 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:49.564746 (+ 12548us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:49.564781 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:16:49.564801 (+    20us) tablet.cc:662] Decoding operations
0709 13:16:49.565148 (+   347us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:49.565231 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:49.565265 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:49.565444 (+   179us) tablet.cc:701] Row locks acquired
0709 13:16:49.565462 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:16:49.565591 (+   129us) write_op.cc:270] Start()
0709 13:16:49.565698 (+   107us) write_op.cc:276] Timestamp: P: 1783603009565549 usec, L: 0
0709 13:16:49.565719 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:16:49.567743 (+  2024us) log.cc:844] Serialized 595 byte log entry
0709 13:16:49.571003 (+  3260us) op_driver.cc:464] REPLICATION: finished
0709 13:16:49.571564 (+   561us) write_op.cc:301] APPLY: starting
0709 13:16:49.571647 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:49.571761 (+   114us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:49.573195 (+  1434us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:49.575294 (+  2099us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:49.575453 (+   159us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:49.575506 (+    53us) write_op.cc:312] APPLY: finished
0709 13:16:49.575718 (+   212us) log.cc:844] Serialized 23 byte log entry
0709 13:16:49.575930 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:49.576020 (+    90us) write_op.cc:454] Released schema lock
0709 13:16:49.576061 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:49.576412 (+   351us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:49.576445 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14358,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":277,"apply.run_cpu_time_us":2052,"apply.run_wall_time_us":4811,"num_ops":1,"prepare.queue_time_us":328608,"prepare.run_cpu_time_us":3128,"prepare.run_wall_time_us":23537,"replication_time_us":5162,"thread_start_us":212822,"threads_started":1}]]}
I20260709 13:16:49.656230  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:49.668313  7969 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:49.670253  7969 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:49.638552  7964 tablet_copy_client.cc:538] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:49.672652  7969 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:49.686553  7232 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:49.695716  6668 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:49.697922  6669 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:49.703696  7949 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:49.707005  7890 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:49.709444  7964 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:49.711050  6668 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.713829  7904 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:49.728888  6595 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:49.732775  7517 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:49.737951  7230 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:49.738854  7898 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:49.736838  6418 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:49.739782  7898 tablet_bootstrap.cc:492] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:49.745040  6593 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:49.741163  6418 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:49.748713  7779 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:16:49.764991  7898 ts_tablet_manager.cc:1403] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.430s	user 0.029s	sys 0.008s
I20260709 13:16:49.751747  7949 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:16:49.840225  6517 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.813982  7898 raft_consensus.cc:359] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:49.841596  7898 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:49.842370  7898 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:49.867357  7236 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:16:49.867772  7236 pending_rounds.cc:85] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 4
I20260709 13:16:49.867990  7236 pending_rounds.cc:107] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:16:49.868263  7236 raft_consensus.cc:2889] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.896755  6420 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 3, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:49.902748  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:49.956285  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 54175.8us
I20260709 13:16:49.881649  7880 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:16:50.001595  7898 ts_tablet_manager.cc:1434] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.213s	user 0.010s	sys 0.000s
I20260709 13:16:49.888679  7902 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
W20260709 13:16:50.006075  7947 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.074s	user 0.001s	sys 0.002s
I20260709 13:16:49.963599  6516 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:50.013442  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-35df31b6c7b2475cac55c1e114ba24dc received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:50.022954  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-35df31b6c7b2475cac55c1e114ba24dc on tablet 35df31b6c7b2475cac55c1e114ba24dc with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:50.053648  6518 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:50.033619  7908 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:16:50.095073  6669 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:50.108361  7914 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:50.097450  7947 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:16:50.132575  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 73141.5us
I20260709 13:16:50.099304  7959 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:16:50.151748  7234 raft_consensus.cc:1217] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:50.154770  7947 tablet_bootstrap.cc:492] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:50.155696  7947 ts_tablet_manager.cc:1403] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.625s	user 0.040s	sys 0.046s
I20260709 13:16:50.162883  7944 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:16:50.196069  6517 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:50.208163  7234 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:50.208961  7235 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:16:50.216890  7989 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.199s	user 0.002s	sys 0.000s
W20260709 13:16:50.247822  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 151583us
W20260709 13:16:50.266788  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 168191us
W20260709 13:16:50.304700  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 61567us
I20260709 13:16:50.255369  7908 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:16:50.250986  7902 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:16:50.435107  7947 raft_consensus.cc:359] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:50.436038  7947 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
W20260709 13:16:50.438014  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 100547us
W20260709 13:16:50.448352  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 145057us
I20260709 13:16:50.384859  7230 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:50.469195  7947 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:50.488863  7981 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:50.498536  6663 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:50.488819  7947 ts_tablet_manager.cc:1434] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.333s	user 0.002s	sys 0.007s
I20260709 13:16:50.521849  6591 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:50.543237  7981 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:16:50.550743  6665 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:50.555825  7964 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:50.555570  6417 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:50.556696  7964 tablet_bootstrap.cc:492] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:50.557399  7964 ts_tablet_manager.cc:1403] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.848s	user 0.025s	sys 0.042s
I20260709 13:16:50.560194  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-fd28ad0fdaad48d586176f4a8e3f1aa3 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:50.560662  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-fd28ad0fdaad48d586176f4a8e3f1aa3 on tablet fd28ad0fdaad48d586176f4a8e3f1aa3 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:50.620798  6418 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:50.624460  6418 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:50.652938  8014 raft_consensus.cc:1064] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:50.655596  8014 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:50.730413  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:50.736629  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 52278.9us
I20260709 13:16:50.885088  7230 raft_consensus.cc:1217] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
W20260709 13:16:50.907408  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.962181  7234 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:51.004611  7967 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:16:50.994391  7996 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.994484  7964 raft_consensus.cc:359] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:51.016903  7964 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
W20260709 13:16:51.041599  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 221928us
I20260709 13:16:51.018523  7996 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:51.047595  7964 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:51.047536  7935 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.069424  7935 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:16:51.081805  8034 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:16:51.082988  7935 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:51.086385  7964 ts_tablet_manager.cc:1434] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.529s	user 0.012s	sys 0.000s
W20260709 13:16:51.086779  7964 ts_tablet_manager.cc:1467] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Tablet startup took 529ms
I20260709 13:16:51.092446  7882 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:16:51.111367  7882 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:51.126899  7882 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:51.127904  7996 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:51.128674  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:51.129390  7232 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
W20260709 13:16:51.087105  7964 ts_tablet_manager.cc:1469] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Trace:
0709 13:16:48.993713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.993871 (+   158us) service_pool.cc:224] Handling call
0709 13:16:49.036094 (+ 42223us) ts_tablet_manager.cc:934] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
0709 13:16:49.409424 (+373330us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:49.573340 (+163916us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:49.588571 (+ 15231us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:49.633573 (+ 45002us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:49.633890 (+   317us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:49.634212 (+   322us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:49.634278 (+    66us) inbound_call.cc:177] Queueing success response
0709 13:16:49.709037 (+ 74759us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:16:50.566859 (+857822us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:16:50.859508 (+292649us) tablet_replica.cc:261] Starting instrumentation
0709 13:16:50.868732 (+  9224us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":36074,"mutex_wait_us":1075,"raft.queue_time_us":2890,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":7000,"tablet-copy.queue_time_us":9152,"tablet-download-pool.queue_time_us":115,"tablet-download-pool.run_cpu_time_us":1016,"tablet-download-pool.run_wall_time_us":2702,"thread_start_us":106165,"threads_started":2}
I20260709 13:16:51.136662  6515 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:51.132133  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:51.139672  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-23ba4e226865438d85ee0c061786f3db received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:51.139835  7882 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:51.140089  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-23ba4e226865438d85ee0c061786f3db on tablet 23ba4e226865438d85ee0c061786f3db with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:51.140353  7882 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:16:51.140784  7882 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.144423  7880 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:51.164127  7882 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:51.168663  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:51.176007  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:16:51.176698  7235 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:16:51.180966  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:16:51.192284  6593 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:16:51.197775  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:51.199139  6668 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:16:51.208643  6666 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:51.208643  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:16:51.215149  6671 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:51.227758  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:51.228857  8038 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:51.229233  8038 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.230645  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:51.231626  8038 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.243937  7902 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:16:51.238022  7949 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:51.261072  7949 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.261554  7949 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.294862  6415 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:51.301612  6417 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 222) took 1551 ms. Trace:
I20260709 13:16:51.302047  6417 rpcz_store.cc:276] 0709 13:16:49.750141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.750265 (+   124us) service_pool.cc:224] Handling call
0709 13:16:51.301551 (+1551286us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:50.589284 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:50.722948 (+133664us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:50.725785 (+  2837us) write_op.cc:435] Acquired schema lock
0709 13:16:50.739486 (+ 13701us) tablet.cc:662] Decoding operations
0709 13:16:50.902719 (+163233us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:50.902810 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:50.902847 (+    37us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:50.903052 (+   205us) tablet.cc:701] Row locks acquired
0709 13:16:50.903071 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:16:50.903219 (+   148us) write_op.cc:270] Start()
0709 13:16:50.903333 (+   114us) write_op.cc:276] Timestamp: P: 1783603010903178 usec, L: 0
0709 13:16:50.903354 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:16:50.904013 (+   659us) log.cc:844] Serialized 533 byte log entry
0709 13:16:51.138890 (+234877us) op_driver.cc:464] REPLICATION: finished
0709 13:16:51.277928 (+139038us) write_op.cc:301] APPLY: starting
0709 13:16:51.278007 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:51.278098 (+    91us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:51.278229 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:51.278478 (+   249us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:51.278655 (+   177us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:51.278700 (+    45us) write_op.cc:312] APPLY: finished
0709 13:16:51.278885 (+   185us) log.cc:844] Serialized 23 byte log entry
0709 13:16:51.279044 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:51.279117 (+    73us) write_op.cc:454] Released schema lock
0709 13:16:51.279152 (+    35us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:51.279468 (+   316us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:51.279495 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":83200,"child_traces":[["op",{"apply.queue_time_us":138810,"apply.run_cpu_time_us":1531,"apply.run_wall_time_us":1531,"num_ops":1,"prepare.queue_time_us":21403,"prepare.run_cpu_time_us":3494,"prepare.run_wall_time_us":314978,"replication_time_us":235374,"thread_start_us":111711,"threads_started":2}]]}
I20260709 13:16:51.316437  7949 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:51.345549  7949 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:51.355127  8038 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:51.357432  8038 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:51.365311  7969 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:51.368036  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:51.369127  6667 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:16:51.381072  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:51.381951  7234 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:16:51.403680  6515 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:51.416900  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:51.417537  6592 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.411127  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:51.433650  6665 raft_consensus.cc:2368] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4.
W20260709 13:16:51.538529  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 104838us
I20260709 13:16:51.539470  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:51.567760  6561 leader_election.cc:400] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4.
I20260709 13:16:51.568198  6561 leader_election.cc:403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:51.593668  6592 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:16:51.606519  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:51.616510  8039 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:51.594914  7807 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4.
I20260709 13:16:51.637077  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:16:51.659490  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 69885.9us
W20260709 13:16:51.686105  8049 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:16:51.706269  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47162.7us
I20260709 13:16:51.645187  7935 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:16:51.721428  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63492us
I20260709 13:16:51.723616  7935 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:51.769304  8003 raft_consensus.cc:1064] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
W20260709 13:16:51.783555  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76426us
I20260709 13:16:51.786181  7234 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:51.814941  7882 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:51.818325  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:51.789690  7935 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:16:51.829183  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.836439  8003 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:16:51.844882  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 100502us
I20260709 13:16:51.868355  7232 raft_consensus.cc:1217] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:51.880084  6596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:51.880764  6596 raft_consensus.cc:606] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
W20260709 13:16:51.916502  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 294324us
I20260709 13:16:51.932909  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:51.934047  6595 raft_consensus.cc:606] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:16:51.952303  6516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:51.952934  6516 raft_consensus.cc:606] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:51.970677  6664 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:51.977435  6420 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 221) took 1322 ms. Trace:
I20260709 13:16:51.977847  6420 rpcz_store.cc:276] 0709 13:16:50.654801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.660062 (+  5261us) service_pool.cc:224] Handling call
0709 13:16:51.977371 (+1317309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50730,"reported_tablets":4}
I20260709 13:16:51.989068  6595 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:51.992408  8003 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:51.995128  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:51.995357  6663 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:51.995721  6595 raft_consensus.cc:606] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:51.996621  8003 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:51.999020  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:51.999560  6595 raft_consensus.cc:606] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:52.004316  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:52.008842  6417 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:52.013536  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45433.3us
I20260709 13:16:52.015527  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:52.037427  7230 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:52.041126  7914 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:52.048465  7914 raft_consensus.cc:993] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:52.049024  7914 raft_consensus.cc:1081] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:52.049743  7777 raft_consensus.cc:993] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:52.050302  8044 raft_consensus.cc:1081] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:52.051370  6596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:16:52.051914  6596 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:52.052244  6596 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.071554  8003 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.073518  6595 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.082332  6596 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.086036  6596 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:52.086686  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:52.087248  6517 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:52.087525  6517 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:52.087914  6664 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45444
I20260709 13:16:52.088430  6664 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:52.088806  6664 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.089831  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:52.090359  6668 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.088238  6517 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.092509  6517 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.099439  6517 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:16:52.105756  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:52.113507  6664 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.115957  6668 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:16:52.127911  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:52.128499  6516 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.128193  7237 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.131343  6664 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:52.135548  7807 raft_consensus.cc:993] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:52.137163  7807 raft_consensus.cc:1081] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:52.137835  6516 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:16:52.132817  6418 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.144340  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:52.144711  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:52.148243  6516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:60540
I20260709 13:16:52.139773  7981 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:52.148773  6516 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:52.150202  8039 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:52.150815  8039 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:52.152469  6516 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.151679  8039 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.160813  7981 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:52.169782  7882 raft_consensus.cc:993] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:52.170658  6516 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.172399  7882 raft_consensus.cc:1081] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:52.172807  8053 raft_consensus.cc:1064] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:52.174057  7230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:16:52.174422  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:52.173806  6663 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.174993  6664 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.175372  8053 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.174611  7230 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:52.177315  7230 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.161847  7981 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.180771  6516 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:52.181335  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:52.186688  6596 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.187062  6596 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:52.188680  6595 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:52.189038  6595 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:52.193648  6664 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:52.189881  6595 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.194634  6486 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:52.197252  6595 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.201416  6596 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.202435  6596 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.208739  6596 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:52.209347  6595 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:16:52.213497  7913 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:52.214037  7904 raft_consensus.cc:993] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:52.214504  7904 raft_consensus.cc:1081] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:52.200081  6420 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.216773  7913 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:52.233311  7913 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.251637  6667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45444
I20260709 13:16:52.253610  6667 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:52.254209  6667 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.263864  6667 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.268996  6667 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:52.271423  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:52.273316  6515 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.275434  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:52.276062  6595 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.276360  6595 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:52.277114  6595 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.278303  6595 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.279979  6515 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:52.281740  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:52.283768  7997 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:52.286326  6596 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.302045  6595 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:52.284237  7997 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:52.304507  7237 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:52.309815  7233 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:52.316608  6596 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:52.310601  6663 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:52.324872  6667 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:52.323175  8044 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:16:52.347347  7882 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:52.321977  8052 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:16:52.358705  7913 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:52.361380  8003 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:52.363858  7904 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.370860  7997 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:52.371409  6663 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.376827  7237 raft_consensus.cc:2955] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.397409  6418 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:52.399791  6420 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.386401  6516 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.407240  7230 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.413064  6415 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 3 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.423628  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:52.424255  6595 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.424551  6595 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:52.426491  6595 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.427601  6595 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.437130  7230 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:52.440403  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 35df31b6c7b2475cac55c1e114ba24dc with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:52.443231  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:52.444489  6664 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.472770  6596 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:52.473248  6667 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:52.480686  8058 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:52.482685  8058 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:52.499567  8073 tablet_replica.cc:333] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: stopping tablet replica
W20260709 13:16:52.501267  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62817us
I20260709 13:16:52.506846  8073 raft_consensus.cc:2243] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.511632  8073 raft_consensus.cc:2272] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.517027  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 35df31b6c7b2475cac55c1e114ba24dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:52.518740  7913 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.521298  8073 ts_tablet_manager.cc:1916] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:52.551568  6415 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 4, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:52.558784  6515 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.563089  6663 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.568558  6595 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:52.571394  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:52.572152  8058 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:52.572608  8058 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:52.573532  8058 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.589565  6664 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:52.619107  6415 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.594398  7235 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:52.629974  6420 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.595381  6668 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:52.595826  7230 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.641498  6417 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.605902  6596 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.607779  8073 ts_tablet_manager.cc:1929] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:52.649194  8073 log.cc:1199] T 35df31b6c7b2475cac55c1e114ba24dc P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/35df31b6c7b2475cac55c1e114ba24dc
I20260709 13:16:52.609252  7237 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:52.651786  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 35df31b6c7b2475cac55c1e114ba24dc (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:52.639390  7913 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:16:52.664085  7914 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:52.610162  6595 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:52.662652  8012 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:16:52.669994  7779 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:52.693171  8052 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.695047  7230 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.758451  6515 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:52.773949  6591 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:52.778234  7785 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:52.799331  6517 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:52.803373  7981 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:52.805140  7235 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:52.821223  8051 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:52.848224  6517 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:52.850303  6667 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:52.854290  8051 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:52.859987  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fd28ad0fdaad48d586176f4a8e3f1aa3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:52.861482  7235 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.842718  8002 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.866879  6591 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:52.867935  7997 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:52.868392  6589 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:52.862670  7237 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.873474  6517 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:52.878751  6592 raft_consensus.cc:2955] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.872319  6664 raft_consensus.cc:2955] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.874548  7779 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.887203  6668 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:52.890995  8051 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:52.897107  7785 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:52.903160  6589 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:52.902777  6418 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.908560  6595 raft_consensus.cc:2955] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:52.905408  7981 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:16:52.910948  8003 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:16:52.931772  8020 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:52.944500  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23ba4e226865438d85ee0c061786f3db with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:52.958640  6420 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:53.014784  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bab5f01bd4774a168d3c01940a615cce with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:53.018502  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:53.045616  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet fd28ad0fdaad48d586176f4a8e3f1aa3 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:53.109817  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet bab5f01bd4774a168d3c01940a615cce with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:53.111065  6417 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:53.163559  8096 tablet_replica.cc:333] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:53.188730  6664 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:53.215940  8096 raft_consensus.cc:2243] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.217264  8096 raft_consensus.cc:2272] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.205673  7999 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:16:53.221412  8096 ts_tablet_manager.cc:1916] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:53.227394  8098 tablet_replica.cc:333] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:53.248739  8098 raft_consensus.cc:2243] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.249876  8098 raft_consensus.cc:2272] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.261489  8098 ts_tablet_manager.cc:1916] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:16:53.273536  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 67035.1us
I20260709 13:16:53.275430  6596 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:53.291424  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 23ba4e226865438d85ee0c061786f3db with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:53.301295  7999 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:53.307793  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:53.397261  8098 ts_tablet_manager.cc:1929] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:53.398921  8098 log.cc:1199] T bab5f01bd4774a168d3c01940a615cce P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/bab5f01bd4774a168d3c01940a615cce
I20260709 13:16:53.406831  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet bab5f01bd4774a168d3c01940a615cce (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:53.422708  8096 ts_tablet_manager.cc:1929] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:16:53.423213  8096 log.cc:1199] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/fd28ad0fdaad48d586176f4a8e3f1aa3
I20260709 13:16:53.428114  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet fd28ad0fdaad48d586176f4a8e3f1aa3 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:53.430226  8096 tablet_replica.cc:333] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:53.431263  8096 raft_consensus.cc:2243] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:53.431982  8096 raft_consensus.cc:2272] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:53.438683  8096 ts_tablet_manager.cc:1916] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:53.455385  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:53.455998  6591 raft_consensus.cc:606] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:16:53.460328  6664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:53.460870  6664 raft_consensus.cc:606] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:53.467063  8096 ts_tablet_manager.cc:1929] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:53.467505  8096 log.cc:1199] T 23ba4e226865438d85ee0c061786f3db P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/23ba4e226865438d85ee0c061786f3db
I20260709 13:16:53.468986  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 23ba4e226865438d85ee0c061786f3db (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:53.488494  6518 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:53.489089  6518 raft_consensus.cc:606] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:53.492057  6664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:53.492592  6664 raft_consensus.cc:606] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:53.493732  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:16:53.495566  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:53.622018  7981 raft_consensus.cc:993] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:53.622578  7904 raft_consensus.cc:1081] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:53.624295  6518 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:60540
I20260709 13:16:53.624851  6518 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:53.625249  6518 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.631770  6518 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:53.634166  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:53.634713  6591 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:53.635000  6591 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:53.635859  6591 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:53.636868  6591 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.641948  6518 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:53.648222  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:53.652316  7237 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.654263  6591 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:53.656705  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:53.657549  8097 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:53.658064  8097 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:53.658901  8097 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:53.674829  7237 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:53.708470  6415 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:53.773563  8095 raft_consensus.cc:993] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:53.774078  8095 raft_consensus.cc:1081] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:53.775875  7237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46666
I20260709 13:16:53.776479  7237 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:53.777311  7237 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.793185  7237 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:53.796658  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:53.797284  6518 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.797562  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:53.798151  6664 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.798422  6664 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:53.799288  6664 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:53.800380  6664 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.810150  7237 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:53.815201  6664 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:16:53.816601  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:53.817430  7935 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:53.818028  7935 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:53.819787  7935 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:53.828904  6518 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:16:53.838598  6415 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:53.975028  7785 raft_consensus.cc:993] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:53.975517  8095 raft_consensus.cc:1081] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:53.977396  6592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:53.977953  6592 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:53.978300  6592 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.000540  6592 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:54.004539  8097 raft_consensus.cc:993] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:54.005048  8097 raft_consensus.cc:1081] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:54.006587  6664 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:60486
I20260709 13:16:54.007133  6664 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:54.007462  6664 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.014256  6664 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:54.019127  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:54.019735  6518 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:54.020018  6518 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:54.020864  6518 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:54.027113  6518 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.036026  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:54.036651  6668 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:54.036950  6668 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:54.037721  6668 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:54.038738  6668 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.039376  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:54.039908  7237 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.044322  6664 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:54.045670  6668 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:54.050122  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:54.052073  7882 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:54.052582  7882 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:54.053490  7882 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:54.060048  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:54.060645  6591 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.067642  6591 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:54.069533  7237 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:54.073704  6637 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:54.073745  6592 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:54.076701  8036 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:54.077337  6518 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:54.077522  8036 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:54.078583  8036 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:54.102083  6415 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:54.110982  6417 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:54.137883  7237 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:54.139365  8002 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:54.210781  6592 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:54.212584  8002 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:54.225554  6518 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:54.227401  7935 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:54.274912  6664 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:54.276255  7999 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:54.471355  6518 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:54.472280  7237 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:54.473024  8095 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:54.473826  8044 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:54.498453  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:54.557320  6591 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:54.558779  6667 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:54.563349  7981 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:54.563103  8095 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:16:54.634665  6667 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:54.644280  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:54.743208  6636 consensus_peers.cc:597] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a758d1835d6148668ab495504660d72a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:54.750432  6667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:54.751163  6667 raft_consensus.cc:606] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:54.755393  6516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:54.755913  6516 raft_consensus.cc:606] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:54.762579  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:54.765036  6591 raft_consensus.cc:606] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:16:54.763199  6592 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:54.763746  6518 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:54.767066  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:16:54.768500  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:54.775965  8120 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:54.780009  8120 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:54.798220  8119 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:54.801143  6591 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:54.814337  6516 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:54.825109  8095 raft_consensus.cc:993] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:54.825699  8036 raft_consensus.cc:1081] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:54.828819  6592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:54.829506  6592 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:54.830024  6592 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.829344  6415 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:54.834478  6487 consensus_peers.cc:597] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 943ce377e70a422a8376b294da2840fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:54.836746  6596 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:54.840610  8002 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:54.840669  6592 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:54.846495  6667 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:54.847908  8002 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:54.848359  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:54.849077  6516 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.852213  6592 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:54.853367  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:54.854127  6668 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:54.854543  6668 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:54.855556  6668 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:54.857173  6668 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:54.859985  6516 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:54.862592  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:54.864745  7981 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:54.865238  7981 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:54.866190  7981 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:54.870942  6668 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:54.872460  6664 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:54.869442  7914 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:54.879491  6516 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:54.900127  6591 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:54.919512  6415 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:54.974692  6415 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:54.996723  6516 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:16:55.019989  6487 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c3ba394f290437b90045c4a98fb75ca. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:55.021054  6664 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:55.023947  8133 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:55.033538  6591 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:55.041917  8136 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.045368  8135 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:55.047619  6664 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.065496  6591 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.074525  7904 raft_consensus.cc:993] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:55.075047  6596 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:55.078035  6417 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.075094  7904 raft_consensus.cc:1081] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:55.085070  6518 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:60540
I20260709 13:16:55.085554  6518 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:55.085888  6518 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.093897  6518 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:55.096199  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:55.096705  6592 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:55.096993  6592 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:55.097729  6592 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:55.099097  6592 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.105326  6516 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:55.106370  6668 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:55.106855  8044 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:55.107945  6562 consensus_peers.cc:597] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f9f5da6098b64b229cd872c6346351f3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:55.108047  7981 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:55.115664  8135 raft_consensus.cc:993] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:55.116140  8134 raft_consensus.cc:1081] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:55.117944  6664 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:60486
I20260709 13:16:55.118471  6664 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:55.119524  6664 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:55.135311  6664 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:55.136000  6518 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:55.136837  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:55.137434  6668 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.137713  6592 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:16:55.138939  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:55.139684  8133 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:55.140152  8133 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:55.141206  6664 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:55.140988  8133 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:55.146238  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:55.151341  6518 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:55.151640  6518 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:55.142539  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:55.155349  6518 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:55.155714  7235 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:55.156651  6518 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:55.155643  6664 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.144825  7904 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.189369  6668 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:16:55.192387  7235 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
W20260709 13:16:55.193657  8141 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:16:55.193717  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:55.194604  8095 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:55.195091  8095 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:55.195873  8095 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:55.198030  6515 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.200380  6518 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:55.201579  6417 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:55.204146  6415 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:55.257999  8145 ts_tablet_manager.cc:933] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:55.265779  6417 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:55.269891  8147 ts_tablet_manager.cc:933] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:55.327889  8145 tablet_copy_client.cc:323] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:16:55.329607  6680 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet a758d1835d6148668ab495504660d72a from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:16:55.330140  6680 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet a758d1835d6148668ab495504660d72a from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-a758d1835d6148668ab495504660d72a
I20260709 13:16:55.334229  8147 tablet_copy_client.cc:323] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:55.336611  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 943ce377e70a422a8376b294da2840fa from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:55.337150  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 943ce377e70a422a8376b294da2840fa from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-943ce377e70a422a8376b294da2840fa
I20260709 13:16:55.347354  6664 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:55.350958  8044 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:55.364053  6515 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:55.369926  7882 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:55.389395  6532 tablet_copy_source_session.cc:215] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:55.400468  8147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943ce377e70a422a8376b294da2840fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.428568  6680 tablet_copy_source_session.cc:215] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:55.431926  8145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a758d1835d6148668ab495504660d72a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.442790  8147 tablet_copy_client.cc:806] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:55.443439  8147 tablet_copy_client.cc:670] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:55.462553  8153 ts_tablet_manager.cc:933] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:16:55.469336  8147 tablet_copy_client.cc:538] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:55.522370  8147 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
W20260709 13:16:55.534574  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56684.8us
I20260709 13:16:55.549353  8153 tablet_copy_client.cc:323] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:16:55.579479  8145 tablet_copy_client.cc:806] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:55.587594  7237 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:55.599722  6595 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:55.603873  8145 tablet_copy_client.cc:670] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:55.611985  8145 tablet_copy_client.cc:538] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:55.618005  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 1c3ba394f290437b90045c4a98fb75ca from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:16:55.618542  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 1c3ba394f290437b90045c4a98fb75ca from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-1c3ba394f290437b90045c4a98fb75ca
I20260709 13:16:55.604002  8120 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:16:55.606527  8134 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:16:55.652884  6532 tablet_copy_source_session.cc:215] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:55.661652  8153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c3ba394f290437b90045c4a98fb75ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.665702  8145 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:55.696933  6515 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:55.698654  8163 ts_tablet_manager.cc:933] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:55.698696  8095 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:55.733587  6668 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:55.784360  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:55.781658  8136 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:16:55.832367  8163 tablet_copy_client.cc:323] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:16:55.845067  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet f9f5da6098b64b229cd872c6346351f3 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:16:55.845660  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet f9f5da6098b64b229cd872c6346351f3 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-f9f5da6098b64b229cd872c6346351f3
I20260709 13:16:55.851807  8147 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:55.852854  8147 tablet_bootstrap.cc:492] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:55.853516  8147 ts_tablet_manager.cc:1403] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.350s	user 0.033s	sys 0.037s
I20260709 13:16:55.860677  8153 tablet_copy_client.cc:806] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:55.861299  8153 tablet_copy_client.cc:670] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:55.873425  8153 tablet_copy_client.cc:538] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:55.920107  8147 raft_consensus.cc:359] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:55.927557  8147 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:55.928287  8147 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:55.918442  6607 tablet_copy_source_session.cc:215] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:55.968696  8163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f5da6098b64b229cd872c6346351f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:55.985514  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 130930us
I20260709 13:16:55.992206  8153 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:56.000568  8147 ts_tablet_manager.cc:1434] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.147s	user 0.006s	sys 0.003s
I20260709 13:16:56.002270  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-943ce377e70a422a8376b294da2840fa received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:56.002775  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-943ce377e70a422a8376b294da2840fa on tablet 943ce377e70a422a8376b294da2840fa with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:56.049525  8163 tablet_copy_client.cc:806] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:56.050379  8163 tablet_copy_client.cc:670] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:56.050722  8145 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:56.051734  8145 tablet_bootstrap.cc:492] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:56.052448  8145 ts_tablet_manager.cc:1403] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.387s	user 0.021s	sys 0.015s
I20260709 13:16:56.054188  8163 tablet_copy_client.cc:538] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:56.067456  8145 raft_consensus.cc:359] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:56.088047  8145 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:56.088827  8145 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:56.103292  8145 ts_tablet_manager.cc:1434] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 13:16:56.108429  6680 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-a758d1835d6148668ab495504660d72a received from {username='slave'} at 127.0.0.1:51542
I20260709 13:16:56.108876  6680 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-a758d1835d6148668ab495504660d72a on tablet a758d1835d6148668ab495504660d72a with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:56.121685  8163 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:16:56.202396  8153 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:56.203747  8153 tablet_bootstrap.cc:492] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:56.204507  8153 ts_tablet_manager.cc:1403] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.213s	user 0.037s	sys 0.034s
I20260709 13:16:56.208833  8153 raft_consensus.cc:359] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:56.209873  8153 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:56.210326  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:56.210664  8153 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:56.213271  8153 ts_tablet_manager.cc:1434] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:16:56.219336  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-1c3ba394f290437b90045c4a98fb75ca received from {username='slave'} at 127.0.0.1:59988
I20260709 13:16:56.219786  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-1c3ba394f290437b90045c4a98fb75ca on tablet 1c3ba394f290437b90045c4a98fb75ca with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:56.234186  7237 raft_consensus.cc:1217] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:56.308111  6592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:56.308710  6592 raft_consensus.cc:606] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:16:56.313153  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:56.313786  6665 raft_consensus.cc:606] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:56.325935  6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "943ce377e70a422a8376b294da2840fa"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:56.326505  6515 raft_consensus.cc:606] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:56.327739  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:16:56.329306  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:56.329983  8163 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:56.330933  8163 tablet_bootstrap.cc:492] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:16:56.331604  8163 ts_tablet_manager.cc:1403] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.210s	user 0.030s	sys 0.031s
I20260709 13:16:56.334805  8163 raft_consensus.cc:359] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:56.335638  8163 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:16:56.336304  8163 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:56.350302  8163 ts_tablet_manager.cc:1434] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 13:16:56.360101  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-f9f5da6098b64b229cd872c6346351f3 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:16:56.360584  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-f9f5da6098b64b229cd872c6346351f3 on tablet f9f5da6098b64b229cd872c6346351f3 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:56.434283  7237 raft_consensus.cc:1217] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:16:56.452042  7235 raft_consensus.cc:1217] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:56.528649  7981 raft_consensus.cc:993] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:56.529337  8171 raft_consensus.cc:1081] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:56.533389  6515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:60540
I20260709 13:16:56.533988  6515 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:56.534336  6515 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.545816  6515 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:56.548825  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:56.549921  6595 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:56.550312  6595 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:56.551299  6595 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:56.552551  6595 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.559077  6515 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:56.598242  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:56.598869  6664 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.614267  8119 raft_consensus.cc:1064] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:56.617457  6664 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:56.617112  8119 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:56.625057  6595 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:56.632592  7235 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:56.664075  6488 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:56.668941  6515 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:56.706521  8182 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:56.707386  8182 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:56.710108  6595 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:56.710923  8182 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:56.716099  7237 raft_consensus.cc:1217] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:16:56.736917  8119 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:56.774451  8120 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:56.794679  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 57991.7us
I20260709 13:16:56.795300  8175 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:16:56.853667  8134 raft_consensus.cc:993] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:56.854303  8134 raft_consensus.cc:1081] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:56.856621  6664 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "943ce377e70a422a8376b294da2840fa"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:60486
I20260709 13:16:56.857308  6664 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:56.857646  6664 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:56.890218  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 65792.5us
I20260709 13:16:56.960379  6417 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.976722  8168 raft_consensus.cc:1064] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
W20260709 13:16:56.977228  8168 raft_consensus.cc:1068] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Unable to promote non-voter 64b95375974b48d18c596a23c9cac75c: Service unavailable: leader transfer in progress
W20260709 13:16:56.977517  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 124062us
I20260709 13:16:56.988440  6664 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:57.001366  6664 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:57.050875  8134 raft_consensus.cc:1064] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:57.054034  8134 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:57.057337  8036 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.060766  8119 raft_consensus.cc:993] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:57.061352  8175 raft_consensus.cc:1081] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:57.064293  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:57.064760  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:57.064921  6515 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:57.065266  6515 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:57.065312  6592 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:57.066200  6515 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:57.068061  6515 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:57.073207  6592 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:57.086937  6592 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.092378  6591 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:57.092938  6591 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:57.093281  6591 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:57.096994  7235 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.104100  6665 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.105319  6515 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:16:57.105763  6591 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:57.112926  8168 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.119166  6664 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:57.119545  7237 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:57.121699  8133 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:57.124640  6595 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:57.128975  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:57.129273  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:57.130389  6591 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:57.132723  8119 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:57.133296  8119 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:57.133774  8182 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:57.129957  6515 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:57.134292  8119 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:57.136091  8134 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:57.146404  6517 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.164106  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:57.164716  6664 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:57.164988  6664 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:57.165524  6515 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:16:57.180680  6664 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:57.182248  6664 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:57.182776  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:57.193336  8044 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:57.193823  8044 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:57.198083  6664 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:16:57.199167  8044 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:57.243062  6592 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.262850  8170 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:57.309592  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 82374.4us
I20260709 13:16:57.330322  6420 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.344782  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:57.382958  8166 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:16:57.441712  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 73680us
I20260709 13:16:57.447204  8044 raft_consensus.cc:1064] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:57.495426  8044 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:57.501757  6591 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.486908  6665 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:16:57.516836  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 98126.5us
W20260709 13:16:57.552467  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52240.3us
I20260709 13:16:57.561657  6665 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:57.564410  7235 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:57.575210  6667 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:57.574402  6418 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.576764  6418 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.577448  8211 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:57.578254  6517 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:57.580533  7882 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:57.597339  7230 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:57.596385  7235 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.604012  6591 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:57.607425  6415 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.609925  7882 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:57.610813  8200 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:57.635062  8209 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:57.663753  6591 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:57.668764  8120 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.675798  8200 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:57.673507  6596 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.697446  8216 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.698782  7235 raft_consensus.cc:2955] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.701603  7237 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.741291  7230 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Advancing to term 2
I20260709 13:16:57.762396  6515 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:57.777055  8200 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:57.805245  6515 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:57.825754  7882 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:57.839532  7230 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:57.841693  6420 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.842484  8119 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:57.843845  6420 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.849713  6667 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.862776  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet a758d1835d6148668ab495504660d72a with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:57.865101  6517 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.886044  6665 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:57.888818  7882 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:57.901266  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a758d1835d6148668ab495504660d72a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:57.929080  6521 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:57.933419  8227 tablet_replica.cc:333] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:57.934746  8227 raft_consensus.cc:2243] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:57.935648  8227 raft_consensus.cc:2272] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:57.948172  7237 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:57.948566  6591 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:57.949635  8170 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:57.951936  8166 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:57.954277  6591 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:57.966602  7235 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:57.970309  6665 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:57.969549  7232 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.971802  8214 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:57.979580  7882 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:57.955369  8227 ts_tablet_manager.cc:1916] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:57.980618  8211 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.982110  8201 raft_consensus.cc:1064] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:16:57.965665  8182 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.992893  7235 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.002708  6665 raft_consensus.cc:2955] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.006229  6591 raft_consensus.cc:2955] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.034114  6418 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:57.985540  8201 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.058926  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f9f5da6098b64b229cd872c6346351f3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:58.079730  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c3ba394f290437b90045c4a98fb75ca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:58.094337  6417 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.113405  7235 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:58.117544  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c3ba394f290437b90045c4a98fb75ca with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45428
I20260709 13:16:58.118907  6515 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:58.122574  8120 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:58.126513  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:58.117587  6591 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:58.154104  8200 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:58.155380  8232 tablet_replica.cc:333] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:16:58.156005  8227 ts_tablet_manager.cc:1929] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:58.156648  8232 raft_consensus.cc:2243] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:58.183183  8227 log.cc:1199] T a758d1835d6148668ab495504660d72a P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/a758d1835d6148668ab495504660d72a
I20260709 13:16:58.183625  8232 raft_consensus.cc:2272] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:58.174921  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet f9f5da6098b64b229cd872c6346351f3 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 5)) from {username='slave'} at 127.0.0.1:60532
I20260709 13:16:58.187544  8232 ts_tablet_manager.cc:1916] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:58.188256  8227 tablet_replica.cc:333] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:16:58.188831  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet a758d1835d6148668ab495504660d72a (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:58.200450  8209 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:58.218549  8227 raft_consensus.cc:2243] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:58.221851  8227 raft_consensus.cc:2272] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:58.222534  8200 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.228070  8227 ts_tablet_manager.cc:1916] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:58.237831  8232 ts_tablet_manager.cc:1929] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:16:58.238389  8232 log.cc:1199] T 1c3ba394f290437b90045c4a98fb75ca P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/1c3ba394f290437b90045c4a98fb75ca
I20260709 13:16:58.241930  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 1c3ba394f290437b90045c4a98fb75ca (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:58.246959  6591 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.257385  6521 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.264503  7235 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.285463  6415 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 2 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.306998  8227 ts_tablet_manager.cc:1929] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:16:58.307706  8227 log.cc:1199] T f9f5da6098b64b229cd872c6346351f3 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/f9f5da6098b64b229cd872c6346351f3
I20260709 13:16:58.308579  6663 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.321790  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet f9f5da6098b64b229cd872c6346351f3 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:58.342139  7235 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:58.342545  6521 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:58.347421  8200 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:58.348626  8201 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:58.357204  8209 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.363781  7235 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.386458  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:58.387233  6591 raft_consensus.cc:606] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:16:58.390350  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:58.390998  6521 raft_consensus.cc:606] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:58.391223  6517 raft_consensus.cc:2955] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.394258  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:16:58.394878  6591 raft_consensus.cc:606] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:58.406922  6663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:58.408022  6663 raft_consensus.cc:606] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:16:58.409528  6417 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:16:58.413945  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:16:58.415395  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 943ce377e70a422a8376b294da2840fa with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:58.417402  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:58.435434  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet 943ce377e70a422a8376b294da2840fa with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:48452
I20260709 13:16:58.448563  8233 tablet_replica.cc:333] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:16:58.449705  8233 raft_consensus.cc:2243] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:58.450495  8233 raft_consensus.cc:2272] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:58.460419  8233 ts_tablet_manager.cc:1916] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:58.481709  8233 ts_tablet_manager.cc:1929] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:16:58.482164  8233 log.cc:1199] T 943ce377e70a422a8376b294da2840fa P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/943ce377e70a422a8376b294da2840fa
I20260709 13:16:58.484203  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet 943ce377e70a422a8376b294da2840fa (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:16:58.529862  7882 raft_consensus.cc:993] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:58.530385  7882 raft_consensus.cc:1081] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:16:58.533352  7235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:16:58.533879  7235 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.534207  7235 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.542798  7235 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.545667  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:58.546249  6591 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:58.546540  6591 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:58.547426  6591 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.548776  7235 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:58.548873  6591 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.551116  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:58.551652  6663 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.558647  6591 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:58.560714  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:16:58.561599  8146 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.562062  8146 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:16:58.562942  8146 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.564014  6663 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:16:58.578090  6416 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 2 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.599785  8214 raft_consensus.cc:993] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:58.600275  8214 raft_consensus.cc:1081] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:16:58.603820  6663 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45444
I20260709 13:16:58.604316  6663 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.604629  6663 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:58.610217  6663 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:58.611709  6663 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:16:58.612686  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:58.613250  6517 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:58.613150  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:16:58.615123  6591 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:58.615450  6591 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:58.616173  6591 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:58.617300  6591 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:58.622913  6517 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:16:58.624204  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:58.625012  8201 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:58.625507  8201 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:58.626366  8201 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:58.629266  6591 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:16:58.640442  8200 raft_consensus.cc:993] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:58.640933  8200 raft_consensus.cc:1081] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:16:58.642793  6521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:48592
I20260709 13:16:58.643309  6521 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.643635  6521 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.655678  6521 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.655560  6416 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 3 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.659030  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:58.659574  6663 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:58.659875  6663 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:16:58.660650  6663 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.661721  6663 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.677889  6521 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:16:58.678807  6663 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:58.682118  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:16:58.682220  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:16:58.682830  7235 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.684508  8182 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.685086  8182 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:58.693010  7235 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:16:58.686141  8182 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:16:58.740267  6416 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.969000  8170 raft_consensus.cc:993] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:58.969492  8170 raft_consensus.cc:1081] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:58.971566  6591 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:58.971378  6596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:16:58.974640  6596 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.975016  6596 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.983208  8215 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:58.986593  6665 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:58.988873  8215 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:58.991935  6596 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:59.049382  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:16:59.050050  6517 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.050135  6591 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:59.050402  6517 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:16:59.051376  6517 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:59.051929  8120 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:59.052740  6517 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.065994  6596 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:16:59.067174  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:16:59.067853  6663 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.076275  6663 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:16:59.084764  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:16:59.085811  8214 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.086659  8214 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:16:59.087697  8214 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:16:59.103417  6663 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:59.105367  8166 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:59.159344  6516 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:59.163884  8120 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:59.247257  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
W20260709 13:16:59.253758  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79550.5us
I20260709 13:16:59.253919  6416 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.322528  7235 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:59.324141  8170 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:16:59.364152  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 110505us
I20260709 13:16:59.424733  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:59.611631  6668 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:59.633961  6596 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:59.636727  8261 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:16:59.642246  6636 consensus_peers.cc:597] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff27e205e84e4edf946bc02c573a2c25. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:59.642829  6516 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:59.647377  8261 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:59.655109  8120 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.668967  6592 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.672106  6668 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.684386  8206 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.694759  6521 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.698772  8206 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.717854  6596 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:59.724870  6516 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:16:59.738888  6562 consensus_peers.cc:597] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a2845fc76124deab8a20512c6112df7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:59.739518  6668 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:59.752405  6415 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.761147  8211 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:59.793849  6517 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:59.797667  8273 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:59.806185  6668 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.803473  8211 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.827174  6417 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.838069  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:59.850104  6517 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.856873  6521 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:59.889416  6668 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:59.890976  6590 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:59.895582  6487 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2c67ec2db9643088bc65fe9d83aa732. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:59.896646  8257 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.897809  8258 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.940338  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835ac063488a40878ccb02b662d3898b"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:16:59.940948  6521 raft_consensus.cc:606] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:16:59.944141  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:16:59.944697  6665 raft_consensus.cc:606] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:16:59.946065  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:16:59.947840  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:59.955173  8258 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.959741  6590 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.972702  6665 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:16:59.983822  8120 raft_consensus.cc:993] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:59.984320  8120 raft_consensus.cc:1081] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:16:59.987798  6596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55990
I20260709 13:16:59.988335  6596 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.988691  6596 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.996245  6592 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:16:59.998883  6596 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:00.001497  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:00.002025  6517 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:00.002588  6417 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:00.006047  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:00.006669  6668 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:00.006975  6668 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:00.007694  6668 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:00.008724  6668 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:00.012705  6517 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:00.013893  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:00.015033  8277 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:00.015491  8277 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:00.016347  8277 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:00.041002  6596 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:00.056821  6668 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
W20260709 13:17:00.090401  6562 consensus_peers.cc:597] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91a4b9d0715e4a3ba0f22cebfe819bb1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:00.099947  6668 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:00.100420  6417 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:00.101363  8206 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:00.104740  6521 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:00.116901  8211 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:00.143040  8214 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:00.152060  6517 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:00.165740  6668 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:00.176620  6417 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:00.189968  8258 raft_consensus.cc:993] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:17:00.190524  8258 raft_consensus.cc:1081] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:17:00.206207  6667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "835ac063488a40878ccb02b662d3898b"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:60486
I20260709 13:17:00.206941  6667 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:00.210357  6667 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:00.295693  8286 ts_tablet_manager.cc:933] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:00.313009  8285 ts_tablet_manager.cc:933] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:00.296808  6667 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:00.317382  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:00.317914  6521 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:00.318168  6521 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:00.318823  6521 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:00.319837  6521 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:00.338043  6521 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:00.346154  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:00.347304  8255 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:00.347750  8255 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:00.348579  8255 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:00.340229  6667 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:00.341980  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:00.358042  6592 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:00.364879  8285 tablet_copy_client.cc:323] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:17:00.367072  6592 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:00.368435  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 1a2845fc76124deab8a20512c6112df7 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:17:00.369014  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 1a2845fc76124deab8a20512c6112df7 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-1a2845fc76124deab8a20512c6112df7
I20260709 13:17:00.383337  6417 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:00.391000  6607 tablet_copy_source_session.cc:215] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:00.395713  8286 tablet_copy_client.cc:323] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:17:00.397647  6680 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet ff27e205e84e4edf946bc02c573a2c25 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:17:00.398209  6680 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet ff27e205e84e4edf946bc02c573a2c25 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-ff27e205e84e4edf946bc02c573a2c25
I20260709 13:17:00.406021  6680 tablet_copy_source_session.cc:215] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:00.408102  8285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a2845fc76124deab8a20512c6112df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:00.424144  8286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff27e205e84e4edf946bc02c573a2c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:00.428205  8285 tablet_copy_client.cc:806] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:00.428944  8285 tablet_copy_client.cc:670] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:00.433265  8285 tablet_copy_client.cc:538] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:00.442824  8285 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:00.447597  8286 tablet_copy_client.cc:806] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:00.448233  8286 tablet_copy_client.cc:670] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:00.450850  8290 ts_tablet_manager.cc:933] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:17:00.455008  8286 tablet_copy_client.cc:538] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:00.461656  8290 tablet_copy_client.cc:323] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:17:00.463651  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet b2c67ec2db9643088bc65fe9d83aa732 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:17:00.464237  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet b2c67ec2db9643088bc65fe9d83aa732 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-b2c67ec2db9643088bc65fe9d83aa732
I20260709 13:17:00.479444  6532 tablet_copy_source_session.cc:215] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:00.484933  6521 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.486799  8290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c67ec2db9643088bc65fe9d83aa732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:00.488797  8277 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.492673  6667 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.494318  8277 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.495959  8286 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:00.526669  8290 tablet_copy_client.cc:806] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:00.527242  8290 tablet_copy_client.cc:670] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:00.545456  8285 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:00.556892  8290 tablet_copy_client.cc:538] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:00.586891  8290 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:00.587406  8285 tablet_bootstrap.cc:492] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:00.599380  8285 ts_tablet_manager.cc:1403] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.157s	user 0.023s	sys 0.013s
I20260709 13:17:00.632236  8285 raft_consensus.cc:359] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:00.633152  8285 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:00.633859  8285 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:17:00.643275  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 52604us
I20260709 13:17:00.653035  8285 ts_tablet_manager.cc:1434] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:17:00.675464  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-1a2845fc76124deab8a20512c6112df7 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:17:00.675966  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-1a2845fc76124deab8a20512c6112df7 on tablet 1a2845fc76124deab8a20512c6112df7 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:00.683090  8286 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:00.702364  8285 ts_tablet_manager.cc:933] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:00.711151  7237 raft_consensus.cc:1217] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:17:00.711221  8286 tablet_bootstrap.cc:492] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:00.715327  8286 ts_tablet_manager.cc:1403] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.220s	user 0.026s	sys 0.000s
I20260709 13:17:00.723517  8290 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:00.724491  8290 tablet_bootstrap.cc:492] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:00.725152  8290 ts_tablet_manager.cc:1403] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.149s	user 0.022s	sys 0.001s
I20260709 13:17:00.728268  8285 tablet_copy_client.cc:323] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:17:00.728441  8290 raft_consensus.cc:359] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:00.729305  8290 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:00.730439  8286 raft_consensus.cc:359] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:00.731562  8286 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:00.730424  8290 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:00.732435  8286 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:00.735472  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:17:00.736055  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-91a4b9d0715e4a3ba0f22cebfe819bb1
I20260709 13:17:00.743449  8290 ts_tablet_manager.cc:1434] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:17:00.747239  6607 tablet_copy_source_session.cc:215] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:00.749490  6590 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.761101  8261 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.775439  8286 ts_tablet_manager.cc:1434] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
I20260709 13:17:00.779488  6680 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-ff27e205e84e4edf946bc02c573a2c25 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:17:00.779911  6680 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-ff27e205e84e4edf946bc02c573a2c25 on tablet ff27e205e84e4edf946bc02c573a2c25 with peer 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:00.796710  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61097.6us
I20260709 13:17:00.798008  6517 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.798058  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-b2c67ec2db9643088bc65fe9d83aa732 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:17:00.799213  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-b2c67ec2db9643088bc65fe9d83aa732 on tablet b2c67ec2db9643088bc65fe9d83aa732 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:00.801290  8263 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.801996  8285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:00.806116  7237 raft_consensus.cc:1217] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 13:17:00.869939  7200 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletCopyService.BeginTabletCopySession -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 69126.7us
I20260709 13:17:00.948714  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:00.954718  8285 tablet_copy_client.cc:806] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:00.955421  8285 tablet_copy_client.cc:670] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:00.974244  8285 tablet_copy_client.cc:538] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:01.010655  8285 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:01.011021  7235 raft_consensus.cc:1217] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:01.133733  8285 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:01.135599  8285 tablet_bootstrap.cc:492] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:01.136334  8285 ts_tablet_manager.cc:1403] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.135s	user 0.023s	sys 0.004s
I20260709 13:17:01.144217  8285 raft_consensus.cc:359] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:01.146610  8285 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:01.147513  8285 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:01.156534  8285 ts_tablet_manager.cc:1434] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:17:01.159977  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-91a4b9d0715e4a3ba0f22cebfe819bb1 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:17:01.160386  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-91a4b9d0715e4a3ba0f22cebfe819bb1 on tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:01.191052  8255 raft_consensus.cc:1064] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:01.195119  8255 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:01.212746  8277 raft_consensus.cc:1064] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:01.216269  8277 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:01.283109  7235 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:01.284711  8263 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:01.291710  7232 raft_consensus.cc:1217] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:17:01.318351  6592 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:01.319236  6517 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:01.323256  8263 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:01.325405  8263 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:01.354188  8263 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.362125  7235 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:01.365602  6517 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:01.369136  8277 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:01.370100  8211 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:01.393399  8258 raft_consensus.cc:1064] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:01.396019  8258 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:01.415999  6667 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:01.417886  7237 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:01.420930  8257 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:01.423520  6590 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:01.426882  8257 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:01.427990  8253 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:01.430694  6665 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:01.466398  8211 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:01.490931  8257 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.519904  6667 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.524013  6415 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.543272  6416 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.583506  6521 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:01.601605  6665 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:01.611912  8313 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:17:01.622313  8206 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.502267  7237 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.591333  6590 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.634637  7235 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.637389  6521 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:17:01.640314  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 121582us
I20260709 13:17:01.650511  7233 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.663282  7234 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:01.665416  6415 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.670202  6668 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:01.686187  8258 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:17:01.703155  7232 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:01.714438  8257 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:01.717586  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:01.731071  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47159.9us
I20260709 13:17:01.731992  8261 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:01.735477  8257 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.746376  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b2c67ec2db9643088bc65fe9d83aa732 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:01.747336  7234 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.765923  6416 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 1 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.771703  6663 raft_consensus.cc:2955] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.782719  8261 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.782768  6515 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.787871  7232 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.801251  6517 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:17:01.830943  8326 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:17:01.822201  8263 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:17:01.820207  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet b2c67ec2db9643088bc65fe9d83aa732 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48452
I20260709 13:17:01.840989  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ff27e205e84e4edf946bc02c573a2c25 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:01.856431  6417 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:01.861042  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 43526us
I20260709 13:17:01.868672  6517 raft_consensus.cc:2955] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.883649  8333 tablet_replica.cc:333] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:17:01.884886  8333 raft_consensus.cc:2243] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.885674  8333 raft_consensus.cc:2272] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:01.896754  6590 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:01.901615  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet ff27e205e84e4edf946bc02c573a2c25 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:48452
I20260709 13:17:01.907397  6521 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:01.908171  8333 ts_tablet_manager.cc:1916] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:01.909137  8206 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:01.918241  8299 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.919917  6521 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.930765  6665 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.938050  8297 raft_consensus.cc:1064] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:01.938994  6590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:17:01.940601  8297 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:01.953987  7232 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:01.963686  8206 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:01.979323  6417 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:01.995043  8333 ts_tablet_manager.cc:1929] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:02.012480  8333 log.cc:1199] T b2c67ec2db9643088bc65fe9d83aa732 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/b2c67ec2db9643088bc65fe9d83aa732
I20260709 13:17:01.996486  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a2845fc76124deab8a20512c6112df7 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 5)) from {username='slave'} at 127.0.0.1:45428
I20260709 13:17:02.023432  8333 tablet_replica.cc:333] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:17:02.026321  8333 raft_consensus.cc:2243] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.031232  8335 tablet_replica.cc:333] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:17:02.032747  7232 raft_consensus.cc:2955] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.034433  8335 raft_consensus.cc:2243] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.035559  8335 raft_consensus.cc:2272] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.040068  8335 ts_tablet_manager.cc:1916] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:02.044026  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1a2845fc76124deab8a20512c6112df7 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:02.044921  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet b2c67ec2db9643088bc65fe9d83aa732 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:02.045749  8333 pending_rounds.cc:70] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Trying to abort 1 pending ops.
I20260709 13:17:02.046164  8333 pending_rounds.cc:77] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7305637974809174016 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "ff27e205e84e4edf946bc02c573a2c25" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } } new_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } } }
I20260709 13:17:02.047844  8333 raft_consensus.cc:2917] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted
I20260709 13:17:02.048369  8333 raft_consensus.cc:2889] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.048743  8333 raft_consensus.cc:2272] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.056614  6517 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:02.059798  8211 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:02.061275  8333 ts_tablet_manager.cc:1916] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:02.106495  8333 ts_tablet_manager.cc:1929] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:17:02.106951  8333 log.cc:1199] T ff27e205e84e4edf946bc02c573a2c25 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/ff27e205e84e4edf946bc02c573a2c25
I20260709 13:17:02.108376  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet ff27e205e84e4edf946bc02c573a2c25 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:02.130888  7237 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:02.141255  6665 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:02.159410  6590 raft_consensus.cc:606] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:02.178449  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:17:02.179080  6665 raft_consensus.cc:606] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:17:02.179997  8299 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:02.188714  6590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:17:02.189406  6590 raft_consensus.cc:606] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:17:02.196559  8211 raft_consensus.cc:993] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:17:02.197021  8211 raft_consensus.cc:1081] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:17:02.199366  6517 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:60540
I20260709 13:17:02.199885  6517 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.200239  6517 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.202298  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:02.204489  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:02.233211  6517 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:02.236027  8338 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:02.256752  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:02.257326  6596 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:02.257591  6596 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:02.258270  6596 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:02.259613  6596 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.269536  8336 raft_consensus.cc:993] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:17:02.270911  8297 raft_consensus.cc:1081] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:17:02.286962  8336 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.297755  7237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:17:02.298452  7237 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.298955  7237 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.298357  7232 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.307197  6517 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:02.309185  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:02.309791  6665 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.316920  6665 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:02.318722  6486 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:02.319595  8327 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:02.320305  8327 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:02.321702  8327 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:02.335227  6417 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.309815  8335 ts_tablet_manager.cc:1929] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:17:02.348579  8335 log.cc:1199] T 1a2845fc76124deab8a20512c6112df7 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/1a2845fc76124deab8a20512c6112df7
I20260709 13:17:02.350541  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 1a2845fc76124deab8a20512c6112df7 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:02.351773  6596 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:02.377696  6521 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.379515  7237 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:02.381729  7237 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:02.382700  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:02.383276  6515 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.383692  6665 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.389402  6415 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.395658  6515 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:02.397047  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:02.397862  8331 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:02.398334  8331 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:02.399158  8331 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:02.411440  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:02.412022  6590 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:02.412322  6590 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:02.412998  6590 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:02.414170  6590 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.419889  6417 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.426038  6596 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:02.444991  6590 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:02.481142  6665 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:02.481382  7237 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:02.482815  8336 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:02.517316  8297 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.522927  7237 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.553011  6417 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.572692  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60532
I20260709 13:17:02.625241  8336 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:02.639954  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:02.669344  6667 raft_consensus.cc:2955] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:02.682035  8329 raft_consensus.cc:993] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:17:02.683120  8261 raft_consensus.cc:1081] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:17:02.696769  8344 tablet_replica.cc:333] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:17:02.697808  6521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:48592
I20260709 13:17:02.701030  6521 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.701424  6521 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.745033  8344 raft_consensus.cc:2243] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.757629  8344 raft_consensus.cc:2272] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.770551  8344 ts_tablet_manager.cc:1916] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:02.777503  6521 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:02.779851  6521 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:02.780788  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:02.781364  6665 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:02.781782  6665 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:02.782528  6665 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:02.783742  6665 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.785655  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:02.786239  7234 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.791431  8344 ts_tablet_manager.cc:1929] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:17:02.791954  8344 log.cc:1199] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/91a4b9d0715e4a3ba0f22cebfe819bb1
I20260709 13:17:02.793339  7234 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:02.793864  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 91a4b9d0715e4a3ba0f22cebfe819bb1 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:02.794862  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:02.795713  8253 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:02.796171  8253 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:02.797011  8253 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:02.802330  6665 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:02.824589  6590 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:02.825811  8258 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:02.828498  6415 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 1 to 2, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.863292  6665 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:02.868192  8258 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:02.914593  6521 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:02.917128  8260 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:02.921813  6590 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:02.927217  8331 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:03.205395  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:03.218122  6668 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:03.219650  8253 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:03.257910  7237 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:03.260540  8258 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:03.667620  6521 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:17:03.673770  6487 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 40b7cf612223418ea8fe2d636986bb3f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.676937  6665 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:03.678509  8328 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:03.682163  6590 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:03.686205  8328 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:17:03.702826  8327 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:03.725538  6663 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:03.746196  6590 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:03.761901  6415 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:03.780079  6592 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
W20260709 13:17:03.997459  6562 consensus_peers.cc:597] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17e76729a2fc4273abedd8be9331a882. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.997818  6665 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:03.999298  8299 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:04.000134  6521 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:04.005589  8299 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:04.025970  8299 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.044190  6515 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.034768  6665 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.054966  6590 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:04.068580  6417 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.092600  6668 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:04.094722  8297 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:04.093621  6515 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:17:04.096815  6562 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6eb5773be2294301bdb1bbedf5765666. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:04.098047  8336 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:04.115242  8367 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.125100  6515 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.133944  6521 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:04.133847  6665 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.143852  6417 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.150082  6668 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:17:04.153365  6487 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d83f8b945cd641db90647da0461be4a3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:04.153990  6590 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:04.157177  8258 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:17:04.162109  8253 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:04.184712  8328 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.186897  6668 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.203069  6590 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:04.207940  6417 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.220252  8374 ts_tablet_manager.cc:933] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:17:04.264406  8374 tablet_copy_client.cc:323] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:17:04.268117  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 40b7cf612223418ea8fe2d636986bb3f from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:17:04.269189  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 40b7cf612223418ea8fe2d636986bb3f from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-40b7cf612223418ea8fe2d636986bb3f
I20260709 13:17:04.298163  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:04.310571  6532 tablet_copy_source_session.cc:215] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:04.322711  8374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40b7cf612223418ea8fe2d636986bb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:04.342177  8374 tablet_copy_client.cc:806] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:04.342883  8374 tablet_copy_client.cc:670] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:04.346722  8374 tablet_copy_client.cc:538] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:04.358163  8374 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:04.419552  8374 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:04.420572  8374 tablet_bootstrap.cc:492] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:04.421350  8374 ts_tablet_manager.cc:1403] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.063s	user 0.026s	sys 0.014s
I20260709 13:17:04.426440  6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:04.427091  6515 raft_consensus.cc:606] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:04.431552  8377 ts_tablet_manager.cc:933] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:04.433442  8374 raft_consensus.cc:359] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:04.434316  8374 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:04.435177  8374 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:04.437654  8374 ts_tablet_manager.cc:1434] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.000s
I20260709 13:17:04.431391  6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:04.439337  6515 raft_consensus.cc:606] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:17:04.440502  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:04.442513  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:04.444353  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-40b7cf612223418ea8fe2d636986bb3f received from {username='slave'} at 127.0.0.1:59988
I20260709 13:17:04.444803  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-40b7cf612223418ea8fe2d636986bb3f on tablet 40b7cf612223418ea8fe2d636986bb3f with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:04.449826  8377 tablet_copy_client.cc:323] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:17:04.453011  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 17e76729a2fc4273abedd8be9331a882 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:17:04.453588  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 17e76729a2fc4273abedd8be9331a882 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-17e76729a2fc4273abedd8be9331a882
I20260709 13:17:04.470818  6607 tablet_copy_source_session.cc:215] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:04.474797  8377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17e76729a2fc4273abedd8be9331a882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:04.492554  8377 tablet_copy_client.cc:806] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:04.493170  8377 tablet_copy_client.cc:670] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:04.499109  8377 tablet_copy_client.cc:538] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:04.508941  8377 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:04.572410  8374 ts_tablet_manager.cc:933] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:17:04.571969  8377 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:04.579367  8377 tablet_bootstrap.cc:492] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:04.580098  8377 ts_tablet_manager.cc:1403] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.071s	user 0.029s	sys 0.006s
I20260709 13:17:04.595582  8374 tablet_copy_client.cc:323] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:17:04.725823  8377 raft_consensus.cc:359] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:04.737512  8377 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:04.738322  8377 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:04.741339  8377 ts_tablet_manager.cc:1434] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.161s	user 0.017s	sys 0.010s
I20260709 13:17:04.849222  8258 raft_consensus.cc:993] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:04.849750  8389 raft_consensus.cc:1081] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:04.851423  6595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:17:04.851966  6595 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:04.852392  6595 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:04.857046  7237 raft_consensus.cc:1217] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:17:04.859726  6595 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:04.861646  6595 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:04.878479  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet d83f8b945cd641db90647da0461be4a3 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:17:04.879097  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet d83f8b945cd641db90647da0461be4a3 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-d83f8b945cd641db90647da0461be4a3
I20260709 13:17:04.890735  6532 tablet_copy_source_session.cc:215] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:04.902643  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-17e76729a2fc4273abedd8be9331a882 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:17:04.903118  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-17e76729a2fc4273abedd8be9331a882 on tablet 17e76729a2fc4273abedd8be9331a882 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:04.906742  7237 raft_consensus.cc:1217] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:17:04.907284  8374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d83f8b945cd641db90647da0461be4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:04.915562  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:04.916198  6515 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:04.916496  6515 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:04.917140  6515 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:04.918098  6515 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:04.926811  8399 ts_tablet_manager.cc:933] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:04.936189  8399 tablet_copy_client.cc:323] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:17:04.946794  6515 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:04.947311  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:04.947880  6668 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:04.971750  6668 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:04.972407  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 6eb5773be2294301bdb1bbedf5765666 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:17:04.973032  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 6eb5773be2294301bdb1bbedf5765666 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-6eb5773be2294301bdb1bbedf5765666
I20260709 13:17:04.983773  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:04.984875  8397 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:04.985412  8397 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:04.986325  8397 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:04.993857  6607 tablet_copy_source_session.cc:215] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:05.010309  8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eb5773be2294301bdb1bbedf5765666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.011822  8374 tablet_copy_client.cc:806] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:05.012426  8374 tablet_copy_client.cc:670] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
W20260709 13:17:05.015277  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47894us
I20260709 13:17:05.019032  8374 tablet_copy_client.cc:538] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:17:05.022921  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 55750.2us
I20260709 13:17:05.033871  8393 raft_consensus.cc:993] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:17:05.034329  8393 raft_consensus.cc:1081] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:17:05.037299  8399 tablet_copy_client.cc:806] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:05.037890  8399 tablet_copy_client.cc:670] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:05.043509  8374 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:05.044551  6665 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:60486
I20260709 13:17:05.045127  6665 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:05.045470  6665 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:05.052421  6665 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.054893  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:05.055436  6515 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:05.055718  6515 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:05.056433  6515 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.057477  6515 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:05.059908  6665 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:05.066318  6415 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.069486  8399 tablet_copy_client.cc:538] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:05.073750  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:05.074297  7232 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:05.080859  7232 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:05.082058  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:05.082886  8261 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:05.083320  8261 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:05.084057  8261 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.090071  8374 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:05.091084  8374 tablet_bootstrap.cc:492] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:05.092053  8374 ts_tablet_manager.cc:1403] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.049s	user 0.020s	sys 0.012s
I20260709 13:17:05.093206  6515 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:05.095563  8374 raft_consensus.cc:359] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:05.097700  8374 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:05.098364  8374 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:05.102707  8374 ts_tablet_manager.cc:1434] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:05.104430  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-d83f8b945cd641db90647da0461be4a3 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:17:05.104856  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-d83f8b945cd641db90647da0461be4a3 on tablet d83f8b945cd641db90647da0461be4a3 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:05.114068  8399 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:05.120890  6416 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.156248  7232 raft_consensus.cc:1217] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:17:05.181201  8399 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:05.182204  8399 tablet_bootstrap.cc:492] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:05.182984  8399 ts_tablet_manager.cc:1403] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.011s
I20260709 13:17:05.185371  8399 raft_consensus.cc:359] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:05.186066  8399 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:05.186618  8399 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:05.188802  8399 ts_tablet_manager.cc:1434] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:17:05.190733  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-6eb5773be2294301bdb1bbedf5765666 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:17:05.191161  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-6eb5773be2294301bdb1bbedf5765666 on tablet 6eb5773be2294301bdb1bbedf5765666 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:05.277087  7232 raft_consensus.cc:1217] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:17:05.323820  8389 raft_consensus.cc:1064] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:05.327471  8389 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.340278  6665 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:05.341089  7232 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:05.341575  6590 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:05.343292  8393 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:05.344472  8328 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:05.355124  8389 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:17:05.397784  8328 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.401944  6590 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.402540  7237 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.420048  6665 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.425974  6415 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.442881  6668 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:05.444469  8397 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:05.443455  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:05.478211  6521 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.494758  6515 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:05.504251  7237 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:05.504961  6667 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:05.513182  8393 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:05.515623  8393 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.524430  7237 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:05.526204  8398 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:05.526876  8328 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.532527  8402 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:05.537143  7237 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.539135  6667 raft_consensus.cc:2955] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.542789  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 40b7cf612223418ea8fe2d636986bb3f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:05.561530  8417 raft_consensus.cc:1064] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:05.565181  6521 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:17:05.568267  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71696.3us
I20260709 13:17:05.568195  8401 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:05.594149  6417 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 4 to 5, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.566165  8417 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.627432  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet 40b7cf612223418ea8fe2d636986bb3f with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:48452
I20260709 13:17:05.661108  8425 tablet_replica.cc:333] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:17:05.662297  8425 raft_consensus.cc:2243] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:05.670202  7232 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:05.672230  6521 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:05.673938  8417 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:17:05.692531  8423 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:17:05.706360  8417 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:05.709432  8425 raft_consensus.cc:2272] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:05.732165  8393 raft_consensus.cc:1064] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:05.734562  8393 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.756880  8425 ts_tablet_manager.cc:1916] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:05.818228  6589 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:05.819608  8392 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.824215  7237 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:05.825366  6667 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:05.827025  6665 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:05.828543  8392 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.835788  8392 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.857559  8417 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:05.873695  8417 raft_consensus.cc:1064] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:05.876569  8417 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:05.888334  7232 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:05.889818  8420 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:05.890904  6516 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:05.892477  8420 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.892808  8394 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.901242  6667 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.901917  7237 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.914980  8391 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.916373  6521 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.933099  7232 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.941640  6663 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:05.945621  8419 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.955533  6417 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:05.956792  6420 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.011543  8425 ts_tablet_manager.cc:1929] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:17:06.017052  8425 log.cc:1199] T 40b7cf612223418ea8fe2d636986bb3f P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/40b7cf612223418ea8fe2d636986bb3f
I20260709 13:17:06.018920  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet 40b7cf612223418ea8fe2d636986bb3f (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:06.019682  8419 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.023416  6590 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.035473  7237 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.035518  6516 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.046236  6522 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:06.053939  6590 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:06.055962  8394 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:06.056687  6663 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.055985  7237 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:06.063580  8394 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:06.071686  8393 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.076510  6415 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.075742  6663 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } attrs { replace: true } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.089558  6589 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.096138  7237 raft_consensus.cc:2955] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.109005  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d83f8b945cd641db90647da0461be4a3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:06.118517  6417 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.120527  6589 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:06.128203  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet d83f8b945cd641db90647da0461be4a3 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45428
I20260709 13:17:06.129004  6516 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:06.131129  8419 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:06.133970  7237 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:06.135710  8419 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:06.138597  8398 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.140478  8443 tablet_replica.cc:333] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:17:06.141511  8443 raft_consensus.cc:2243] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:06.142244  8443 raft_consensus.cc:2272] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:06.145780  8443 ts_tablet_manager.cc:1916] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:06.147207  7233 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.152657  6516 raft_consensus.cc:2955] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.170866  8443 ts_tablet_manager.cc:1929] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:17:06.171950  8443 log.cc:1199] T d83f8b945cd641db90647da0461be4a3 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/d83f8b945cd641db90647da0461be4a3
I20260709 13:17:06.172838  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6eb5773be2294301bdb1bbedf5765666 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:06.175261  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet d83f8b945cd641db90647da0461be4a3 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:06.178093  6589 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:06.200455  6415 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 5 to 6, VOTER 53fee9894731461a821a131be24dbbca (127.5.72.131) evicted. New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.202342  7237 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:06.204071  8419 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:06.211486  6663 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:06.218263  8420 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:06.222743  6659 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eb5773be2294301bdb1bbedf5765666 with delete_type TABLET_DATA_TOMBSTONED (TS 53fee9894731461a821a131be24dbbca not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45428
I20260709 13:17:06.225668  8443 tablet_replica.cc:333] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:17:06.228283  8443 raft_consensus.cc:2243] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:06.233268  8443 raft_consensus.cc:2272] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:06.237347  8443 ts_tablet_manager.cc:1916] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:06.242667  8419 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.260509  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 17e76729a2fc4273abedd8be9331a882 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:06.255290  6668 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.277150  6418 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.279989  7233 raft_consensus.cc:2955] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:06.288980  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 17e76729a2fc4273abedd8be9331a882 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60532
I20260709 13:17:06.278169  8443 ts_tablet_manager.cc:1929] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:17:06.294284  8443 log.cc:1199] T 6eb5773be2294301bdb1bbedf5765666 P 53fee9894731461a821a131be24dbbca: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/wals/6eb5773be2294301bdb1bbedf5765666
I20260709 13:17:06.302080  8444 tablet_replica.cc:333] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:17:06.303200  8444 raft_consensus.cc:2243] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:06.303967  8444 raft_consensus.cc:2272] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:06.306945  6405 catalog_manager.cc:5028] TS 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): tablet 6eb5773be2294301bdb1bbedf5765666 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:06.308717  8444 ts_tablet_manager.cc:1916] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:06.322073  8444 ts_tablet_manager.cc:1929] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:17:06.322425  8444 log.cc:1199] T 17e76729a2fc4273abedd8be9331a882 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/17e76729a2fc4273abedd8be9331a882
I20260709 13:17:06.323936  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 17e76729a2fc4273abedd8be9331a882 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:06.538903  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:06.577710  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:17:06.578294  6589 raft_consensus.cc:606] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:17:06.580523  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "943ce377e70a422a8376b294da2840fa"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:17:06.580987  6668 raft_consensus.cc:606] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:06.581976  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:06.583338  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:06.880200  8430 raft_consensus.cc:993] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:17:06.881314  8430 raft_consensus.cc:1081] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:17:06.883917  7237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "943ce377e70a422a8376b294da2840fa"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46666
I20260709 13:17:06.885005  7237 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.885573  7237 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.911218  7237 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:06.914350  7237 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:06.915382  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:06.916426  6517 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.917371  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:06.918043  6668 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:06.918334  6668 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:06.919600  6668 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:06.921448  6668 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.928658  6517 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:06.930137  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:06.930262  6668 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:06.931052  8448 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:06.931598  8448 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:06.932708  8448 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:06.949317  6418 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.996685  8398 raft_consensus.cc:993] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 921c423406da4f289884afca51ca198f to start an election
I20260709 13:17:06.997267  8420 raft_consensus.cc:1081] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 921c423406da4f289884afca51ca198f to start an election
I20260709 13:17:06.998751  6517 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4"
dest_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:60540
I20260709 13:17:06.999130  6517 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.999378  6517 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:07.004930  6517 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:07.006862  6517 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:07.007819  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:07.007692  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:07.008404  6590 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:07.008497  6668 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:07.008744  6590 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:07.009562  6590 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:07.011003  6590 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:07.017962  6668 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:07.018754  6590 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:07.019446  6486 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:07.020365  8449 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:07.020829  8449 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:07.021842  8449 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:07.032897  6418 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.380597  6668 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:07.382822  8448 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:07.385200  6668 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:17:07.394968  6636 consensus_peers.cc:597] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fb598c0b9b54f788ae6eefe2894e3a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.399360  6590 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:07.404268  6516 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:07.406898  8429 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:07.407905  8447 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:07.419417  6517 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:07.420271  8430 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.437862  8448 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:07.443307  6590 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.452381  6665 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:07.452944  6589 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:07.455116  8449 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.460261  8449 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.466043  6521 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:07.480495  6418 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.500985  6516 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.546540  6665 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:17:07.547643  6487 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b75fb632ebbc415092b08b38d0835144. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.548406  8258 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.560400  8392 raft_consensus.cc:2955] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.564936  6589 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:07.568070  8449 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.574833  6589 raft_consensus.cc:2955] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.571841  6668 raft_consensus.cc:2955] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.582417  6417 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.584376  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:07.584646  6521 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:07.591015  6665 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:17:07.592002  6487 consensus_peers.cc:597] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 938e1826b2ce4babab3a70e5219411a4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.592837  6595 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:07.593919  8392 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.596879  8449 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.603996  8449 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.606734  6665 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.615167  6595 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.631742  6415 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca reported cstate change: config changed from index -1 to 4, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.653477  6595 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:07.668792  6668 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:07.668853  6521 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:07.670303  8420 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.671140  8398 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:17:07.675837  6562 consensus_peers.cc:597] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Couldn't send request to peer 64b95375974b48d18c596a23c9cac75c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a72647203b143708b26072192a9c102. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.681072  8420 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.682760  6668 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.683579  6521 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.698489  6415 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index -1 to 3, NON_VOTER 64b95375974b48d18c596a23c9cac75c (127.5.72.132) added. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:07.864540  8476 ts_tablet_manager.cc:933] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:07.873931  8476 tablet_copy_client.cc:323] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.131:38019
I20260709 13:17:07.876302  6680 tablet_copy_service.cc:140] P 53fee9894731461a821a131be24dbbca: Received BeginTabletCopySession request for tablet 2fb598c0b9b54f788ae6eefe2894e3a5 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:51542)
I20260709 13:17:07.876859  6680 tablet_copy_service.cc:161] P 53fee9894731461a821a131be24dbbca: Beginning new tablet copy session on tablet 2fb598c0b9b54f788ae6eefe2894e3a5 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:51542: session id = 64b95375974b48d18c596a23c9cac75c-2fb598c0b9b54f788ae6eefe2894e3a5
I20260709 13:17:07.883121  6680 tablet_copy_source_session.cc:215] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:07.885850  8476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb598c0b9b54f788ae6eefe2894e3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:07.899273  8476 tablet_copy_client.cc:806] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:07.899852  8476 tablet_copy_client.cc:670] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:07.903342  8476 tablet_copy_client.cc:538] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:07.911988  8476 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:07.933532  8476 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:07.934604  8476 tablet_bootstrap.cc:492] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:07.935369  8476 ts_tablet_manager.cc:1403] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.024s	user 0.023s	sys 0.000s
I20260709 13:17:07.938567  8476 raft_consensus.cc:359] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:07.939451  8476 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:07.940092  8476 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:07.943069  8476 ts_tablet_manager.cc:1434] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:17:07.944809  6680 tablet_copy_service.cc:342] P 53fee9894731461a821a131be24dbbca: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-2fb598c0b9b54f788ae6eefe2894e3a5 received from {username='slave'} at 127.0.0.1:51542
I20260709 13:17:07.945276  6680 tablet_copy_service.cc:434] P 53fee9894731461a821a131be24dbbca: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-2fb598c0b9b54f788ae6eefe2894e3a5 on tablet 2fb598c0b9b54f788ae6eefe2894e3a5 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:08.092993  8476 ts_tablet_manager.cc:933] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:08.106426  8476 tablet_copy_client.cc:323] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.130:37977
I20260709 13:17:08.109376  6607 tablet_copy_service.cc:140] P 3ceee54e49d041be9f7f48a2052709c5: Received BeginTabletCopySession request for tablet 7a72647203b143708b26072192a9c102 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:34536)
I20260709 13:17:08.109965  6607 tablet_copy_service.cc:161] P 3ceee54e49d041be9f7f48a2052709c5: Beginning new tablet copy session on tablet 7a72647203b143708b26072192a9c102 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:34536: session id = 64b95375974b48d18c596a23c9cac75c-7a72647203b143708b26072192a9c102
I20260709 13:17:08.119616  6607 tablet_copy_source_session.cc:215] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:08.123224  8476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a72647203b143708b26072192a9c102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.139971  8476 tablet_copy_client.cc:806] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:08.140648  8476 tablet_copy_client.cc:670] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:08.141788  8480 ts_tablet_manager.cc:933] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:17:08.146654  8476 tablet_copy_client.cc:538] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:08.149996  8480 tablet_copy_client.cc:323] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:17:08.152119  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet b75fb632ebbc415092b08b38d0835144 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:17:08.152658  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet b75fb632ebbc415092b08b38d0835144 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-b75fb632ebbc415092b08b38d0835144
I20260709 13:17:08.162053  8476 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:08.189465  6532 tablet_copy_source_session.cc:215] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:08.205770  8480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75fb632ebbc415092b08b38d0835144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.227586  8485 ts_tablet_manager.cc:933] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Initiating tablet copy from peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121)
I20260709 13:17:08.258733  8485 tablet_copy_client.cc:323] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Beginning tablet copy session from remote peer at address 127.5.72.129:34121
I20260709 13:17:08.259397  8480 tablet_copy_client.cc:806] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:08.260030  8480 tablet_copy_client.cc:670] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:08.260401  6532 tablet_copy_service.cc:140] P 921c423406da4f289884afca51ca198f: Received BeginTabletCopySession request for tablet 938e1826b2ce4babab3a70e5219411a4 from peer 64b95375974b48d18c596a23c9cac75c ({username='slave'} at 127.0.0.1:59988)
I20260709 13:17:08.260936  6532 tablet_copy_service.cc:161] P 921c423406da4f289884afca51ca198f: Beginning new tablet copy session on tablet 938e1826b2ce4babab3a70e5219411a4 from peer 64b95375974b48d18c596a23c9cac75c at {username='slave'} at 127.0.0.1:59988: session id = 64b95375974b48d18c596a23c9cac75c-938e1826b2ce4babab3a70e5219411a4
I20260709 13:17:08.268450  6532 tablet_copy_source_session.cc:215] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:08.268510  8480 tablet_copy_client.cc:538] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:08.271947  8485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 938e1826b2ce4babab3a70e5219411a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.272137  8476 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:08.275754  8476 tablet_bootstrap.cc:492] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:08.276552  8476 ts_tablet_manager.cc:1403] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.115s	user 0.030s	sys 0.012s
I20260709 13:17:08.280244  8476 raft_consensus.cc:359] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.281143  8476 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:08.282315  8476 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.288079  8485 tablet_copy_client.cc:806] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:08.288636  8485 tablet_copy_client.cc:670] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:08.292768  8480 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:08.294476  8476 ts_tablet_manager.cc:1434] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.004s
I20260709 13:17:08.298138  8485 tablet_copy_client.cc:538] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:08.299628  6607 tablet_copy_service.cc:342] P 3ceee54e49d041be9f7f48a2052709c5: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-7a72647203b143708b26072192a9c102 received from {username='slave'} at 127.0.0.1:34536
I20260709 13:17:08.300061  6607 tablet_copy_service.cc:434] P 3ceee54e49d041be9f7f48a2052709c5: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-7a72647203b143708b26072192a9c102 on tablet 7a72647203b143708b26072192a9c102 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:08.307667  8485 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap starting.
I20260709 13:17:08.330538  8480 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:08.331553  8480 tablet_bootstrap.cc:492] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:08.332299  8480 ts_tablet_manager.cc:1403] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.040s	user 0.029s	sys 0.006s
I20260709 13:17:08.334231  8485 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:08.335027  8485 tablet_bootstrap.cc:492] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Bootstrap complete.
I20260709 13:17:08.335563  8485 ts_tablet_manager.cc:1403] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Time spent bootstrapping tablet: real 0.028s	user 0.023s	sys 0.001s
I20260709 13:17:08.335454  8480 raft_consensus.cc:359] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.336458  8480 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:08.337168  8480 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.339044  8485 raft_consensus.cc:359] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.339818  8485 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Initialized, Role: LEARNER
I20260709 13:17:08.340451  8480 ts_tablet_manager.cc:1434] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:17:08.340417  8485 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.343228  8485 ts_tablet_manager.cc:1434] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:17:08.345232  6531 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-938e1826b2ce4babab3a70e5219411a4 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:17:08.345655  6531 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-938e1826b2ce4babab3a70e5219411a4 on tablet 938e1826b2ce4babab3a70e5219411a4 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:08.346791  6532 tablet_copy_service.cc:342] P 921c423406da4f289884afca51ca198f: Request end of tablet copy session 64b95375974b48d18c596a23c9cac75c-b75fb632ebbc415092b08b38d0835144 received from {username='slave'} at 127.0.0.1:59988
I20260709 13:17:08.347168  6532 tablet_copy_service.cc:434] P 921c423406da4f289884afca51ca198f: ending tablet copy session 64b95375974b48d18c596a23c9cac75c-b75fb632ebbc415092b08b38d0835144 on tablet b75fb632ebbc415092b08b38d0835144 with peer 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:08.413450  7237 raft_consensus.cc:1217] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:08.521371  7232 raft_consensus.cc:1217] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:17:08.545560  7232 raft_consensus.cc:1217] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:17:08.664716  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:08.716012  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:08.716681  6521 raft_consensus.cc:606] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:08.718120  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:17:08.719734  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:08.767692  8392 raft_consensus.cc:993] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:08.768311  8482 raft_consensus.cc:1081] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:08.769896  6595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:17:08.770468  6595 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:08.770841  6595 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:08.776585  7232 raft_consensus.cc:1217] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:17:08.778332  6595 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.782667  6595 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:08.787774  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:08.788331  6516 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:08.788599  6516 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:08.789304  6516 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.794096  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:08.794792  6668 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:08.790843  6516 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:08.801117  6668 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:08.808388  6516 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:08.808267  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:08.809653  8398 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:08.810789  8398 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:08.811817  8398 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:08.825210  8447 raft_consensus.cc:1064] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:08.827064  6417 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 3, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:08.827792  8447 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:08.835728  7232 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 LEARNER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:08.836445  6589 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:08.837895  6516 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:08.837774  8447 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:08.839131  8491 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:08.840821  8430 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:08.874516  8491 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.878965  7232 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.879837  6516 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.883072  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48452
I20260709 13:17:08.883682  6589 raft_consensus.cc:606] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:17:08.888757  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48452
I20260709 13:17:08.889298  6595 raft_consensus.cc:606] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Received request to transfer leadership
W20260709 13:17:08.890427  6407 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for b75fb632ebbc415092b08b38d0835144 already in progress
I20260709 13:17:08.897375  6417 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f reported cstate change: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.910487  6595 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.916664  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48452
I20260709 13:17:08.917268  6589 raft_consensus.cc:606] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Received request to transfer leadership
W20260709 13:17:08.923614  6468 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:17:08.923996  6468 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for b75fb632ebbc415092b08b38d0835144 already in progress
W20260709 13:17:08.924278  6468 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for b75fb632ebbc415092b08b38d0835144 already in progress
I20260709 13:17:08.928234  6668 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:08.934504  7232 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:08.936069  8491 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:08.934537  6595 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:08.937925  8447 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:08.945533  8430 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.947175  7232 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.950542  6595 raft_consensus.cc:2955] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.956266  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2fb598c0b9b54f788ae6eefe2894e3a5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:08.968382  6417 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 3 to 4, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 1 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:08.983834  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 2fb598c0b9b54f788ae6eefe2894e3a5 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60532
I20260709 13:17:08.986258  8502 tablet_replica.cc:333] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:17:08.987347  8502 raft_consensus.cc:2243] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.988126  8502 raft_consensus.cc:2272] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.993130  8502 ts_tablet_manager.cc:1916] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:09.014972  8502 ts_tablet_manager.cc:1929] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:09.015545  8502 log.cc:1199] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/2fb598c0b9b54f788ae6eefe2894e3a5
I20260709 13:17:09.020651  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 2fb598c0b9b54f788ae6eefe2894e3a5 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:09.053488  8398 raft_consensus.cc:1064] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:09.056282  8398 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:09.062310  7232 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:09.063318  6665 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:09.063513  6516 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:17:09.064848  8398 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:09.065712  8420 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:09.067346  8420 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:09.095737  7237 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:09.095423  8420 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:09.118083  6418 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:09.116449  6665 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:09.132660  6521 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:09.150547  6595 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:09.159258  6665 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:09.160526  8398 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:09.163287  7237 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:09.166916  8420 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:09.180974  8504 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:09.189656  7232 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:09.197656  6668 raft_consensus.cc:2955] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:09.203027  6407 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7a72647203b143708b26072192a9c102 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:09.240481  8392 raft_consensus.cc:1064] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:09.243285  8392 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:09.746189  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:09.954322  8392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.012s	sys 0.066s
W20260709 13:17:09.954873  8392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.012s	sys 0.066s
W20260709 13:17:09.966646  7232 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.774s	user 0.028s	sys 0.045s
I20260709 13:17:10.166708  7237 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:10.263144  6417 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 4 to 5, VOTER 921c423406da4f289884afca51ca198f (127.5.72.129) evicted. New cstate: current_term: 2 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:10.267674  8392 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.268386  8392 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.273039  8526 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.278640  8526 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.287843  8392 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.288612  8392 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:10.352463  8541 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.353093  8541 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.355711  8538 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.356257  8538 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.362567  8541 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.363168  8541 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.363736  8541 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.367415  8541 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.365576  8538 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.368083  8538 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.368732  8538 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.372002  8538 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.405022  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:10.386128  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 95216.9us
I20260709 13:17:10.394845  8540 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:10.421528  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:10.414194  8529 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:10.426416  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:10.417608  8525 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.434027  8538 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.434783  8538 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.438592  8538 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.422184  8514 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.450575  8392 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.439677  8529 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.457285  8543 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.457914  8543 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.458370  8529 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.450205  8525 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.460388  8543 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.456089  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:10.467324  8514 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.461760  8525 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:10.474882  8514 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:10.469087  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48771.9us
I20260709 13:17:10.484226  6665 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:10.467536  8392 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:10.491093  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 96529.8us
I20260709 13:17:10.491556  8392 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:10.432576  8540 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.519279  8526 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.520053  8545 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.520426  8514 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.520751  8545 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.520957  8514 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.521430  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:10.545324  8524 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.545976  8524 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.547978  8540 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.548544  8514 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:10.554829  8392 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.556252  8392 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.557245  8536 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.557792  8536 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.558351  8392 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:10.559821  8536 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:10.559969  8533 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.560439  8519 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.560894  8531 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:10.561496  8542 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.561767  8548 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.562191  8553 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.552100  8543 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.568955  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:10.566810  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 77996.2us
I20260709 13:17:10.701100  8392 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.675613  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:10.629599  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57829.1us
I20260709 13:17:10.569283  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:10.755590  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:10.755090  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:10.670995  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:10.779949  8392 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.754696  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:10.755785  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:10.559821  8524 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:10.671553  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:10.751772  8525 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.731020  8536 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.778963  6665 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:10.840011  8535 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.841540  8516 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.697855  8526 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.844591  8544 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.662182  8548 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.675490  8553 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.752854  8545 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:10.753712  8541 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.730767  8514 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.569424  8519 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.850724  8543 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.671208  8517 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.667908  8515 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.752127  8537 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.698092  8552 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:10.699260  8550 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:10.700901  8539 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.879513  8523 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:10.880049  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 132758us
I20260709 13:17:10.618196  8533 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.618058  8531 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.630126  8542 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:10.925974  8392 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:10.999212  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 240660us
I20260709 13:17:11.075387  8392 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:11.077580  8392 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:11.073901  8545 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:10.997879  8538 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.940824  8536 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.045037  8520 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.044795  8549 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.198575  8392 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.131692  8542 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.130136  8543 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.229328  8560 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.967888  8525 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:11.110682  8554 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.075214  8514 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:11.129931  7258 debug-util.cc:398] Leaking SignalData structure 0x7b0800300060 after lost signal to thread 5413
I20260709 13:17:11.072517  8526 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:11.130116  8527 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.179046  8563 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:11.129601  8551 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.073092  8517 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.232832  8392 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.255587  8514 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.255898  8525 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.246016  8556 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.256129  8517 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.246042  8558 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:11.257655  7258 debug-util.cc:398] Leaking SignalData structure 0x7b0800300740 after lost signal to thread 6454
I20260709 13:17:11.258931  8526 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.179916  8559 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:10.968039  8516 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:11.259208  7258 debug-util.cc:398] Leaking SignalData structure 0x7b080014bc20 after lost signal to thread 6546
I20260709 13:17:11.081403  8515 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.264125  8515 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.074672  8552 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.267058  8514 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.267786  8514 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:11.269377  8552 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.270358  8514 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.273298  8516 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.090797  8523 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.278163  8523 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.209872  8549 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.257277  8558 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.083727  8539 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:10.940966  8535 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.279738  6517 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:11.198558  8538 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.081308  8541 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.283499  8557 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.198895  8545 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.294893  8521 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.279455  8540 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:11.294327  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 242306us
I20260709 13:17:11.073506  8544 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.203557  8520 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.090605  8537 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.259645  8554 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.259954  8559 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } attrs { replace: true } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.274422  8527 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.083626  8550 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.322006  6667 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
W20260709 13:17:11.280165  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 204146us
I20260709 13:17:11.285346  8542 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.299878  8571 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:11.302419  8523 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.335461  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.283107  8551 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:11.310089  7258 debug-util.cc:398] Leaking SignalData structure 0x7b0800141640 after lost signal to thread 6621
I20260709 13:17:11.336165  6663 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:11.310753  8557 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.310907  8540 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.256734  8556 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.300491  8560 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:11.340013  7258 debug-util.cc:398] Leaking SignalData structure 0x7b08000951c0 after lost signal to thread 6622
I20260709 13:17:11.384171  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:11.335824  8542 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.285676  8563 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.392132  8573 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.394964  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.396136  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.344894  8523 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.406705  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.407745  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.407308  6671 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:11.410018  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.408604  6664 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:11.390904  8536 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.393289  8543 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:11.410869  6666 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:11.387959  8519 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.399200  8531 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.322487  8521 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.407048  8533 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.420406  8543 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.433017  8545 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.432312  8548 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.465005  8516 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:11.462076  7258 debug-util.cc:398] Leaking SignalData structure 0x7b0800111f40 after lost signal to thread 6695
I20260709 13:17:11.432539  8539 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.347887  8571 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.465209  8517 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.468812  8543 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.452700  8576 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.475072  8561 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.477303  8538 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.420802  8573 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:11.485989  8528 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.486065  8518 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.480882  8516 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:11.496479  7258 debug-util.cc:398] Leaking SignalData structure 0x7b08001df000 after lost signal to thread 7261
I20260709 13:17:11.512919  8551 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.486565  8517 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.491182  8568 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.530975  8537 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.498648  8549 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.513989  8516 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.509895  8540 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.514359  8535 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.542089  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.543464  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.544653  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.545784  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.547287  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.548386  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.557763  6417 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 327) took 1361 ms. Trace:
I20260709 13:17:11.511389  8554 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.534557  8560 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.488438  8576 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.528745  8518 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.575302  8518 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.497756  8561 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.559258  8557 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.526842  8528 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.590063  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2015) took 1028 ms. Trace:
I20260709 13:17:11.590474  6589 rpcz_store.cc:276] 0709 13:17:10.561252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.565925 (+  4673us) service_pool.cc:224] Handling call
0709 13:17:10.778987 (+213062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.161635 (+382648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.161677 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:11.161734 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:11.161784 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.589906 (+428122us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400080554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:11.589934 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.590015 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879378816}
I20260709 13:17:11.593873  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.596203  7237 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 2 LEARNER]: Advancing to term 3
I20260709 13:17:11.596793  8517 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.596261  8563 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.541225  8568 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.596843  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.598219  6592 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:11.615979  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.619166  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.619810  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2019) took 1053 ms. Trace:
I20260709 13:17:11.595248  8571 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.620293  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.620271  6591 rpcz_store.cc:276] 0709 13:17:10.562456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.562767 (+   311us) service_pool.cc:224] Handling call
0709 13:17:11.613818 (+1051051us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:11.615391 (+  1573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.615553 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.615576 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:11.615616 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:11.615647 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.615699 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.615789 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6435712}
I20260709 13:17:11.621382  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.621223  8543 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.621613  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.622213  6591 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:11.620082  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.626533  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.622437  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.594691  8561 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.558214  6417 rpcz_store.cc:276] 0709 13:17:10.196820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.197403 (+   583us) service_pool.cc:224] Handling call
0709 13:17:11.557700 (+1360297us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:10.433986 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:10.434519 (+   533us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:10.434551 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:17:10.434572 (+    21us) tablet.cc:662] Decoding operations
0709 13:17:10.434925 (+   353us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:10.434994 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:10.435026 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:10.435210 (+   184us) tablet.cc:701] Row locks acquired
0709 13:17:10.435227 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:17:10.435352 (+   125us) write_op.cc:270] Start()
0709 13:17:10.435458 (+   106us) write_op.cc:276] Timestamp: P: 1783603030435311 usec, L: 0
0709 13:17:10.435480 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:17:10.436279 (+   799us) log.cc:844] Serialized 533 byte log entry
0709 13:17:11.091339 (+655060us) op_driver.cc:464] REPLICATION: finished
0709 13:17:11.382562 (+291223us) write_op.cc:301] APPLY: starting
0709 13:17:11.385443 (+  2881us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:11.390712 (+  5269us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:11.415984 (+ 25272us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:11.432920 (+ 16936us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:11.466106 (+ 33186us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:17:11.468180 (+  2074us) write_op.cc:312] APPLY: finished
0709 13:17:11.474716 (+  6536us) log.cc:844] Serialized 23 byte log entry
0709 13:17:11.480037 (+  5321us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:11.482584 (+  2547us) write_op.cc:454] Released schema lock
0709 13:17:11.483688 (+  1104us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:11.490056 (+  6368us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:11.491433 (+  1377us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3912,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":223022,"apply.run_cpu_time_us":2688,"apply.run_wall_time_us":139467,"num_ops":1,"prepare.queue_time_us":152489,"prepare.run_cpu_time_us":23241,"prepare.run_wall_time_us":53386,"raft.queue_time_us":418514,"replication_time_us":610567,"spinlock_wait_cycles":61445376,"thread_start_us":444984,"threads_started":4,"wal-append.queue_time_us":76545}]]}
I20260709 13:17:11.630815  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:11.645980  7237 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:11.646663  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:11.647630  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.596232  8573 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.649430  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.649537  6637 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:11.618011  8540 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:11.650403  6596 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:11.628582  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.652418  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.653352  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.653906  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.653476  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.654580  6596 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:11.655041  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.651295  8526 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:11.623348  8543 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.658805  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.659142  8528 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.659312  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.659235  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.659912  8576 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.657804  8573 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:11.671046  8573 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.657984  8538 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:11.666714  8528 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:11.681710  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.666982  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.669286  8541 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.643031  8557 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.686270  8557 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.688504  8533 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:11.662480  8540 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.689481  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.689785  8557 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.674850  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.627523  8568 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.677111  6517 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:11.677296  8573 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.677954  8558 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.678843  8520 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.678908  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.680066  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.691339  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 1801) took 1016 ms. Trace:
I20260709 13:17:11.692703  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2016) took 1131 ms. Trace:
I20260709 13:17:11.692353  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.682310  6521 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:11.697701  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:11.685667  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.699919  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.696450  6667 rpcz_store.cc:276] 0709 13:17:10.675155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.675332 (+   177us) service_pool.cc:224] Handling call
0709 13:17:11.691180 (+1015848us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b540023ffd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:11.691281 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246169984}
I20260709 13:17:11.697512  6590 rpcz_store.cc:276] 0709 13:17:10.561570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.563328 (+  1758us) service_pool.cc:224] Handling call
0709 13:17:10.563558 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.132479 (+568921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.132509 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:11.132549 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:11.132588 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.692485 (+559897us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54002fdfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:11.692531 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.692645 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303198976}
I20260709 13:17:11.700548  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:11.710136  8530 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.710947  8530 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.713878  8561 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:11.714170  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.714469  8561 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.715010  8561 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.715639  8530 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.698287  7228 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:11.691082  8540 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.711843  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2022) took 1132 ms. Trace:
I20260709 13:17:11.721999  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.724390  6486 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:11.691901  8544 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:11.718570  6595 rpcz_store.cc:276] 0709 13:17:10.571452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.571596 (+   144us) service_pool.cc:224] Handling call
0709 13:17:11.679940 (+1108344us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:11.701459 (+ 21519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4759296}
I20260709 13:17:11.734189  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.710225  8576 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.763912  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.769240  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.771052  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:11.756819  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:11.772459  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.701651  8553 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.752534  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.774099  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:11.778990  6667 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:11.780056  8582 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:11.780054  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.780591  8582 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.781018  8582 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:11.782528  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.782948  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.783088  6668 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:11.769459  8543 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:11.784294  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.784461  8392 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:11.785737  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.786391  6663 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:11.785821  8392 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.786742  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:11.787231  8392 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:11.784626  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.790709  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.775617  8573 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:11.775578  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.792608  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.791303  7235 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:11.786055  8543 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.794971  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:11.794988  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.788972  8515 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.795830  8566 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:11.774065  8561 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.796082  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.796190  7230 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:11.844556  6415 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 324) took 1430 ms. Trace:
I20260709 13:17:11.844724  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.841455  8563 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.845880  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.846477  8563 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.847283  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.848529  8563 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.792232  8514 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.792783  8573 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:11.849234  8514 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.849896  8573 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:11.851006  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.786471  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.798388  8527 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.798568  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.798943  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.799067  8550 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.819777  8523 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.821946  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.822741  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.823493  6663 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.827171  8392 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.828094  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.852357  8556 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.858188  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:11.858762  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.850952  6505 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a72647203b143708b26072192a9c102 with delete_type TABLET_DATA_TOMBSTONED (TS 921c423406da4f289884afca51ca198f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60532
I20260709 13:17:11.859685  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:11.852182  8528 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:17:11.836124  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.860425  8540 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:11.860543  8544 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:11.836035  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.861017  8540 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:11.861158  8544 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.861672  8540 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.861884  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.839969  8566 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.848471  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.862728  6668 raft_consensus.cc:1813] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Handling vote request from an unknown peer 921c423406da4f289884afca51ca198f
I20260709 13:17:11.862809  8566 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:11.844952  6415 rpcz_store.cc:276] 0709 13:17:10.414182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.414301 (+   119us) service_pool.cc:224] Handling call
0709 13:17:11.844497 (+1430196us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":684288}
I20260709 13:17:11.863339  6668 raft_consensus.cc:2410] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:17:11.837741  6418 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 246) took 1539 ms. Trace:
I20260709 13:17:11.834960  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.818730  8543 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:11.859336  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.774457  8576 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.860962  8528 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:11.850898  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.840817  8582 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.856268  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:11.863724  6418 rpcz_store.cc:276] 0709 13:17:10.298848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.299055 (+   207us) service_pool.cc:224] Handling call
0709 13:17:11.837680 (+1538625us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":823552}
I20260709 13:17:11.868495  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.870241  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.872385  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.845911  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.875016  8538 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.880447  8392 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.885701  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:11.798050  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.874269  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.888411  8528 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:11.789793  8559 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.917096  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:11.917925  8568 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:11.918354  8568 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.918880  8568 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.919209  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:11.875424  8542 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.898449  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:11.873343  8521 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.873452  8527 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:11.891536  8551 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.911218  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.911800  8528 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:11.914395  8561 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:11.915751  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:11.923950  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.925069  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.926106  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.929888  6663 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:11.931082  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.918905  8573 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.935861  8392 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.871443  8515 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.938915  8392 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.933050  6662 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:11.950278  8528 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.941649  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.939113  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.926821  8517 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:11.952209  8517 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:11.952647  8517 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.943419  8551 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:11.920744  8563 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:11.955209  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.931128  8542 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:11.984099  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:11.982738  6667 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:12.006585  8569 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:12.007131  8569 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:12.007337  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:11.992671  6670 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:12.007529  8569 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.921936  8538 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.023345  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.024514  8538 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.983491  8514 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.026571  8392 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:11.940356  8573 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.018883  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:12.020035  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:12.021144  8522 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:12.028247  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.029492  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.034921  8392 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:12.034950  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 170646us
I20260709 13:17:12.036144  8522 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.038476  6596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:17:12.039100  6596 raft_consensus.cc:606] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.041188  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.042312  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:17:12.009193  8542 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:12.044279  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:12.045056  6563 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.053000  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.048281  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:12.045586  8392 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:12.054719  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.057387  8532 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.057565  8514 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.058037  8514 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.058406  8514 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.057431  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.022313  8543 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:17:12.067523  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 131344us
I20260709 13:17:12.068810  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.069063  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.052228  8542 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.071970  8392 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.018225  8563 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.076040  8565 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.075956  8574 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.026086  8540 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.065058  8584 tablet_replica.cc:333] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:17:12.079006  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.070376  8532 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.066286  8541 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:12.087397  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.069134  8545 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.088560  8527 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.074831  8549 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:12.103426  6516 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:12.117964  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.110446  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.110829  8522 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.112932  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.113622  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.077069  8521 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.116335  8568 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.106765  6520 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:17:12.124166  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.124635  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.125010  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.126906  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:12.127496  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.135715  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.145624  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.089946  8574 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:12.148406  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79678.4us
I20260709 13:17:12.147287  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.105418  8553 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.149706  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.152140  7237 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.150151  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.150321  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.140515  8521 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.165773  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.183547  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.184067  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.136636  8523 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.140494  8576 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.209151  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.206252  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.198264  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.215646  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.167976  8536 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.217061  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.205543  8562 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.220065  7231 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.105846  8541 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.225112  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.136919  8549 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.229789  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:12.236159  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.234282  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.240150  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.203750  8523 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.231834  8515 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.269971  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.275458  6671 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.255759  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.278539  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.274504  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.286798  8520 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.290110  8520 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.292300  8520 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.293747  8566 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.295641  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.308000  8544 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.309139  8521 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.280797  8568 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.311061  8574 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:12.311360  8585 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.170s	user 0.003s	sys 0.000s
I20260709 13:17:12.276714  8563 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.315747  8540 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.317423  8541 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:12.317655  8582 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.244134  8558 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:17:12.320472  8558 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:12.320837  8558 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.321822  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.321652  8558 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.322974  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.323477  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.323634  8549 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:12.247885  8578 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:17:12.324028  6663 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.324682  8578 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:12.325039  8578 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.325826  8578 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.327811  8569 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.334661  6593 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:12.338560  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:12.340512  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:12.341578  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 145905us
I20260709 13:17:12.349036  8566 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.349478  6517 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:12.350207  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.354570  8540 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:12.350742  6663 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:12.343717  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 179359us
I20260709 13:17:12.360232  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.362514  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.359246  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.366240  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.288360  8543 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.367883  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.363137  6518 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:12.338711  8569 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.366947  8574 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.372130  8528 raft_consensus.cc:1064] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
W20260709 13:17:12.372534  8528 raft_consensus.cc:1068] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Unable to promote non-voter 64b95375974b48d18c596a23c9cac75c: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:17:12.376408  8523 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.282277  8542 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.383718  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:12.384649  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 145169us
I20260709 13:17:12.361466  8558 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.391121  8542 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.391621  8514 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.394532  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.364811  8578 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:12.364284  8570 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:12.370874  8392 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.395334  6519 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:12.395219  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.397342  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:12.447952  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53019.6us
W20260709 13:17:12.383674  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 88296.9us
I20260709 13:17:12.414963  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:12.415973  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.416603  8589 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:17:12.447675  8570 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.418728  8551 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.454242  8570 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.455264  8570 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:12.452587  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58118.7us
I20260709 13:17:12.457340  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.458889  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.459524  8565 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:12.423797  6486 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:12.384444  8517 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.462603  8517 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.388065  8582 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:12.463233  8582 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.463667  8582 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.464516  7229 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.466607  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.430936  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.447956  8574 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.470811  8589 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.471354  8584 raft_consensus.cc:2243] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:12.472128  8584 raft_consensus.cc:2272] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:12.399200  8566 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.445327  8556 raft_consensus.cc:993] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:17:12.472870  8556 raft_consensus.cc:1081] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:17:12.474959  7229 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.445665  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.396485  8514 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.447168  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.485050  8551 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.485181  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.486151  8551 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.486876  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.482563  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.487973  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:17:12.488224  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64483.3us
I20260709 13:17:12.488956  8392 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:12.489180  8526 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:12.489601  8526 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.489758  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.490015  8526 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:12.491528  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 43547.8us
I20260709 13:17:12.492266  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:12.483079  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.486969  6488 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.493379  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.494733  8522 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.493989  8549 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.491160  8538 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.495515  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.495796  8538 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:12.496212  8538 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.496249  8549 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.498942  8582 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.499809  7233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:17:12.505496  7233 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:12.505523  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.506098  7233 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.500967  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.501916  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.514348  6415 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:12.515450  8582 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.510704  8584 ts_tablet_manager.cc:1916] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:12.507865  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:12.511307  6488 raft_consensus.cc:2686] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:17:12.517208  8517 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.517230  6488 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.518640  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.518976  8522 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.507472  8551 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.522289  8541 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:12.522676  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.522078  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.522965  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.520524  8392 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:12.521448  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.522225  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.525312  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.526273  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:12.517010  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50466.6us
I20260709 13:17:12.526845  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.527163  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.527554  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.528095  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.524770  8392 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:12.513837  7233 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.528975  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.532179  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.533181  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.529548  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.533221  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.533388  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.492882  8578 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:17:12.531340  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.532768  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.528550  8517 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.557735  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.539438  8530 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.539161  8542 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:12.533910  8551 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.541810  8574 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.543346  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.526714  8582 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.537072  6597 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:12.547901  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.551857  6596 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:12.552742  7233 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.554442  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.513316  8526 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.571384  6669 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.555085  6588 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:12.572865  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.556427  8543 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.559461  6664 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:12.560688  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.564245  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.565243  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.565701  8590 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.565832  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.566949  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.570438  8538 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.573289  6590 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:12.572783  8526 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.575356  8578 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:12.578773  8556 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:12.598913  8578 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.598976  8556 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.599466  8578 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.599936  8556 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.600579  8538 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.601200  8556 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.575771  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.587546  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:12.590257  6669 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:12.592008  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.593559  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.598242  8543 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:12.604167  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:12.604321  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.603406  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.605697  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.606921  8517 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.607074  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:12.609186  8530 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.610008  8590 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.606582  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.605139  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.611847  8527 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.612483  8570 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.612963  8570 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.613586  8578 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.620247  8558 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.620977  8558 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.621636  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.622273  8527 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:12.622874  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.623370  8523 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.623618  8550 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:12.624017  8550 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.625023  8527 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.624856  8550 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
W20260709 13:17:12.626310  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.628697  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.629508  8551 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.630671  8517 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:12.630899  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.631201  8517 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.631664  8517 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.631841  8538 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.632930  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.634037  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:12.635200  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:12.636251  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.624029  8523 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:12.636808  6520 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:12.637765  8570 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.639834  8578 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.640599  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.640703  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.641738  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:12.639928  6520 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:12.642748  8586 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.641800  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.645056  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.646188  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.647202  8578 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.647293  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.643296  8586 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.648087  8574 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.648193  8570 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.626379  8550 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.649240  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.649405  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.630753  8590 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:12.652585  8392 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.655166  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.658547  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:12.652789  8590 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:12.660107  8590 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.653209  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:12.653405  8530 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:17:12.663252  8586 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.663582  8530 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:12.663921  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:12.665355  8569 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.665748  8569 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.665820  8527 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.666654  8569 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.663935  8530 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:12.674751  8543 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.675535  8530 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.676862  8530 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.678380  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.679540  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.681373  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.684234  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.689903  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:12.690270  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.682547  8556 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:12.682001  8578 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.682763  8527 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.691709  8514 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:17:12.691825  8556 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:12.683396  8570 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.703469  8523 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.705634  8514 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:12.706003  8514 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:12.703538  8556 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.686455  8586 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.706897  8514 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.702550  8569 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.709789  8514 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.713603  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.701838  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.702695  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.718372  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.720532  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.720743  6521 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:12.726172  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.703680  8559 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.750903  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:12.754108  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.754289  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:12.756510  8521 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.754719  8528 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:17:12.725528  8543 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:12.741477  8523 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.761353  6663 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:12.763535  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.764379  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.748515  8558 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.750267  8578 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.752925  8542 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.767273  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.767911  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.767427  7237 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:12.764156  8528 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.768810  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:12.771531  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.772886  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:12.775875  6590 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.776331  6588 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.769995  8515 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.777803  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.773391  8528 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.779013  8566 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.779387  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.779899  8528 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.780885  8540 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.781155  8528 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.781608  8540 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:12.770196  8570 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.782970  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.783179  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.783663  7229 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.783756  6597 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:12.784788  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.785284  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.786598  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.785879  8559 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.787242  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.787334  8559 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.788460  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.772224  8521 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.790098  6665 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:17:12.791846  8566 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:12.792773  8540 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:12.793527  8540 consensus_queue.cc:1243] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:12.793632  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:12.794042  8540 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:12.794453  8540 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:12.795037  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.795660  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.796427  8541 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.796931  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.797362  8541 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:12.797753  8541 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:12.798103  8541 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.753530  8538 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:17:12.799754  8392 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.801975  8392 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.804284  6595 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:12.767505  8543 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:12.802194  8574 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.794243  8517 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.896339  8517 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:12.799479  8576 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:12.898362  8574 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.900766  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.900465  8540 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.902064  8540 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.900895  8574 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.907038  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.907871  6590 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:12.909302  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:12.790124  8559 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.892585  8522 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.919139  8392 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:12.782974  8570 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:12.935588  8522 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:12.950955  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.953141  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:12.955796  6593 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:12.923097  8566 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:12.946254  8542 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:12.966413  6588 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:12.967907  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:12.969393  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:12.969096  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 201425us
I20260709 13:17:12.971104  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.970726  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:12.923285  8556 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.964694  8566 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:12.994511  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:13.004081  7237 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:12.993716  8543 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:12.963438  8536 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:17:12.995494  8517 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:13.011289  8605 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:13.001705  8549 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:13.011902  8536 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.012305  8536 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:13.012527  8584 ts_tablet_manager.cc:1929] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:17:13.013001  8584 log.cc:1199] T 7a72647203b143708b26072192a9c102 P 921c423406da4f289884afca51ca198f: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/wals/7a72647203b143708b26072192a9c102
I20260709 13:17:13.013139  8536 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.005133  8551 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:13.014328  8584 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60532 (request call id 86) took 1168 ms. Trace:
I20260709 13:17:13.014737  8584 rpcz_store.cc:276] 0709 13:17:11.846049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.850797 (+  4748us) service_pool.cc:224] Handling call
0709 13:17:11.935237 (+ 84440us) ts_tablet_manager.cc:1151] Deleting tablet 7a72647203b143708b26072192a9c102
0709 13:17:11.935296 (+    59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:17:12.585012 (+649716us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:13.003008 (+417996us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:13.014266 (+ 11258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":387911,"tablet-delete.queue_time_us":75297,"thread_start_us":39931,"threads_started":1}
I20260709 13:17:12.974522  8542 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.019330  6407 catalog_manager.cc:5028] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): tablet 7a72647203b143708b26072192a9c102 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
I20260709 13:17:13.002543  8522 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:13.023423  8596 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:17:13.025672  8542 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:13.020145  8556 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:13.026970  8566 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.028828  8566 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.030824  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:13.033573  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:13.035874  6519 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.037696  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:13.002008  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:13.040369  6664 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:13.042049  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:13.047238  6592 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:13.051307  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:13.053009  8392 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:13.000990  8581 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:13.023445  8595 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 13:17:13.051398  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58277us
I20260709 13:17:12.972548  8586 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:17:13.056464  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:13.058899  6665 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:13.052647  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:13.068718  8570 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:13.075147  6519 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:13.046013  8561 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:13.078989  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:13.088457  8586 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.089061  8586 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:13.080423  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:13.054366  8517 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:13.090961  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.092983  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:13.097620  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 126550us
I20260709 13:17:13.097726  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:13.099176  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:13.100555  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:13.100030  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:13.101446  8538 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:13.103399  8551 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:13.103996  8556 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:13.104348  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:13.104511  8556 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:13.102931  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:13.105786  8551 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:13.106204  8551 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:13.106645  8551 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.080210  8526 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:17:13.105414  8556 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:13.107260  6593 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:13.110693  8599 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:17:13.099665  8559 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:13.108381  8517 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:13.131193  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 178255us
I20260709 13:17:13.132141  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:13.131175  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:13.134873  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:13.136982  8570 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:13.121158  8559 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:13.125170  8563 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:13.149144  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:13.158337  6595 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:13.158046  8559 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.160717  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:13.163430  6593 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:13.175365  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:13.178103  6595 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:17:13.178670  6595 pending_rounds.cc:85] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 3
I20260709 13:17:13.178928  6595 pending_rounds.cc:107] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:17:13.233484  6595 raft_consensus.cc:2889] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:13.158217  8588 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:13.158151  8599 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:13.273350  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 130848us
I20260709 13:17:13.171926  8576 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:13.233266  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:13.266286  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:13.320611  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:13.321081  8581 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:13.353470  6592 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:13.360988  7235 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:13.346846  8606 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:13.347530  7231 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 3 LEARNER]: Advancing to term 4
I20260709 13:17:13.384892  6663 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:13.313623  8517 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.373934  8606 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:13.401721  8606 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.373641  8581 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:13.403393  8581 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:13.467463  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48296.9us
I20260709 13:17:13.470557  6519 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:13.470746  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52473.5us
W20260709 13:17:13.476838  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 301580us
I20260709 13:17:13.484314  8586 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.485692  8586 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.481252  7200 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:13.502151  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:13.493172  8561 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:13.512861  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:13.509080  8563 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:17:13.520862  8540 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.000s
I20260709 13:17:13.486332  8551 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:13.528661  6596 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:13.531072  8588 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:13.534143  8541 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.736s	user 0.006s	sys 0.002s
I20260709 13:17:13.540266  8530 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:13.541347  8566 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:13.553959  8541 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.546715  8530 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:13.562031  8530 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.534606  8606 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:13.563108  8530 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:13.563740  8615 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:13.565302  7231 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:17:13.565860  7231 pending_rounds.cc:85] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 3
I20260709 13:17:13.566093  7231 pending_rounds.cc:107] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:17:13.566352  7231 raft_consensus.cc:2889] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:13.572007  8606 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:13.572294  8559 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.584232  6593 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:13.584271  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:13.596760  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:13.597800  8392 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:13.604218  8578 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:13.605841  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:13.607177  8392 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:13.604274  8561 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:13.609650  8392 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.610579  8568 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:13.620055  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:13.621145  8578 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:13.621924  8622 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:13.622961  8622 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.623299  8622 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:13.624274  8622 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:13.625694  8622 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.627455  6663 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:13.629110  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:13.609097  8561 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:13.633193  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 297081us
I20260709 13:17:13.635633  6591 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:13.608042  8541 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:13.649988  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:13.650486  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:13.648826  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:13.652081  6593 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:13.632062  8603 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.655066  6520 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:13.659448  8602 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:17:13.660400  8598 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:17:13.691152  6521 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 2: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:13.695700  6588 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:13.707197  8627 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:17:13.708165  8627 consensus_queue.cc:1243] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:17:13.730425  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 86205.1us
I20260709 13:17:13.666606  8603 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:13.639339  8620 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:17:13.714959  8558 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:17:13.738641  8606 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.776084  6520 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:13.782096  8631 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:17:13.812227  8522 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.832301  8526 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.832773  8526 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:13.841490  8551 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:13.845019  8536 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.775202  8606 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:13.868280  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 121371us
I20260709 13:17:13.774044  8563 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:17:13.976365  8621 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
W20260709 13:17:13.984982  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 321815us
W20260709 13:17:13.998031  8632 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:17:13.816485  8603 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.066139  8526 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:14.070960  8637 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:17:14.079666  8606 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.092674  6588 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:17:14.099756  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.101542  6588 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:14.103183  8561 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:17:14.109992  8586 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.002s
W20260709 13:17:14.114786  8639 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:17:14.120594  8605 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:14.127995  8644 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:14.082129  8606 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.136677  6592 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:14.137907  8598 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:14.120271  8624 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:17:14.135370  8526 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.178661  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:14.181195  6592 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:14.145222  8644 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:14.193596  8530 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:14.195250  8644 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.207566  6589 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:14.215714  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:14.199749  8563 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.146538  8605 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.221984  8538 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:14.240933  8559 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.250303  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:14.250919  8603 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.251276  8605 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.254295  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:14.257203  7198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.000s	sys 0.021s
W20260709 13:17:14.258769  7198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.021s
W20260709 13:17:14.260144  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 603785us
I20260709 13:17:14.265828  6669 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:14.289105  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:14.292150  6670 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.294898  6663 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:14.231325  8563 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.263120  8605 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.321493  6669 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:14.319187  6415 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 4, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 4 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } } }
I20260709 13:17:14.327656  6415 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:14.338044  6670 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:14.302903  8608 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.322973  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:14.323418  8627 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.326087  8558 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.370105  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:14.371335  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:14.373453  8622 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.747s	user 0.008s	sys 0.005s
I20260709 13:17:14.374043  8622 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:14.374522  8622 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.376811  8622 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:14.378298  8645 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:17:14.308630  8642 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:14.379808  8642 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.381949  8642 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.382457  8558 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.384323  6667 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:14.384554  8520 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.385038  8520 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.387190  8520 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.369788  8608 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:14.384580  8558 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.388690  8608 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.391497  8607 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.391934  8607 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.393831  8607 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.394951  6664 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.395409  8574 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:14.367990  8561 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:14.397037  8561 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.399088  8644 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.362300  8569 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.400617  8569 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.335860  8605 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:14.403821  8605 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.404879  8627 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.399152  8561 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:14.412835  6667 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:14.413805  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.426440  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.426877  8558 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:17:14.427007  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:14.454185  6522 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:14.362008  8634 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.433900  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.421777  8607 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:14.425698  8569 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.427939  8556 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:14.462677  6517 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:14.461161  6594 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:14.453569  6516 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
W20260709 13:17:14.484457  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.464360  8526 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.475863  8634 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.476310  8607 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.515609  8627 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:14.485347  8556 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.638283  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.640640  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:14.533874  8628 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:14.703058  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 331187us
I20260709 13:17:14.539842  8563 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.707850  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.712730  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:14.729696  6522 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:14.729792  7237 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:14.713654  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:14.715361  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:14.522807  8526 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.641906  8558 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.740193  8558 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.741271  8607 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.742367  8558 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:14.742836  8556 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.667994  6520 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 2: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:14.731484  7232 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
W20260709 13:17:14.723598  8650 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:17:14.748764  8574 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.749325  8574 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.751988  8661 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.752625  8574 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.752549  8661 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.753213  8574 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.753728  8574 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:14.752961  8576 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:14.738121  8526 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:14.756373  8576 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.758693  8576 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:14.760526  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.761819  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.762161  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.763458  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.764061  6518 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:14.764125  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:14.762885  6516 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:14.764684  6669 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:14.769101  7230 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:14.775079  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:14.778211  6670 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:14.775059  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:14.744264  8652 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:17:14.781643  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:14.729154  8598 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:14.784025  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:14.785531  7234 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:14.791597  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:14.792280  6516 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:14.795882  8598 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.798776  8568 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:14.799238  8568 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:14.799624  8568 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.801774  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:14.802925  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:14.806063  7234 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
W20260709 13:17:14.805162  8603 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.554s	user 0.004s	sys 0.006s
I20260709 13:17:14.806110  7228 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:14.803506  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:14.770869  8661 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:14.726799  8628 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:14.807734  8574 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:14.827934  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:14.826983  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:14.912575  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 104198us
I20260709 13:17:14.990694  6522 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:14.811764  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:14.855279  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44365.3us
I20260709 13:17:14.993871  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:14.994192  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:14.994783  6669 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:14.995922  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:17:14.935997  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79372.9us
I20260709 13:17:14.809475  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:15.002599  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 190937us
I20260709 13:17:14.877574  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:15.006013  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 196617us
W20260709 13:17:15.032559  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 155075us
W20260709 13:17:15.051415  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44842.7us
W20260709 13:17:15.053063  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 55821.9us
W20260709 13:17:15.053349  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49533.3us
I20260709 13:17:15.055323  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:15.051215  8624 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.807806  8608 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:14.993651  8556 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:15.059410  8659 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:17:15.067247  8608 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.080119  8608 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.058615  8642 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:15.086011  8642 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.086374  8642 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:15.087049  8556 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:15.089120  8556 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:15.065734  8581 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:17:15.091897  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:15.093678  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:15.051388  8657 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:15.052263  8661 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:15.099709  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:15.100822  8606 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:15.102001  8606 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:15.102634  8606 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.103094  8606 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.105113  8606 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.107935  8634 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:15.109596  8628 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.097822  8657 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:15.110532  8657 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:15.111430  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 485) took 1116 ms. Trace:
I20260709 13:17:15.111986  6596 rpcz_store.cc:276] 0709 13:17:13.994845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.995795 (+   950us) service_pool.cc:224] Handling call
0709 13:17:13.996107 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:13.996453 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.996483 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:14.002268 (+  5785us) log.cc:844] Serialized 26 byte log entry
0709 13:17:14.533595 (+531327us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:14.533653 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.533739 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:15.111127 (+577388us) raft_consensus.cc:1685] finished
0709 13:17:15.111162 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:15.111351 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5384,"thread_start_us":407346,"threads_started":1,"wal-append.queue_time_us":790542}
I20260709 13:17:15.117043  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:15.118657  8514 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:15.119200  8514 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.121182  8514 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.122036  8628 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:15.122462  8628 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.052372  8598 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:15.122826  8628 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:15.091316  8624 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.126821  8613 raft_consensus.cc:1064] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: attempting to promote NON_VOTER 64b95375974b48d18c596a23c9cac75c to VOTER
I20260709 13:17:15.129079  8613 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.131637  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:15.132579  7229 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:15.135221  8569 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:15.136664  8569 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.064677  8619 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:17:15.135890  8520 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:15.135516  8556 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:15.062112  8558 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:15.153743  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:15.162789  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:15.163998  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.171481  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:15.172158  6670 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:15.176589  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:15.178913  8624 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:15.178844  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:15.166733  8558 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.180315  8558 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:15.179808  7228 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:15.184293  6668 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.192286  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:15.192982  6666 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:15.098016  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:15.197341  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:15.203187  8541 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:15.203644  8541 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.204083  8541 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:15.162356  8556 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.206308  8556 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.115371  8661 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.163264  8520 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.201776  8568 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.159746  8644 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.226608  8644 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.229012  8644 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:15.210124  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:15.177949  8642 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.198319  6668 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:15.202162  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:15.208053  8569 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.226032  8520 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.197659  8636 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:15.238332  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:15.243085  8568 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:15.209731  8661 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.209896  7231 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:15.257477  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:15.262784  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:15.275365  6662 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:17:15.230320  8556 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.239024  8619 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:15.239785  8605 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:15.289485  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:15.245965  8628 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.299126  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.246868  8569 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:15.303360  6596 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:15.305393  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:15.305552  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 300638us
I20260709 13:17:15.287148  8659 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:15.316835  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:15.309414  6593 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:15.300683  8605 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.321966  6669 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:15.289350  8619 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:15.287326  8644 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:17:15.327549  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:15.326053  6517 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:15.267382  8636 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:15.339371  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.340026  6593 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:15.341411  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.341938  6593 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
W20260709 13:17:15.343576  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 265487us
I20260709 13:17:15.290113  8556 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:15.354339  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.320917  8659 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.319705  8561 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:15.335083  8619 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.282853  8657 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.325933  8605 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:15.347324  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:15.363088  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:15.314729  8569 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.360034  8659 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:15.397938  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.391731  7234 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:15.465613  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 99024.8us
I20260709 13:17:15.483146  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.484282  6596 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:15.363498  8646 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 13:17:15.511746  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 142547us
I20260709 13:17:15.451959  8613 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:15.550294  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.594702  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:15.595842  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 248583us
I20260709 13:17:15.569792  8642 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.612916  6592 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:15.612804  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:15.606451  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:15.615479  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.616775  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:15.611920  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:15.616359  6592 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:17:15.564450  6522 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:15.568840  8613 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:15.593406  8681 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:17:15.655123  8613 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.659611  8536 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:15.658335  8541 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.660452  8586 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:15.661057  8559 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:15.661581  8588 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:15.662341  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:15.580365  8673 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:15.663811  8628 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.672214  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:15.592196  8677 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:15.692162  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:15.695438  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:15.696403  6590 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:15.699667  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:15.693974  8608 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
W20260709 13:17:15.703547  8683 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:17:15.704984  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:15.705583  8657 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:15.706473  8556 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.689826  8520 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:15.707708  8541 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.697510  8586 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:15.709620  8677 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:15.712685  8558 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.000s
I20260709 13:17:15.688360  8536 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.716025  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.146340  6592 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
W20260709 13:17:16.013124  6664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.011s	sys 0.009s
W20260709 13:17:16.147760  6664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.012s	sys 0.016s
W20260709 13:17:15.943503  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 264279us
I20260709 13:17:16.149338  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:16.149996  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:16.150460  8672 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:16.158506  8672 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.158903  8672 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:15.720916  8687 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:15.697885  8588 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.151741  8520 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.761044  8686 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:15.697525  8673 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.759181  8635 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.152429  8558 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.152984  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:15.760834  8605 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.742177  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.153821  8536 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:15.697145  8559 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:15.743237  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 43689.2us
I20260709 13:17:15.701761  8568 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.024565  8607 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:16.224562  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 71640.6us
W20260709 13:17:15.833406  8653 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:17:15.975497  8561 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:15.863135  8688 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:16.008886  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 309587us
I20260709 13:17:15.807806  6519 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:16.226166  6519 raft_consensus.cc:606] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:16.171528  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:15.768409  8569 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.217356  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:16.238363  7230 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:16.218845  7236 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 LEARNER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:16.156427  8677 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:15.760208  8660 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:16.180403  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 363) took 1326 ms. Trace:
I20260709 13:17:16.231988  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:16.234236  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:17:16.204048  8520 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.234550  6594 raft_consensus.cc:2368] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
W20260709 13:17:16.235987  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.246011  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 1885) took 1072 ms. Trace:
I20260709 13:17:16.215548  8686 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:15.721486  8608 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.202243  8687 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.223414  8635 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:16.229092  8613 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.005s	sys 0.013s
I20260709 13:17:16.212678  8673 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:16.252899  6597 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:16.267961  8588 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.334496  8701 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:16.334914  8699 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:16.335002  8701 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.335551  8699 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.335748  8701 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.335978  8699 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:16.248267  6664 rpcz_store.cc:276] 0709 13:17:14.854333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.854974 (+   641us) service_pool.cc:224] Handling call
0709 13:17:14.855342 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.855608 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:15.231585 (+375977us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.148128 (+916543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:16.148214 (+    86us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:16.148276 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.148369 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:16.180337 (+ 31968us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":452019,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":171388,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":14801,"thread_start_us":1279562,"threads_started":3,"wal-append.queue_time_us":435347}
W20260709 13:17:16.271881  8661 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.985s	user 0.010s	sys 0.000s
I20260709 13:17:16.227133  8568 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.267961  8605 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.286424  8628 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:16.374564  8613 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.005s	sys 0.020s
I20260709 13:17:16.275918  8559 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.269047  8679 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:16.235738  8607 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.276612  8586 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.235138  8688 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:16.391419  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 219787us
I20260709 13:17:16.251874  6670 rpcz_store.cc:276] 0709 13:17:15.173142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.174012 (+   870us) service_pool.cc:224] Handling call
0709 13:17:15.174257 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:15.174459 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:15.801940 (+627481us) spinlock_profiling.cc:243] Waited 981 us on lock 0x7b18003dd680. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf
0709 13:17:15.802135 (+   195us) log.cc:844] Serialized 14 byte log entry
0709 13:17:16.245587 (+443452us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:16.245653 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:16.245712 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.245810 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:16.245941 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8132,"prepare.queue_time_us":150260,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":120,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":2011,"spinlock_wait_cycles":2257152,"thread_start_us":638989,"threads_started":2}
I20260709 13:17:16.244436  8561 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:16.398483  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 164991us
I20260709 13:17:16.378599  8661 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:16.415102  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 189833us
I20260709 13:17:16.396061  8559 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:16.400647  8586 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:16.392249  8628 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.408602  8660 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:16.425011  6521 raft_consensus.cc:2368] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:16.433935  8561 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.434551  8561 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.435078  8561 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.437160  8561 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.446827  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:16.425998  8586 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.449141  8586 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:15.618527  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 132088us
I20260709 13:17:16.450565  8605 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:16.458345  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:16.459281  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.459926  6589 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:16.459844  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:16.461018  6591 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:17:16.465896  8661 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.468580  8695 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.468713  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:16.469157  8695 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.469679  8700 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:16.471364  8695 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:16.471697  8659 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.021s	user 0.007s	sys 0.003s
I20260709 13:17:16.471575  8700 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:16.472230  8659 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.472958  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:16.473531  6595 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:16.474509  8659 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:16.475782  8619 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.101s	user 0.006s	sys 0.005s
I20260709 13:17:16.476171  8619 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.477865  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.478333  8619 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.478559  6664 raft_consensus.cc:2368] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:16.430346  8558 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.478751  8605 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:16.478768  6520 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:16.479451  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.479773  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.489938  6670 raft_consensus.cc:2368] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:16.480506  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:16.480780  8708 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:17:16.482939  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.483978  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.440410  8714 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.490724  6665 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:16.491425  6671 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:16.492720  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
W20260709 13:17:16.490517  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 72445us
I20260709 13:17:16.423470  8559 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.445525  8561 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.507414  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.508205  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 1772) took 1166 ms. Trace:
I20260709 13:17:16.510365  8659 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.510921  8659 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.511672  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2101) took 1184 ms. Trace:
I20260709 13:17:16.512032  6520 rpcz_store.cc:276] 0709 13:17:15.327099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.327243 (+   144us) service_pool.cc:224] Handling call
0709 13:17:16.511523 (+1184280us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003f75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:16.511620 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2625386368}
I20260709 13:17:16.512838  8659 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.513809  8659 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.514324  8659 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.517405  8659 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.524605  8635 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.531257  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:16.532725  8716 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:16.533244  8716 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.533704  8716 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.533921  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 678) took 1479 ms. Trace:
I20260709 13:17:16.534354  7234 rpcz_store.cc:276] 0709 13:17:15.054922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.055077 (+   155us) service_pool.cc:224] Handling call
0709 13:17:16.533717 (+1478640us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54002e4ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:16.533829 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":772161408}
I20260709 13:17:16.536497  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 495) took 1164 ms. Trace:
I20260709 13:17:16.536904  6594 rpcz_store.cc:276] 0709 13:17:15.371646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.371838 (+   192us) service_pool.cc:224] Handling call
0709 13:17:16.536315 (+1164477us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54002fdad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:16.536439 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1910640384}
I20260709 13:17:16.549578  8636 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.553236  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:16.553947  6519 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:16.554227  8598 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.554790  8716 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.553267  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:16.565524  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 339190us
I20260709 13:17:16.615999  8659 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:16.616926  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:16.618659  6415 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 326) took 2623 ms. Trace:
I20260709 13:17:16.621003  8716 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.621819  8598 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.494737  8627 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.495106  8695 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.622746  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:16.624810  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.627702  6563 leader_election.cc:400] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:16.647905  6563 leader_election.cc:403] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:16.630074  8670 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:16.631584  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.635046  6561 leader_election.cc:400] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:16.636304  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.508531  6588 rpcz_store.cc:276] 0709 13:17:15.341809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.342637 (+   828us) service_pool.cc:224] Handling call
0709 13:17:16.507261 (+1164624us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:16.508147 (+   886us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16246528}
I20260709 13:17:16.652176  6561 leader_election.cc:403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:16.493728  8605 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.499086  8714 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.659950  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.636229  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:16.661027  6670 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.652259  6664 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:16.627878  6418 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:16.635962  6415 rpcz_store.cc:276] 0709 13:17:13.995352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.996506 (+  1154us) service_pool.cc:224] Handling call
0709 13:17:16.618545 (+2622039us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:14.721661 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:14.752943 (+ 31282us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:14.752983 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:17:14.753002 (+    19us) tablet.cc:662] Decoding operations
0709 13:17:14.753426 (+   424us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:14.753520 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:14.753549 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:17:14.753941 (+   392us) tablet.cc:701] Row locks acquired
0709 13:17:14.753974 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:17:14.754163 (+   189us) write_op.cc:270] Start()
0709 13:17:14.754315 (+   152us) write_op.cc:276] Timestamp: P: 1783603034754102 usec, L: 0
0709 13:17:14.754346 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:17:14.795381 (+ 41035us) log.cc:844] Serialized 975 byte log entry
0709 13:17:15.356616 (+561235us) op_driver.cc:464] REPLICATION: finished
0709 13:17:15.668469 (+311853us) write_op.cc:301] APPLY: starting
0709 13:17:15.668542 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:15.668647 (+   105us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:15.668745 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:15.694212 (+ 25467us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:15.706875 (+ 12663us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:17:15.706920 (+    45us) write_op.cc:312] APPLY: finished
0709 13:17:15.707115 (+   195us) log.cc:844] Serialized 29 byte log entry
0709 13:17:15.707263 (+   148us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:15.707348 (+    85us) write_op.cc:454] Released schema lock
0709 13:17:15.707394 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:15.707684 (+   290us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:15.707711 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":54812,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":311598,"apply.run_cpu_time_us":1853,"apply.run_wall_time_us":39211,"num_ops":2,"prepare.queue_time_us":278396,"prepare.run_cpu_time_us":13937,"prepare.run_wall_time_us":338789,"raft.queue_time_us":255844,"raft.run_cpu_time_us":74986,"raft.run_wall_time_us":362468,"replication_time_us":602137,"thread_start_us":643113,"threads_started":4,"wal-append.queue_time_us":269475}]]}
I20260709 13:17:16.651453  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.637598  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.653048  6561 leader_election.cc:400] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
W20260709 13:17:16.647563  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 493) took 1280 ms (client timeout 1626 ms). Trace:
I20260709 13:17:16.657855  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:16.658475  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:16.658957  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:16.667956  6418 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.669641  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.676815  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.677918  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:16.678385  6588 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.664579  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:17:16.664983  6593 rpcz_store.cc:269] 0709 13:17:15.366700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.366844 (+   144us) service_pool.cc:224] Handling call
0709 13:17:16.560384 (+1193540us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:16.647508 (+ 87124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3201408}
I20260709 13:17:16.679307  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.679935  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.505851  8608 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.673106  6666 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:16.680886  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.682401  6561 leader_election.cc:400] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:16.689888  6561 leader_election.cc:403] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:16.685776  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:16.686789  6668 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:16.691238  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:16.691787  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:16.692346  7237 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:16.694085  7198 leader_election.cc:400] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:16.694219  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:16.694833  7235 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:16.695240  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:16.695873  7230 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:16.635887  8586 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:16.696878  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:16.481281  8660 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.673678  8659 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.654102  8677 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:16.702579  7236 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.705538  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:16.704599  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:16.681427  8698 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.281s	user 0.000s	sys 0.000s
I20260709 13:17:16.655169  8568 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.720484  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:16.754906  6665 raft_consensus.cc:2368] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:16.755316  6518 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:16.755429  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50862.8us
I20260709 13:17:16.681774  8700 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.757033  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:16.758162  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:16.780314  6517 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:16.681314  8716 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:16.685496  8535 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.715900  8636 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.700178  8704 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:16.687894  8686 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.704002  8635 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.671098  8695 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.766217  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:16.706125  8723 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:17:16.711683  8687 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.680716  8627 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.698699  8670 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.803130  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:16.787654  8607 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.712471  8586 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.811393  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.508111  8561 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.825837  6588 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:16.831650  8688 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.833148  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:16.861081  7237 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:16.835582  8598 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.820264  8636 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.808296  8556 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:16.809619  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:16.766551  8536 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.860752  6663 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:16.838194  7229 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:16.900089  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:16.908715  8711 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:17:16.909979  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 373) took 1151 ms. Trace:
I20260709 13:17:16.910310  6667 rpcz_store.cc:276] 0709 13:17:15.758801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.761098 (+  2297us) service_pool.cc:224] Handling call
0709 13:17:15.761552 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:15.761974 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:15.762016 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.198588 (+436572us) log.cc:844] Serialized 26 byte log entry
0709 13:17:16.395180 (+196592us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:16.395220 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.397109 (+  1889us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:16.909770 (+512661us) raft_consensus.cc:1685] finished
0709 13:17:16.909791 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:16.909921 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5675,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":167166,"threads_started":1,"wal-append.queue_time_us":275721}
W20260709 13:17:16.882220  8672 consensus_meta.cc:287] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.001s
I20260709 13:17:16.884063  8695 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.820962  8700 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.818310  8535 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.837913  8687 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:16.841972  8697 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:16.818676  8716 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.865093  8679 raft_consensus.cc:2764] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:16.887708  8718 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:16.828504  8635 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.956117  6591 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:16.940697  6670 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:16.935704  8599 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:16.952674  8699 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
W20260709 13:17:16.973517  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 195501us
I20260709 13:17:16.951082  8559 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.899147  8556 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.943481  8687 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: NON_VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: true } }
I20260709 13:17:16.951568  8697 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:16.995724  8687 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:16.936671  8672 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:16.995932  8697 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.004053  8670 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:16.987591  8699 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.004895  8659 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.012275  8714 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:16.987540  8718 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.012969  8707 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.024055  8716 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.023558  8718 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:17.065973  7200 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.626s (SENT) [suppressed 1 similar messages]
W20260709 13:17:17.066071  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 328217us
I20260709 13:17:17.056458  6513 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:17.071406  8586 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.056999  6514 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:16.990672  8599 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.075194  6516 raft_consensus.cc:2368] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:17.076771  8599 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.002831  8605 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:17.083835  8520 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.001s
I20260709 13:17:17.084671  8520 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:17.092733  8709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.010s	sys 0.011s
W20260709 13:17:17.093323  8709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.010s	sys 0.011s
I20260709 13:17:17.064858  6588 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:17.102119  8569 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.103016  8569 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.104324  8699 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.107283  8569 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.111610  8636 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:17.112759  8558 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.113794  8558 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.116039  8672 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.116298  8558 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.117118  8687 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:17.118160  8687 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.121130  8687 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.122743  8688 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.122902  8700 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.123543  8688 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.125891  8688 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.097940  8707 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.014875  8689 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
W20260709 13:17:17.126842  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48586.7us
I20260709 13:17:17.130280  8710 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:17.132043  8635 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.135560  6518 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:17.137251  8535 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:17.008965  8627 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.141168  8536 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:17.141803  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50972.4us
I20260709 13:17:17.146472  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:17.148701  8704 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:17.100054  8561 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:17.153070  8627 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:17.161684  8710 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.154604  7229 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:17.149561  8704 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.169108  8717 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:17.170259  8556 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.178874  8558 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:17.186555  8558 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.184262  8717 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:17.167608  8627 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:17.160243  8569 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.122375  8687 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.195961  8717 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.223243  8520 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.224074  8627 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:17.225445  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57880.8us
I20260709 13:17:17.226933  8569 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.228590  8710 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.233418  8556 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:17.243283  8740 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:17:17.248303  6591 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.250182  8675 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.251180  8687 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.252218  8558 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:17.255718  6514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1669) took 1560 ms (client timeout 1724 ms). Trace:
I20260709 13:17:17.257473  8717 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.258989  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:17.259701  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 405) took 1028 ms. Trace:
I20260709 13:17:17.184928  8707 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.235021  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:17.272471  8673 consensus_meta.cc:287] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.898s	user 0.008s	sys 0.009s
I20260709 13:17:17.273125  8679 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:17.287752  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 511615us
W20260709 13:17:17.288393  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 222667us
I20260709 13:17:17.289673  8561 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.290560  8561 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.294698  8561 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:17.314553  8615 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:17.317281  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:17.318409  7228 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:17.318805  6486 leader_election.cc:400] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:17.319290  6486 leader_election.cc:403] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:17:17.324388  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:17.328080  8615 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:17.329564  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:17.339066  8615 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:17.339560  8615 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:17.339972  8615 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.369611  6589 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:17:17.279281  6516 rpcz_store.cc:276] 0709 13:17:16.231567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.231721 (+   154us) service_pool.cc:224] Handling call
0709 13:17:17.259375 (+1027654us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b5400354b54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:17.259568 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877967104}
I20260709 13:17:17.373131  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:17.275363  8675 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.339428  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:17.412835  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 154136us
I20260709 13:17:17.414772  8686 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:17.418820  8686 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:17.279593  6514 rpcz_store.cc:269] 0709 13:17:15.695377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.696024 (+   647us) service_pool.cc:224] Handling call
0709 13:17:17.255288 (+1559264us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54002166d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:17.255583 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3019997568}
W20260709 13:17:17.453794  6485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.000s	sys 0.053s
W20260709 13:17:17.454296  6485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.000s	sys 0.053s
W20260709 13:17:17.454957  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.019s	sys 0.034s
W20260709 13:17:17.455423  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.019s	sys 0.034s
W20260709 13:17:17.455983  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 906515us
W20260709 13:17:17.463536  8660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.026s	sys 0.027s
W20260709 13:17:17.464105  8660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.026s	sys 0.027s
W20260709 13:17:17.487428  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.370000  8556 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.419039  8679 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.367063  8673 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.434965  8644 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:17.440883  8712 raft_consensus.cc:993] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
W20260709 13:17:17.533291  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 194112us
I20260709 13:17:17.547770  8644 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:17.589553  6488 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.019s	sys 0.042s
W20260709 13:17:17.590534  6488 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.019s	sys 0.043s
W20260709 13:17:17.591193  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 830660us
I20260709 13:17:17.592756  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:17.595502  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 143524us
I20260709 13:17:17.600039  6597 raft_consensus.cc:2368] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
W20260709 13:17:17.600615  8701 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.264s	user 0.012s	sys 0.007s
I20260709 13:17:17.601495  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 508) took 1156 ms. Trace:
I20260709 13:17:17.592996  8673 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.601200  8701 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.589790  8644 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.551522  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:17.621023  6594 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:17.604738  8569 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.610982  8675 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.630249  8586 raft_consensus.cc:1081] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:17:17.635341  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:17.636212  7200 leader_election.cc:400] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:17.636611  7200 leader_election.cc:403] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:17:17.636770  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:17.625150  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:17.643350  8624 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:17.643642  6588 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:17.644812  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:17.646855  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:17.646925  8646 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:17.647717  6588 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:17.648766  8689 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:17.698887  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 100293us
I20260709 13:17:17.601986  6597 rpcz_store.cc:276] 0709 13:17:16.445206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.446521 (+  1315us) service_pool.cc:224] Handling call
0709 13:17:17.601226 (+1154705us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54003a1954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:17.601409 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2649261824}
I20260709 13:17:17.649014  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:17.750039  6591 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:17.620779  8561 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:17:17.751363  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 116198us
I20260709 13:17:17.751547  8660 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:17.755098  8673 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:17.755379  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:17.755676  8675 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:17.620738  8700 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.735955  8687 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.653473  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:17.655229  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:17.652349  8686 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.777518  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 514) took 1072 ms. Trace:
I20260709 13:17:17.657500  8620 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:17.661664  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:17.655526  8704 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.761174  8561 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.787820  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:17.656982  8628 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.774297  8673 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.796187  8673 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:17.794226  7236 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.986s	user 0.004s	sys 0.012s
I20260709 13:17:17.794610  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:17.768253  8646 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:17.770288  8675 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:17.773129  8679 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:17.807168  6563 leader_election.cc:400] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
W20260709 13:17:17.807883  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 51242.7us
I20260709 13:17:17.799779  8561 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.797705  8556 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.820048  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:17.815212  7236 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:17.816681  8686 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:17.819577  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:17.779712  6593 rpcz_store.cc:276] 0709 13:17:16.626451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.796963 (+170512us) service_pool.cc:224] Handling call
0709 13:17:16.797334 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.797557 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:17.063160 (+265603us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.619688 (+556528us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:17.623115 (+  3427us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:17.627721 (+  4606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.637641 (+  9920us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.699186 (+ 61545us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54177,"prepare.queue_time_us":20392,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":334156,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":15886,"thread_start_us":487900,"threads_started":2}
W20260709 13:17:17.830178  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 175006us
I20260709 13:17:17.831620  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:17.829172  6663 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
W20260709 13:17:17.825901  8753 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:17:17.839969  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48676.9us
I20260709 13:17:17.843060  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 745) took 1150 ms. Trace:
I20260709 13:17:17.843550  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:17.843554  7236 rpcz_store.cc:276] 0709 13:17:16.692983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.696644 (+  3661us) service_pool.cc:224] Handling call
0709 13:17:17.839165 (+1142521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.827941  8646 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.846933  7237 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:17.838671  7228 raft_consensus.cc:2368] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:17.801533  8689 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.850531  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:17.852846  6589 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:17.828181  7229 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:17.865473  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:17.822984  8700 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.869658  6663 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:17.849371  8624 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:17.835937  8746 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:17.873751  7237 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
W20260709 13:17:17.867607  8717 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.000s
W20260709 13:17:17.872124  8615 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.003s
I20260709 13:17:17.882702  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:17.914570  8675 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.884543  8615 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:17.916736  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.901225  8624 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:17.889544  8660 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:17.947364  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 77870.3us
I20260709 13:17:17.952884  8744 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:17.953683  8744 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:17.954715  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:17.957309  8563 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:17.957700  8563 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:17.958108  8563 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.959868  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2114) took 1202 ms. Trace:
I20260709 13:17:17.960023  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:17.961071  6514 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:17.965109  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:17.960342  6521 rpcz_store.cc:276] 0709 13:17:16.757061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.757171 (+   110us) service_pool.cc:224] Handling call
0709 13:17:16.757474 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.782199 (+ 24725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.782227 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.782718 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.287198 (+504480us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:17.287310 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.287438 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.959408 (+671970us) raft_consensus.cc:1685] finished
0709 13:17:17.959429 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.959631 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24306,"raft.queue_time_us":69419,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":21981,"thread_start_us":443867,"threads_started":1,"wal-append.queue_time_us":1063097}
I20260709 13:17:17.968766  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:17.977056  6589 raft_consensus.cc:2368] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
W20260709 13:17:17.984965  8677 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.192s	user 0.013s	sys 0.005s
I20260709 13:17:17.996641  8701 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:17.969416  8624 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.058039  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:17.962749  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:17.983287  8660 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.075078  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:18.073952  8750 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.185s	user 0.000s	sys 0.002s
I20260709 13:17:18.085366  8615 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.085813  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:18.092132  8689 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.097098  8744 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.100979  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.075774  8677 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:18.079749  8646 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.103719  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.113114  6521 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.115139  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.116954  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.117671  6518 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:18.100762  8624 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.118634  8700 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.132937  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.121919  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.106776  8563 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.138639  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.139487  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.139544  8526 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.140761  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.140079  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:18.142974  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.131420  6514 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:18.144420  6519 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:18.144551  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.144814  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.145102  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.145342  8628 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.145845  8563 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.146157  8735 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:17:18.140199  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.146903  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.139781  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.146517  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.152791  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:18.143980  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 263100us
I20260709 13:17:18.154749  6588 raft_consensus.cc:2368] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:18.156019  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.156693  6594 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:18.159556  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.160125  6595 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:18.161105  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.161677  6590 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.164908  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.165395  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:18.168332  6519 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:18.164650  6518 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:18.163184  8752 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:17:18.171543  6521 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:18.172166  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:18.164263  8644 consensus_meta.cc:287] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.002s
I20260709 13:17:18.175529  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.175155  6515 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:18.178201  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.173593  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.181301  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.179484  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.183243  7233 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:18.140375  8526 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.180754  7230 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:17:18.177675  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.177698  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.187981  6596 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:18.173825  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.177192  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.195423  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.196442  8689 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.198128  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.199138  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:18.199277  8762 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.199795  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.215240  6517 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:18.215456  6514 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:18.172572  8628 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.201515  6513 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.190647  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.202863  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.203219  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.203881  8689 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.205363  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.206887  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.207334  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.207576  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.211077  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.211724  8586 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:18.211829  8762 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.214607  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.237254  8744 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.237511  8755 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.238018  8744 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.240447  8744 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.228821  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.243402  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.243530  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.244172  6597 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:18.246238  8677 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.246594  6591 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.246850  8677 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.250401  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.251103  6668 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:17:18.251772  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.251853  6519 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:18.252244  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.252363  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.252867  6664 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:18.252978  6669 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:18.253197  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.251111  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.254963  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.255034  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:18.255756  6669 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.256268  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.258126  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.258749  6670 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:18.260064  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.260514  8677 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.261418  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.261915  8526 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.231101  6520 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:18.262957  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.263751  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.265465  8762 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.230468  8561 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.266894  7232 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.272214  6488 leader_election.cc:400] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:18.274125  6518 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
W20260709 13:17:18.277858  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 84672.7us
I20260709 13:17:18.224552  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.176986  8644 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.227869  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.281286  6669 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:18.228794  8599 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.230008  7230 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:17:18.230199  8628 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.303112  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.305267  6591 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:18.305760  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.301348  8599 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.306864  6521 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:18.307426  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.299916  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:18.238200  8755 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:18.322690  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50710us
I20260709 13:17:18.321522  8644 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.330196  8599 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.312386  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.331923  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.316972  6592 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:18.332839  6591 raft_consensus.cc:2368] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
W20260709 13:17:18.335485  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.331818  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:18.341461  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:18.334710  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.330772  8762 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.334952  8677 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.319588  8689 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.359643  8677 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.361685  8677 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.347630  8563 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:18.362223  8563 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.362735  8563 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.366739  8561 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:18.354004  7232 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:18.354696  8755 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.355794  8700 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.350370  8526 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.350822  8646 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:17:18.357617  8766 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:17:18.374840  8646 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.375416  8646 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.374796  6590 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:18.358170  7237 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:18.347090  8628 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.358745  8762 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.352135  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.366501  6513 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:18.375144  8526 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.377170  8628 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:18.380951  6592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2085) took 1754 ms (client timeout 2137 ms). Trace:
I20260709 13:17:18.380713  8563 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:18.381368  6592 rpcz_store.cc:269] 0709 13:17:16.634323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.680161 (+ 45838us) service_pool.cc:224] Handling call
0709 13:17:18.380769 (+1700608us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003bd8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:18.380900 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114967808}
I20260709 13:17:18.381990  8762 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.382077  8628 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.375859  6514 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:17:18.382329  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53315.2us
I20260709 13:17:18.383064  8563 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.383841  6488 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:18.384537  8563 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:18.386739  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 159483us
I20260709 13:17:18.388213  8526 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.388473  6488 leader_election.cc:400] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:18.388224  8563 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:18.388895  6488 leader_election.cc:403] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:17:18.390990  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:18.391901  8586 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:18.392457  8586 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:18.393498  8586 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.394791  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.395184  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.396343  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.396540  8628 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:18.397033  8628 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.397461  8628 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.397869  8700 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:18.371984  8660 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.398268  8700 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.398742  8700 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.398600  8660 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.400722  8660 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.403168  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.404559  8627 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.404735  6563 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.405342  8627 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.405767  8556 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:18.407845  8627 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.410454  8628 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.413920  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:18.416337  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:18.417444  6663 raft_consensus.cc:2368] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:18.404915  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.424106  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:18.426987  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.436262  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:18.434896  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.436929  6520 raft_consensus.cc:2376] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4. Re-sending same reply.
I20260709 13:17:18.438334  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.413968  8628 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:18.440982  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.422999  8563 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:18.442301  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:18.443344  6418 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 329) took 4625 ms. Trace:
I20260709 13:17:18.447088  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:17:18.461395  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 63288.6us
I20260709 13:17:18.462700  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.472843  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:18.475451  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 127996us
I20260709 13:17:18.479652  8628 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:18.480178  8628 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.480556  8628 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.479318  6417 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.460996  8660 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:18.456192  8677 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:18.488296  8677 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.488950  8677 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.445757  8689 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:18.490861  8689 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.491247  8689 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.492380  8700 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.496431  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:18.498157  6486 leader_election.cc:400] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:18.498226  8619 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:18.498974  8556 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:18.499661  8556 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:18.500039  8556 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.467566  8701 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:18.500361  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.501557  8686 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:18.501891  8686 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.501936  8627 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:18.502285  8686 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:18.502559  8627 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.503151  8627 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.499549  7200 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:18.507673  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:18.482033  6417 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.494506  8700 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.537302  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.537220  6417 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.539167  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.500372  8556 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.532404  8660 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:18.543709  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.544214  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.544430  7233 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:18.545492  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.545535  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.546178  7233 raft_consensus.cc:2376] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6. Re-sending same reply.
I20260709 13:17:18.546751  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.547843  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.548463  7234 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:18.549299  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.550091  7234 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:18.550026  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.550757  7229 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:17:18.443742  6418 rpcz_store.cc:276] 0709 13:17:13.818289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.821182 (+  2893us) service_pool.cc:224] Handling call
0709 13:17:18.443120 (+4621938us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:16.996536 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:17.019173 (+ 22637us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:17.019211 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:17:17.019230 (+    19us) tablet.cc:662] Decoding operations
0709 13:17:17.020722 (+  1492us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:17.020799 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:17.020831 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:17:17.021542 (+   711us) tablet.cc:701] Row locks acquired
0709 13:17:17.021562 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:17:17.021705 (+   143us) write_op.cc:270] Start()
0709 13:17:17.021994 (+   289us) write_op.cc:276] Timestamp: P: 1783603037021662 usec, L: 0
0709 13:17:17.022016 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:17:17.022685 (+   669us) log.cc:844] Serialized 980 byte log entry
0709 13:17:17.838070 (+815385us) op_driver.cc:464] REPLICATION: finished
0709 13:17:18.188324 (+350254us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04bcab7ef0
0709 13:17:18.408996 (+220672us) write_op.cc:301] APPLY: starting
0709 13:17:18.409142 (+   146us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:18.409317 (+   175us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:18.409351 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:18.409792 (+   441us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:18.410134 (+   342us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:17:18.410176 (+    42us) write_op.cc:312] APPLY: finished
0709 13:17:18.410520 (+   344us) log.cc:844] Serialized 29 byte log entry
0709 13:17:18.410779 (+   259us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.410871 (+    92us) write_op.cc:454] Released schema lock
0709 13:17:18.410936 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:18.411397 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.411428 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":165068,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":570490,"apply.run_cpu_time_us":2477,"apply.run_wall_time_us":2477,"num_ops":2,"prepare.queue_time_us":247118,"prepare.run_cpu_time_us":7976,"prepare.run_wall_time_us":241707,"raft.queue_time_us":562142,"raft.run_cpu_time_us":42637,"raft.run_wall_time_us":500698,"replication_time_us":815814,"spinlock_wait_cycles":13679872,"thread_start_us":975182,"threads_started":4,"wal-append.queue_time_us":4794}]]}
I20260709 13:17:18.553397  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.554069  7229 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:17:18.554680  7200 leader_election.cc:400] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:18.555619  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.557518  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.558835  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.558979  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.559597  7236 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:18.560117  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.560808  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.561587  7236 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:18.562845  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.563637  7200 leader_election.cc:400] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:18.563962  7200 leader_election.cc:403] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:17:18.563880  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.564524  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.564689  7231 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:18.532598  8762 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:18.565168  7232 raft_consensus.cc:2368] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:18.565069  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.566056  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.566681  7230 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:18.566872  8586 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.567541  8586 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.566345  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.574749  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.578486  8646 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.530833  8749 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:18.563660  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.577853  8755 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:18.578583  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:18.579579  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 106825us
I20260709 13:17:18.549105  8689 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.581872  7235 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
W20260709 13:17:18.579862  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 84282.1us
I20260709 13:17:18.585289  8701 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:18.585700  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:18.583951  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 145971us
I20260709 13:17:18.589787  6563 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:18.584335  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 83140.2us
I20260709 13:17:18.603089  7200 leader_election.cc:400] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:18.608280  8646 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.582132  8563 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:18.554522  8686 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.613333  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.618883  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.623061  8772 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:18.625387  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57540.6us
I20260709 13:17:18.626588  6588 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:18.633307  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.647805  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:18.649798  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44072.1us
I20260709 13:17:18.650717  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.652073  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:18.626027  8771 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:17:18.664121  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51164.3us
I20260709 13:17:18.665925  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:18.667668  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 80017.1us
I20260709 13:17:18.631961  8563 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.677371  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:18.654011  8775 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:18.663126  8771 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.680433  8700 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:18.703356  8773 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:18.703941  8773 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:18.704056  8635 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.705065  8635 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:18.704932  8773 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.705433  8635 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:18.708184  8779 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:18.708756  8779 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:18.709687  8779 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.710798  8628 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.713007  8628 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.711149  8689 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.706296  8635 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.715893  8677 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.699911  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:18.684136  8771 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:18.718189  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 52340.1us
I20260709 13:17:18.715548  8635 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.678741  8556 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.721200  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:18.698020  8774 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:17:18.723078  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 75369.3us
I20260709 13:17:18.692482  8778 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:18.689077  8776 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.733363  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:18.732343  8783 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:18.736013  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.744464  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.745223  7230 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:18.747011  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.729655  8774 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.729996  8762 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:18.750269  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.752143  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:18.752808  8586 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.754369  6595 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:18.755347  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.759538  7235 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:18.754415  8774 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:18.802424  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 126073us
I20260709 13:17:18.754868  8762 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.786201  8686 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:18.743917  8627 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.759312  8563 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:18.822470  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 89350.6us
I20260709 13:17:18.808071  8762 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.787055  7237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46684
I20260709 13:17:18.827914  7237 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.828405  7237 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:18.831218  8769 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:17:18.748373  8776 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.833194  8772 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:18.833640  8772 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:18.834553  8776 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.836563  8677 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.855108  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.839042  8563 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:17:18.858350  8735 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:18.858857  8735 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:18.859412  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:18.874607  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47515.2us
I20260709 13:17:18.875593  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:18.879277  8785 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.879807  8785 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.881899  8785 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:18.885751  8790 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.886960  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:18.887883  7229 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:18.873832  7237 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.889150  7236 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:18.894672  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 77144.2us
I20260709 13:17:18.895090  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:18.867283  8759 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.897847  7237 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:18.907353  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 178908us
I20260709 13:17:18.908080  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.923151  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:18.924469  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:18.892429  8784 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:18.896498  8700 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:18.931528  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:18.864521  8735 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.898682  8787 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:18.933400  8759 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.931236  8526 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:18.939247  8791 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:18.939785  8759 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:18.940048  8781 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.943557  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:18.944475  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:18.934342  8773 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:18.946497  8773 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.945932  8700 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.946910  8773 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:18.948421  7235 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:18.935817  8787 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:18.950408  8787 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.950941  8677 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:18.951380  8677 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.951725  8677 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.959182  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:18.959813  6591 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:18.961297  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.961977  8779 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:18.975418  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 151686us
I20260709 13:17:18.981014  8783 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:18.923758  8790 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:18.981905  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:18.982882  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:18.986759  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62382.2us
I20260709 13:17:18.991106  8627 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:18.991233  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:18.936090  8784 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:18.992913  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.993461  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:19.015301  6665 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.994973  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.995384  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:18.997224  8635 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:18.997885  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.998078  6595 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:18.953399  8735 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.998529  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:18.951746  8772 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:18.999212  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:19.021507  8772 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.999857  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.000108  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:19.004792  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:18.991398  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:19.006485  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:19.027026  6665 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:19.011588  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:19.017019  6670 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:19.026727  6666 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:19.016897  6671 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:19.027452  6520 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:19.027163  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.032174  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.032402  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:19.033910  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:19.034653  8775 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:19.035784  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.035310  8792 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.042762  8791 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:19.043157  8791 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:19.043519  8791 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:19.047281  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65474.3us
I20260709 13:17:19.048431  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:19.049518  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:19.028376  8787 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.035528  8627 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:19.068071  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:19.184072  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.055655  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:19.208045  6520 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 1.
I20260709 13:17:19.063287  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.180120  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:19.054818  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.058503  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:19.150635  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 116776us
W20260709 13:17:19.150900  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 116695us
I20260709 13:17:19.212430  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.050575  6562 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.058322  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.035462  8779 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:19.064224  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.056486  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:19.213371  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:19.213622  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 163105us
I20260709 13:17:19.055202  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.213912  6516 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:19.070174  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:19.214154  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.052068  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:19.214268  6516 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:19.216531  6522 raft_consensus.cc:2368] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:19.216820  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.039527  8686 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:19.223707  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.225239  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.225515  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.225847  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.226918  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:19.056011  6597 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:17:19.199667  8775 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:19.179340  6588 raft_consensus.cc:2955] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:19.254885  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:19.255147  6665 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:19.069819  8646 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.158s
I20260709 13:17:19.044232  8526 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.178s
I20260709 13:17:19.031059  8773 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.238499  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.153677  8800 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:19.058044  8556 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.203998  8627 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:19.219002  8689 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.258966  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:19.227815  8620 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:19.061934  8771 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.269554  6663 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:17:19.269721  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.264431  6666 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:19.064756  8774 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.238363  6516 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.267733  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.277405  6516 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.277981  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.222936  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:19.044741  8635 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.069547  8677 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.052363  8778 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.231402  8686 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:19.281044  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.282976  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.263340  8779 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.267135  8627 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.284425  8686 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.320165  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.297840  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.322525  6671 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:19.323155  6594 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:19.290095  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:19.266964  8800 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:19.327755  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 111030us
I20260709 13:17:19.252851  8783 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:19.340493  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.345973  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.340106  6594 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:19.337083  6589 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:19.342038  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:19.329828  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 106178us
I20260709 13:17:19.362469  6664 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:19.338523  6516 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:19.339339  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.345470  6668 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:19.299495  8787 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.332301  8635 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.323868  8791 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.377450  8791 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.284832  8795 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.282924  8619 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:19.333290  8790 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.378504  6416 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 4 to 5, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 5 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.059072  8784 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.380271  6416 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 3 to 4, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 4 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.380007  8615 raft_consensus.cc:2955] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:17:19.384647  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.297457  8744 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:19.386382  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:19.388305  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:19.286208  8778 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:19.397264  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.290208  8804 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.344488  8781 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.338274  8773 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.408435  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.400321  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.411888  6562 leader_election.cc:400] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:19.413128  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.395141  8744 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:19.399376  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53534.7us
I20260709 13:17:19.327198  8800 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.335004  8771 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.396586  8749 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:19.399502  8778 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:19.402176  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 121177us
I20260709 13:17:19.399073  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:19.398974  8804 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:19.409222  8556 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:19.439369  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.448967  6668 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:19.413759  8701 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:19.453136  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:19.427472  8744 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.456312  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.457217  6592 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:19.453971  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.462013  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.467161  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:19.468761  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55745us
W20260709 13:17:19.469101  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.418390  8809 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.474516  6667 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:19.469575  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.477174  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.440778  8749 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:19.479288  8749 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.462423  8701 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:19.481317  8701 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.455799  8619 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.482867  8714 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:19.483882  8714 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:19.484755  8714 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:19.485137  8714 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:19.486799  8660 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:19.487210  8660 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:19.488052  8660 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.489341  8660 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.484817  6665 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:19.491433  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.493664  6665 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:17:19.494092  6665 pending_rounds.cc:85] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 1
I20260709 13:17:19.494369  6665 pending_rounds.cc:107] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:17:19.494741  6665 raft_consensus.cc:2889] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:19.448109  8804 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:19.458716  8635 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.510110  6520 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:19.458197  6597 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:19.518783  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.494889  8757 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:19.531574  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.531699  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.532364  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:19.583073  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:19.542760  8784 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.544447  8821 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:19.546653  8749 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.517714  8787 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:19.553484  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:19.589071  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 70388.6us
I20260709 13:17:19.496857  8759 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.516989  6665 raft_consensus.cc:2955] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:19.486039  8714 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.463788  8586 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:17:19.495994  8556 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:19.587811  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 118318us
I20260709 13:17:19.406818  8775 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:19.494504  8701 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:19.602941  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 334237us
I20260709 13:17:19.519100  8788 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.628240  7200 leader_election.cc:400] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5.
I20260709 13:17:19.609820  6663 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:19.610543  6667 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:19.586777  8806 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.585472  8619 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:19.633524  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:19.640033  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 125120us
I20260709 13:17:19.604532  8776 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.645000  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:19.585721  8635 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.609689  8646 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:19.631590  6562 leader_election.cc:400] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:19.632416  8714 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.627538  8774 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.629825  8778 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.664263  8810 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.609404  8779 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:17:19.665702  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:19.637696  8788 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:19.708333  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.655463  8620 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:19.713210  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 82128us
I20260709 13:17:19.720115  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:19.627959  8757 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:17:19.679322  8788 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:19.722822  8778 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.723160  8783 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.734014  8783 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:19.723583  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.715448  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.724126  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.724581  8635 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.656190  8619 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.675619  8646 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.726397  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.728193  8822 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.663374  8586 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:17:19.729065  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.722939  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.723064  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:19.735515  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 80579.6us
I20260709 13:17:19.753635  7234 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:19.736727  7230 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:19.734576  8783 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:19.680866  8755 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.728624  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:19.740397  8762 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.889s	user 0.013s	sys 0.000s
W20260709 13:17:19.765527  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 157510us
I20260709 13:17:19.766603  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.766750  8823 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:17:19.774883  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.775971  8779 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.776087  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:19.777495  8819 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:17:19.750506  8556 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.778792  8809 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:19.777315  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.780540  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:19.781039  8734 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:19.781268  8785 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.781514  8734 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:19.782347  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.752708  8822 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:19.716189  8759 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:17:19.760059  8755 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:19.794853  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.796973  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:19.798280  6590 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:19.799789  6595 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:19.800346  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:19.778215  8800 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.789606  8785 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:19.789153  8779 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.782203  8774 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:19.802501  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:19.807726  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 128615us
I20260709 13:17:19.809056  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.768586  7231 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:19.778215  8619 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.808511  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.816834  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.771442  8762 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.799947  8755 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:19.812752  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.791985  8822 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:19.812417  6519 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:19.817981  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.822896  7237 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:19.823241  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.791543  8586 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:17:19.819033  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:19.823935  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.830776  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.831911  6522 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:19.824764  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:19.834905  7230 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
W20260709 13:17:19.838533  8826 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:17:19.811195  8779 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:19.842774  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:19.844970  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.846284  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.816969  8749 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.817791  8785 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:19.849990  7230 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:19.831306  8787 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.850538  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.851090  8771 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:19.851671  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.852803  8677 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.853606  8734 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.854776  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:19.855135  8792 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:19.831370  6514 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:19.821655  8646 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:19.858096  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.854803  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:19.858119  6520 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:19.870728  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:19.880759  8734 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:19.859549  7228 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:19.884440  8796 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.885617  8830 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:19.886489  8830 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:19.886929  8830 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:19.887387  8830 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.891083  8827 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 13:17:19.892055  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:19.847260  8779 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:19.899839  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:19.836534  8774 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.910310  8824 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:19.912940  6522 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:17:19.902794  8686 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.568s	user 0.007s	sys 0.003s
I20260709 13:17:19.867882  8771 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.922505  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:19.882558  8759 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:19.924072  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:19.925803  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:17:19.927558  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:19.929507  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 74755.6us
I20260709 13:17:19.945289  6594 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:19.946386  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:19.879781  8627 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.947175  6593 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:19.924072  8824 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:19.923612  8773 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:17:19.989197  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 78084.9us
I20260709 13:17:19.985872  8804 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.929327  8771 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:19.942605  8792 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:19.994516  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50765.6us
I20260709 13:17:19.943133  8791 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:19.928838  8759 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:19.999173  6662 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:20.002669  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.003763  8811 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:17:19.991453  8824 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.935233  8686 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:19.928009  8677 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:17:19.994920  8773 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:20.010957  8615 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.005s
W20260709 13:17:20.021194  8834 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:17:20.011277  8635 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:20.022300  8762 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:19.995523  8628 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:17:20.023842  8628 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:20.024358  8628 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:20.035341  6666 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.020103  8773 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.013253  8787 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:17:20.060456  8779 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.000499  8785 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.081629  8830 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:20.082512  8841 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:17:20.091373  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 295514us
W20260709 13:17:20.093799  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.095674  6668 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:20.084184  8830 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:20.084576  8627 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.086899  8586 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.111624  8586 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:20.086305  8783 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.108238  8774 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.140756  8835 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.141335  8835 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.144333  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:20.149024  8660 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.004s
I20260709 13:17:20.143585  8835 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.150592  8835 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.089782  8628 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:20.155432  8586 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:20.167021  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:20.167640  6669 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:20.166065  8660 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.169911  8660 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.136852  8804 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.178001  6669 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:20.168978  8628 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:20.179023  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 173171us
I20260709 13:17:20.181345  8795 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:20.171661  8686 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.184794  7235 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
W20260709 13:17:20.183172  8714 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.007s
W20260709 13:17:20.188974  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 196248us
I20260709 13:17:20.190258  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.191470  8830 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:20.191561  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.193053  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.195358  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:20.181708  8795 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:20.201301  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:20.201912  7200 leader_election.cc:400] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:20.202605  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:20.202970  7200 leader_election.cc:403] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:17:20.169301  8835 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:20.204120  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:20.198944  8830 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:20.183055  8804 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:17:20.209005  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:20.211138  8842 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:20.212311  8808 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:20.214053  8660 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:20.214465  8808 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:20.214743  8755 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.217327  8755 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:20.218796  7236 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:20.218948  8839 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:20.219625  8839 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:20.220146  8839 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.203711  8757 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:20.225220  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.220750  8771 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.227514  8620 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:20.227895  8620 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:20.228190  8771 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.228843  8620 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:20.229915  8620 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:20.230535  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:20.233542  6591 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:20.234930  8646 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:20.237596  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:20.238660  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:20.292976  6514 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:20.242424  8783 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.216305  8808 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.235904  8646 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:20.246333  8795 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.291180  8827 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:20.291152  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.298107  6519 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:17:20.300976  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 75833.5us
I20260709 13:17:20.310283  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:20.302603  6520 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:20.304352  7237 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.311936  7234 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:20.312774  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.314468  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:20.299744  8744 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.819s	user 0.015s	sys 0.003s
I20260709 13:17:20.324589  7237 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:20.330137  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.332249  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:20.333351  6486 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:20.335152  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.335805  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.311936  8827 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:20.337499  8827 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.300163  8646 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:20.343156  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:20.343369  6588 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:20.307268  8832 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:20.345353  6596 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:20.312050  8586 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.312884  8830 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:20.352473  6594 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.350090  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:20.350173  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:20.353578  6519 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:20.319249  8771 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:20.354518  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.339054  8795 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.351886  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:20.361594  8762 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:20.347903  8832 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:20.361800  8812 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:20.364980  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:20.381032  6589 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.359067  8830 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.369836  8563 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:20.371834  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.391621  7234 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.392182  6417 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 248) took 6079 ms. Trace:
I20260709 13:17:20.353468  8660 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:20.395300  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:20.379242  8822 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
I20260709 13:17:20.358457  8586 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.378772  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:20.349323  8778 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:20.386847  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:20.403559  6594 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:20.342245  8793 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:17:20.393524  6596 raft_consensus.cc:2410] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:20.335913  8744 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:20.380465  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50430.6us
I20260709 13:17:20.379232  8714 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:20.398801  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.411141  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:20.411976  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:20.411274  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:20.415716  6665 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:20.416380  6588 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
W20260709 13:17:20.417462  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 85305.9us
W20260709 13:17:20.418428  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 85162.2us
I20260709 13:17:20.417934  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:20.419605  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:20.444144  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.408475  8755 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:20.447166  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:20.447954  6663 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:20.448110  6513 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:20.370428  8646 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.371122  8776 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:17:20.460182  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:20.449216  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 95575.1us
I20260709 13:17:20.467150  8843 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:20.467669  8843 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:20.468070  8843 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.385905  8762 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.475230  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:20.392678  6417 rpcz_store.cc:276] 0709 13:17:14.301920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.302061 (+   141us) service_pool.cc:224] Handling call
0709 13:17:20.381405 (+6079344us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:18.700565 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:18.753805 (+ 53240us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:18.753850 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:17:18.753868 (+    18us) tablet.cc:662] Decoding operations
0709 13:17:18.763475 (+  9607us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:18.763542 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:18.763567 (+    25us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:17:18.764014 (+   447us) tablet.cc:701] Row locks acquired
0709 13:17:18.764064 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:17:18.764229 (+   165us) write_op.cc:270] Start()
0709 13:17:18.764383 (+   154us) write_op.cc:276] Timestamp: P: 1783603038764169 usec, L: 0
0709 13:17:18.764420 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:17:18.819465 (+ 55045us) log.cc:844] Serialized 1293 byte log entry
0709 13:17:18.907807 (+ 88342us) op_driver.cc:464] REPLICATION: finished
0709 13:17:18.908313 (+   506us) write_op.cc:301] APPLY: starting
0709 13:17:18.908385 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:18.908519 (+   134us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:18.908602 (+    83us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:18.909184 (+   582us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:18.909409 (+   225us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:17:18.909478 (+    69us) write_op.cc:312] APPLY: finished
0709 13:17:18.909750 (+   272us) log.cc:844] Serialized 35 byte log entry
0709 13:17:18.909956 (+   206us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.910073 (+   117us) write_op.cc:454] Released schema lock
0709 13:17:18.910136 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:18.910532 (+   396us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:18.910571 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10459,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":279,"apply.run_cpu_time_us":2169,"apply.run_wall_time_us":2168,"num_ops":3,"prepare.queue_time_us":68106,"prepare.run_cpu_time_us":4543,"prepare.run_wall_time_us":137639,"replication_time_us":143221,"thread_start_us":40148,"threads_started":1}]]}
W20260709 13:17:20.442831  8844 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.247s	user 0.011s	sys 0.001s
I20260709 13:17:20.477221  6590 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:20.420044  8853 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:17:20.482143  6594 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:20.488363  6517 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:20.458966  8781 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:20.459046  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:20.475847  8615 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:20.407295  8822 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.447584  8660 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
W20260709 13:17:20.502017  8854 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:17:20.490418  8795 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:17:20.512197  8860 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:20.492830  8788 consensus_meta.cc:287] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.745s	user 0.014s	sys 0.002s
I20260709 13:17:20.523742  8788 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.525782  8788 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.512423  8805 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 13:17:20.487201  8856 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:20.506749  8646 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:20.533128  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 55184.7us
I20260709 13:17:20.534026  7233 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:20.513583  8734 consensus_meta.cc:287] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.005s
I20260709 13:17:20.505883  8835 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:20.521881  8744 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.534188  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:20.539702  7234 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:20.507287  7229 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:20.542093  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.527491  8776 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:17:20.547909  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 128359us
I20260709 13:17:20.474834  8843 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.475353  8832 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.511051  8773 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.554723  8863 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:20.556424  8714 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:20.558455  7231 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:20.567500  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.558954  8830 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:20.505872  8759 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.528239  8839 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.643925  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:20.628958  6521 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:20.650218  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:20.549660  8776 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:20.655805  6671 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.643923  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:20.550540  8744 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:20.528070  8824 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.697965  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.701260  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:20.701476  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:20.704815  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:20.707026  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.707795  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:20.708537  6669 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:20.708611  6665 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:20.714738  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:20.629163  8779 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.718662  7230 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:20.719424  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:20.741940  6670 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.741987  7234 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:20.699996  8744 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.699733  8714 consensus_queue.cc:1243] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:20.701740  8759 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:20.626200  8835 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:17:20.720285  8677 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:20.751847  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 306427us
I20260709 13:17:20.717290  8836 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:20.717554  8863 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.749656  8744 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.715809  8781 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:17:20.758208  8714 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:20.764187  8779 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:17:20.768064  8771 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:17:20.785655  8795 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.766245  8863 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:20.897337  6514 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.005s
I20260709 13:17:20.882507  8805 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.790865  8822 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:20.922165  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2132) took 1916 ms. Trace:
I20260709 13:17:20.896585  8835 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.882344  8836 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:20.927109  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 1929) took 1456 ms. Trace:
W20260709 13:17:20.927881  8862 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 13:17:20.778707  8776 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:20.925866  6513 rpcz_store.cc:276] 0709 13:17:19.006132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.006386 (+   254us) service_pool.cc:224] Handling call
0709 13:17:20.012702 (+1006316us) spinlock_profiling.cc:243] Waited 8.32 ms on lock 0x7b5800010598. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:20.012950 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.448481 (+435531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.921877 (+473396us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b540019b4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:20.922092 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019483136}
W20260709 13:17:20.935264  6487 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.675s (SENT)
W20260709 13:17:20.940351  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.940018  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:20.790793  8714 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.951948  6594 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:20.936080  6666 rpcz_store.cc:276] 0709 13:17:19.459799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.480218 (+ 20419us) service_pool.cc:224] Handling call
0709 13:17:19.480494 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.036593 (+556099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.916281 (+879688us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54002bcd54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:20.916461 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275541248}
I20260709 13:17:20.943985  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:20.701086  8660 consensus_queue.cc:1175] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ceee54e49d041be9f7f48a2052709c5" responder_term: 4 status { last_received { term: 4 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:17:20.984582  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.984849  8686 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:20.985420  8686 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:20.986289  8734 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:20.961818  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:20.914171  8864 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:20.989177  8864 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.914510  6514 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:20.992882  6517 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:20.948784  7231 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.994637  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.996142  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.998934  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.976177  8863 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:20.978061  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:20.978742  8814 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:20.979967  8714 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.000716  6513 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:20.986706  8734 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:20.941504  8872 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.019866  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:20.898098  8701 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:20.936255  8869 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:21.027478  8869 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:20.912775  8795 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:20.950338  8835 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.019888  8824 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.064774  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 695) took 2337 ms. Trace:
I20260709 13:17:21.065697  8812 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.066201  8812 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.065330  7228 rpcz_store.cc:276] 0709 13:17:18.727690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.729943 (+  2253us) service_pool.cc:224] Handling call
0709 13:17:18.753422 (+ 23479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.884002 (+1130580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.035202 (+1151200us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54005195d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:21.064641 (+ 29439us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2502660480}
I20260709 13:17:20.942181  8805 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.068854  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:21.069640  7228 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:21.070972  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:21.071429  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 765) took 1254 ms. Trace:
I20260709 13:17:21.071599  7228 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:21.071828  7236 rpcz_store.cc:276] 0709 13:17:19.816795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.816909 (+   114us) service_pool.cc:224] Handling call
0709 13:17:19.817171 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.219179 (+402008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.071185 (+852006us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54004e0554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:21.071347 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923211136}
I20260709 13:17:21.072261  8734 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.029604  8771 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:21.032603  8839 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:20.960531  8836 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.114773  8775 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:21.115303  8775 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:21.116228  8775 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.117374  8775 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:21.110925  8779 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:17:21.120254  8779 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:21.120587  8779 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:21.121495  8779 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.030684  8872 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.125869  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 765) took 1420 ms. Trace:
I20260709 13:17:21.125460  7232 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:17:21.126987  7232 pending_rounds.cc:85] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 4
I20260709 13:17:21.127290  7232 pending_rounds.cc:107] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:17:21.127575  7232 raft_consensus.cc:2889] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:21.126297  7229 rpcz_store.cc:276] 0709 13:17:19.705923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.708007 (+  2084us) service_pool.cc:224] Handling call
0709 13:17:21.122549 (+1414542us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54007a0154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:21.125807 (+  3258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022675200}
I20260709 13:17:21.155066  8811 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:21.158145  8811 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:21.158953  8811 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.160146  8811 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:21.171864  8800 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.135123  8869 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.144263  8771 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.224900  8771 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.202262  8832 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:21.213972  8858 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:21.242388  8858 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:21.243471  8858 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.244907  8858 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:21.230894  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:21.259429  8860 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.260350  8860 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:21.289024  8845 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:21.319456  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.320483  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:21.321537  8820 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:21.328081  8866 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:21.328685  8866 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:21.337980  8866 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:21.340047  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.340590  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.340915  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.341444  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.342185  8563 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.339993  8866 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.342831  8563 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.347378  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2144) took 1513 ms. Trace:
I20260709 13:17:21.347779  6521 rpcz_store.cc:276] 0709 13:17:19.833978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.834164 (+   186us) service_pool.cc:224] Handling call
0709 13:17:21.347226 (+1513062us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b540014e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:21.347324 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535233280}
I20260709 13:17:21.348119  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 760) took 1535 ms. Trace:
I20260709 13:17:21.351169  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 545) took 1024 ms. Trace:
I20260709 13:17:21.351621  6589 rpcz_store.cc:276] 0709 13:17:20.327322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.327710 (+   388us) service_pool.cc:224] Handling call
0709 13:17:20.341173 (+ 13463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.391965 (+ 50792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.350892 (+958927us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b5400212854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:21.351097 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89120000}
I20260709 13:17:21.355618  8773 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.358273  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2153) took 1021 ms. Trace:
I20260709 13:17:21.358755  6517 rpcz_store.cc:276] 0709 13:17:20.336451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.489534 (+153083us) service_pool.cc:224] Handling call
0709 13:17:20.992518 (+502984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:20.993334 (+   816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.358039 (+364705us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54002166d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:21.358203 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121591040}
I20260709 13:17:21.363834  8842 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:21.364409  8842 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:21.365324  8842 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.366977  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:21.373566  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.391311  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:21.392197  6517 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:21.394196  8843 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.395267  8856 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:21.395717  8856 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:21.396663  8856 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.366387  8842 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.409355  8845 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.409489  8865 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.409979  8845 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.410241  8865 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.411185  8776 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:21.407315  8773 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:21.422966  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.427266  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.442154  8783 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:21.442965  8783 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:21.451267  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.511003  8851 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.511826  8851 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.515846  8804 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.516551  8804 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.521905  8795 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.523211  8586 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.525087  8872 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.525238  8835 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:21.525724  8827 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.188s	user 0.017s	sys 0.004s
I20260709 13:17:21.525943  8686 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:21.526244  8827 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.526950  8836 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.528400  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:21.531580  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:21.577302  6521 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:21.533121  8845 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.535025  8762 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:21.348451  7233 rpcz_store.cc:276] 0709 13:17:19.813003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.813126 (+   123us) service_pool.cc:224] Handling call
0709 13:17:19.813454 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:20.537924 (+724470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.347907 (+809983us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b5400259754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:21.348066 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1656725248}
I20260709 13:17:21.536774  6668 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:21.525727  8779 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:21.547398  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 300133us
I20260709 13:17:21.240391  6415 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.534303  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:21.582041  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.552299  6513 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.017s	sys 0.011s
I20260709 13:17:21.522009  8805 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:21.582571  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.585371  6520 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:21.585577  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:21.558553  6519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.006s
W20260709 13:17:21.586942  6519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.004s	sys 0.006s
I20260709 13:17:21.563468  6669 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:21.557631  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:21.575831  6488 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.005s	sys 0.006s
I20260709 13:17:21.578451  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:21.579555  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:21.588653  6515 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
W20260709 13:17:21.571712  8735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.744s	user 0.010s	sys 0.033s
W20260709 13:17:21.595458  8735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.779s	user 0.013s	sys 0.036s
I20260709 13:17:21.535025  8781 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:17:21.591498  6416 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 325) took 6944 ms. Trace:
I20260709 13:17:21.597209  6516 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:21.581835  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:21.582590  6513 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.018s	sys 0.011s
I20260709 13:17:21.582058  6415 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 2, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 2 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:21.571520  8806 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.000s	sys 0.008s
I20260709 13:17:21.549811  8734 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:21.558288  8856 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:21.414294  8773 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.590010  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:21.590304  6488 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.006s	sys 0.006s
W20260709 13:17:21.593413  8620 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.363s	user 0.015s	sys 0.032s
I20260709 13:17:21.600034  6521 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:21.595659  8677 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.589422  8836 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.606973  8865 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.607262  8880 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:17:21.608011  8800 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.577462  8875 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:17:21.610984  7232 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:21.618757  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:21.626763  8762 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:21.627133  8762 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:21.627337  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:21.627506  8762 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:21.628408  7228 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.036s
I20260709 13:17:21.628765  7228 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:21.629042  8853 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:21.629415  8853 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.629760  8853 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:21.602810  8563 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.630899  8563 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.631043  8851 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:21.631348  8563 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.631074  8800 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.632128  8827 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.619622  8677 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.620995  8836 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.596660  6416 rpcz_store.cc:276] 0709 13:17:14.647412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.647594 (+   182us) service_pool.cc:224] Handling call
0709 13:17:21.591430 (+6943836us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:19.656413 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:19.799449 (+143036us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:19.802279 (+  2830us) write_op.cc:435] Acquired schema lock
0709 13:17:19.806004 (+  3725us) tablet.cc:662] Decoding operations
0709 13:17:19.867483 (+ 61479us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:19.879979 (+ 12496us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:19.881957 (+  1978us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:17:19.891014 (+  9057us) tablet.cc:701] Row locks acquired
0709 13:17:19.891030 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:17:19.891200 (+   170us) write_op.cc:270] Start()
0709 13:17:19.891329 (+   129us) write_op.cc:276] Timestamp: P: 1783603039891145 usec, L: 0
0709 13:17:19.891353 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:17:19.975642 (+ 84289us) log.cc:844] Serialized 918 byte log entry
0709 13:17:20.175672 (+200030us) op_driver.cc:464] REPLICATION: finished
0709 13:17:20.541834 (+366162us) write_op.cc:301] APPLY: starting
0709 13:17:20.544070 (+  2236us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:20.551187 (+  7117us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:20.704207 (+153020us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:20.967752 (+263545us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:20.967988 (+   236us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:17:20.968055 (+    67us) write_op.cc:312] APPLY: finished
0709 13:17:20.968296 (+   241us) log.cc:844] Serialized 29 byte log entry
0709 13:17:20.969215 (+   919us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:20.969322 (+   107us) write_op.cc:454] Released schema lock
0709 13:17:20.969370 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:20.969797 (+   427us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:20.969840 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":458891,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":337938,"apply.run_cpu_time_us":6604,"apply.run_wall_time_us":437812,"mutex_wait_us":7868,"num_ops":2,"prepare.queue_time_us":246715,"prepare.run_cpu_time_us":62588,"prepare.run_wall_time_us":446325,"raft.queue_time_us":79964,"raft.run_cpu_time_us":37285,"raft.run_wall_time_us":273674,"replication_time_us":284173,"spinlock_wait_cycles":58240,"thread_start_us":446854,"threads_started":4,"wal-append.queue_time_us":106678}]]}
I20260709 13:17:21.584168  8771 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:21.601466  8806 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.000s	sys 0.009s
W20260709 13:17:21.638206  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48268.3us
I20260709 13:17:21.644593  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 433) took 1325 ms. Trace:
I20260709 13:17:21.645069  6516 rpcz_store.cc:276] 0709 13:17:20.319788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.320057 (+   269us) service_pool.cc:224] Handling call
0709 13:17:20.320421 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.597569 (+1277148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.644329 (+ 46760us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400044f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:21.644523 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2935879424}
W20260709 13:17:21.635905  8808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.108s	user 0.030s	sys 0.082s
I20260709 13:17:21.625245  8620 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:21.637939  8778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.041s	sys 0.071s
I20260709 13:17:21.696590  8872 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.697199  8872 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.699186  8872 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:21.702756  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62187.5us
I20260709 13:17:21.703953  8735 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:21.705215  8735 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:21.705963  8735 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:21.706287  8735 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:21.661350  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:21.710598  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 49365.1us
I20260709 13:17:21.711741  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:21.642839  8851 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:21.632951  8843 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:17:21.643090  8783 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:21.660749  8808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.030s	sys 0.083s
I20260709 13:17:21.724216  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:21.624440  8835 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:17:21.725739  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:21.655707  8881 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:21.659842  8861 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:21.709342  8778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.043s	sys 0.076s
I20260709 13:17:21.719208  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:21.741608  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:21.659629  8869 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:21.754398  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 190555us
I20260709 13:17:21.652694  8804 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:21.749987  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:21.737273  8686 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:21.753844  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 177596us
I20260709 13:17:21.718816  8864 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.741571  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:21.656198  8620 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:21.801898  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.802865  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52951.9us
I20260709 13:17:21.743492  8701 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.737257  8796 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.750751  8783 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.754830  8757 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:21.869073  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:21.869930  7233 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:21.870461  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 128958us
I20260709 13:17:21.870798  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:21.870472  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:21.872031  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:21.873620  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:21.902545  6516 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:21.900765  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:21.725873  8775 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:21.746538  8827 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.883307  7228 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 2: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:21.752606  8526 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.137s
W20260709 13:17:21.797088  7231 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.008s
I20260709 13:17:21.745294  8872 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.126s
W20260709 13:17:21.905710  8882 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:17:21.803565  8808 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:21.906569  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:21.895102  8851 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:21.797132  8788 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:21.911648  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 170079us
I20260709 13:17:21.895828  8800 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.755537  8781 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:21.924022  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.865931  8563 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.802249  8778 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:21.866257  8850 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.743379  8771 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:21.931757  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1718) took 1158 ms. Trace:
I20260709 13:17:21.891554  8620 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:21.936256  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 61482us
I20260709 13:17:21.907179  8804 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.889810  8824 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:21.866643  8812 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.044s
I20260709 13:17:21.755546  8836 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.800748  8820 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:21.908665  8840 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:21.907788  8686 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:21.927802  8864 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:21.956717  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:21.990795  7232 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:21.805259  8869 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.993034  6671 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:21.964918  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64261.9us
I20260709 13:17:21.965899  6513 rpcz_store.cc:276] 0709 13:17:20.773039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.019283 (+246244us) service_pool.cc:224] Handling call
0709 13:17:21.028569 (+  9286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.028758 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:21.600868 (+572110us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.930377 (+329509us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.931398 (+  1021us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:21.931457 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.931548 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.931689 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5064,"prepare.queue_time_us":492541,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":19847,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":11050,"thread_start_us":879484,"threads_started":2}
I20260709 13:17:21.928066  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:21.931159  7231 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:21.755445  8881 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.748906  8853 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.902864  8796 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.031730  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:21.978307  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:21.754926  8861 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:21.902628  8783 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:21.909608  8701 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:21.970505  8842 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.008s
I20260709 13:17:22.042532  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:21.991971  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.043849  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:21.965598  8875 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:21.974659  8810 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:21.994442  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.050469  7234 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:21.973953  8781 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:22.024024  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.025404  8660 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.028961  6665 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:22.025588  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:21.965879  8865 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:17:21.955796  8851 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.032841  6522 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:21.988731  8832 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:21.976109  8808 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:17:21.990777  8864 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.039497  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:21.980048  8850 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.040372  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:22.041174  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:22.096977  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.097235  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.034364  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 782) took 1086 ms. Trace:
I20260709 13:17:22.097977  7231 rpcz_store.cc:276] 0709 13:17:20.947619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.998593 (+ 50974us) service_pool.cc:224] Handling call
0709 13:17:22.030356 (+1031763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:21.966377  8788 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.098886  6596 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:22.100951  8788 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:21.928661  8827 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.043546  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.049307  6666 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:22.102367  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.102409  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.102967  8827 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.103444  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.103610  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.104204  6666 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:22.049666  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:21.982892  8812 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.106494  8812 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.020375  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.034403  8677 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.982749  8824 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.107301  6517 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:22.050963  6519 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:21.978753  8804 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.060416  6671 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:22.060262  8842 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 13:17:22.060738  8889 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:17:22.108188  8775 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:22.110061  8804 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:22.052546  8875 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.110770  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 778) took 1164 ms. Trace:
I20260709 13:17:22.111678  6593 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:22.047590  8795 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:22.061278  8781 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.111208  7236 rpcz_store.cc:276] 0709 13:17:20.946457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.103112 (+156655us) service_pool.cc:224] Handling call
0709 13:17:21.103578 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.547439 (+443861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:21.547492 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:21.578349 (+ 30857us) log.cc:844] Serialized 26 byte log entry
0709 13:17:21.751384 (+173035us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:21.751424 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.751476 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:22.110512 (+359036us) raft_consensus.cc:1685] finished
0709 13:17:22.110536 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:22.110708 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":172,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":25493,"thread_start_us":132262,"threads_started":1,"wal-append.queue_time_us":268805}
I20260709 13:17:22.114923  6593 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:22.112782  8775 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.062075  8899 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:22.116853  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:22.116900  8899 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.118983  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:22.061998  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.119735  6667 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:22.030664  8839 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:22.120687  8839 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.121737  7230 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:22.121451  6593 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:22.122851  8839 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.063197  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.123631  6664 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:22.063262  8805 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:22.124380  8805 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:22.125288  8805 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.127233  8805 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:22.065903  8744 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.285s	user 0.012s	sys 0.008s
I20260709 13:17:22.131112  8744 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.132678  6514 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:22.067857  8783 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:21.987975  8820 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.133460  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.030489  7229 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:22.137037  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.137631  6595 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:17:22.138845  8744 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.057773  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.084949  6665 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:22.093065  7233 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:22.033499  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 132598us
I20260709 13:17:21.987934  8778 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.101193  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.104424  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.096010  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:22.135301  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.139281  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:22.080328  8895 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:17:22.220841  6670 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:21.985548  8771 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.237452  8808 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:22.237816  8898 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.092s	user 0.000s	sys 0.005s
I20260709 13:17:22.237917  8808 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:22.119284  8899 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.238884  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.239264  8771 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.239394  8839 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:22.239657  6589 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:22.239854  8839 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:22.240451  6588 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:17:22.240495  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 182771us
I20260709 13:17:22.221004  6595 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
W20260709 13:17:22.241211  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 95878.7us
I20260709 13:17:22.243304  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.220209  6594 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:22.067471  8881 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.135371  8820 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.246090  8812 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:22.253053  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 113198us
I20260709 13:17:22.254174  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:22.245890  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 257867us
I20260709 13:17:22.249300  6520 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:22.255293  8812 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.256654  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:22.250483  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.236806  6597 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:22.070490  8762 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.165526  6522 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:22.112428  8875 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:22.094971  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.285538  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:22.291671  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:22.291361  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:22.096009  8864 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:22.228657  8804 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:22.246286  8677 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:22.080703  8810 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.330168  6668 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
W20260709 13:17:22.284945  8879 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:17:22.163487  8619 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.132s
I20260709 13:17:22.334200  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.227850  8800 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:17:22.289541  8744 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:22.340320  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 246484us
I20260709 13:17:22.302585  8836 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.292723  8906 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:22.355659  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:22.238899  8808 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.255808  8839 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.357652  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:22.348524  8734 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.000s
W20260709 13:17:22.343528  8878 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.405s	user 0.001s	sys 0.000s
I20260709 13:17:22.338984  8892 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:22.363416  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.247018  8686 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.294875  8904 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:17:22.292716  8842 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:17:22.374320  8812 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.374908  8810 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.368777  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.366261  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.366902  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:22.377013  6665 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:22.346081  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.311508  8881 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:17:22.343744  8744 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:22.379702  8911 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:17:22.382835  8744 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:22.365859  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 80401.5us
W20260709 13:17:22.370512  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.377534  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.347174  8900 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:22.372398  8903 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:22.372905  8762 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:22.335638  8804 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.391798  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.370064  8869 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:22.381417  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.381301  8850 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.361829  8845 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.338563  8677 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.368654  8619 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:17:22.385638  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 94337.4us
I20260709 13:17:22.383836  8701 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.367774  8865 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.391010  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.392568  6671 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:22.376554  6662 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:22.390224  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:22.390259  8894 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 13:17:22.391556  8853 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:22.383546  8734 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.395278  8810 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:22.394209  8806 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.285174  7228 raft_consensus.cc:2955] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:17:22.400746  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 112296us
W20260709 13:17:22.401636  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.401773  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:22.402973  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:22.372994  8893 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
I20260709 13:17:22.386631  8892 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.410964  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.394140  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.399119  8795 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:22.434096  6665 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:22.437744  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.403422  8783 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.408668  8851 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:22.485781  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 74119.5us
W20260709 13:17:22.435415  8860 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.175s	user 0.011s	sys 0.009s
W20260709 13:17:22.485671  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.608760  6666 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:22.613945  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:22.614396  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 212697us
I20260709 13:17:22.489823  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.485093  6670 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:22.404315  8619 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.615667  8899 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:22.443719  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.440248  8734 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.404438  8796 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.427153  8906 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:22.640846  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:22.793263  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:22.495265  8869 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:22.802515  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.737969  8916 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:22.638835  8779 consensus_meta.cc:287] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.048s	user 0.011s	sys 0.000s
I20260709 13:17:22.738996  8660 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.495460  8905 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:22.494822  8773 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.552649  8903 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:22.493302  8835 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:22.491134  8800 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:22.784339  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.495833  8913 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:22.442889  8776 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.028s	user 0.015s	sys 0.007s
I20260709 13:17:22.413582  8781 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:22.407949  8865 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:22.806463  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 168048us
I20260709 13:17:22.412797  8861 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:22.491341  8771 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.576877  8778 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.584460  8832 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:22.639431  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:22.854979  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:22.855793  7233 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:22.607228  8783 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.811481  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:22.763187  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 325480us
I20260709 13:17:22.772171  8845 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:22.826848  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 383187us
I20260709 13:17:22.597870  8795 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.637497  8677 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:22.828266  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.803736  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:22.867331  6522 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:22.809500  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:22.868001  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:22.850652  8701 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:22.852512  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:22.802841  8869 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:22.871286  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.872205  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:22.804189  8916 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.704023  8860 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.875406  7236 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:22.876237  8860 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.876864  8875 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:22.877269  8875 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:22.877635  8875 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:22.443065  8900 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:22.663403  8824 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:22.810050  6513 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:22.879223  8857 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.879716  8857 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.808142  8835 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:22.495671  8917 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.430s
I20260709 13:17:22.807827  8903 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:22.850816  8928 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.868178  6662 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
W20260709 13:17:22.814864  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 411926us
I20260709 13:17:22.616829  8812 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.813529  8734 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.816958  8781 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.804394  8660 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.879426  8900 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.807927  8619 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:22.852046  8865 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:22.823365  8856 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.206s	user 0.011s	sys 0.000s
I20260709 13:17:22.791092  8929 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:22.885080  7200 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.887s (SENT) [suppressed 16 similar messages]
I20260709 13:17:22.865991  8795 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:22.861599  8832 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.816681  8899 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:22.891038  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.859731  8861 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.889086  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:22.492755  8735 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.808758  8800 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.862666  8563 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:22.892511  8860 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:22.899382  8796 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:17:22.858541  8771 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.880776  8911 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.883460  8810 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.893991  8824 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.923421  8781 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:22.951861  8677 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.891185  8903 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:22.894301  8858 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.649s	user 0.014s	sys 0.005s
I20260709 13:17:22.888252  8851 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.867093  8845 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.851867  8913 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.704234  8892 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.583715  8806 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:22.903474  8880 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:17:22.953088  6518 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:22.924090  8928 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.954047  6522 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
W20260709 13:17:22.934340  8620 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.948s	user 0.008s	sys 0.004s
I20260709 13:17:22.961021  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.035388  6588 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:22.961643  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:22.962360  8922 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 13:17:22.924033  8919 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.967612  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:22.967516  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.870508  8701 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.038147  8701 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:23.037539  6517 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:23.040279  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:22.949621  8812 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:22.992079  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.992054  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.047751  6590 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:23.048439  6521 raft_consensus.cc:2376] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5. Re-sending same reply.
I20260709 13:17:22.996208  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:22.967133  8929 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.993810  8735 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:22.996922  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:23.002107  8924 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:23.004805  8923 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.003291  8851 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.960417  8856 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:23.035559  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:23.050467  8918 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:22.968956  8861 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.051019  8929 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.994038  8860 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.970927  8563 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:23.030064  8776 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.001050  8771 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:22.931659  8869 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.100919  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.101586  6519 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:23.103735  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 562) took 1004 ms. Trace:
I20260709 13:17:23.104568  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 413) took 1113 ms. Trace:
I20260709 13:17:23.105209  6668 rpcz_store.cc:276] 0709 13:17:21.991606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.991738 (+   132us) service_pool.cc:224] Handling call
0709 13:17:23.104390 (+1112652us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b540011cbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:23.104491 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659347712}
I20260709 13:17:23.067800  8851 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.070993  8919 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.899209  8906 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.002626  8824 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.003770  8891 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:17:23.079314  8677 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.096019  8776 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.098124  8563 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.061587  8812 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:23.102520  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.133584  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:23.072420  8866 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.726s	user 0.014s	sys 0.001s
I20260709 13:17:23.104287  6596 rpcz_store.cc:276] 0709 13:17:22.099704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.099856 (+   152us) service_pool.cc:224] Handling call
0709 13:17:22.100138 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.239470 (+139332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.239501 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.239986 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.359172 (+119186us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:22.441747 (+ 82575us) log.cc:844] Serialized 14 byte log entry
0709 13:17:22.491710 (+ 49963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.491809 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.103474 (+611665us) raft_consensus.cc:1685] finished
0709 13:17:23.103502 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.103664 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":186072,"prepare.queue_time_us":41165,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":202197,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":88047,"thread_start_us":182693,"threads_started":2,"wal-append.queue_time_us":170010}
I20260709 13:17:23.144332  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.034703  8911 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:22.995337  8835 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.046906  8620 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.157720  6589 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:23.159524  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 287358us
W20260709 13:17:23.098556  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.161994  7233 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:23.088515  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:23.216976  6520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1726) took 1345 ms (client timeout 1705 ms). Trace:
I20260709 13:17:23.217573  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:23.172802  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.134863  8836 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:23.218945  8836 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:23.178079  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:23.171690  6593 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:23.220858  8839 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:23.221252  8858 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:23.221908  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 43937us
I20260709 13:17:23.221860  8858 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.222836  8935 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:23.223271  8935 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:23.223531  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 732) took 1183 ms. Trace:
I20260709 13:17:23.147480  8783 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.224174  7230 rpcz_store.cc:276] 0709 13:17:22.039921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.049415 (+  9494us) service_pool.cc:224] Handling call
0709 13:17:23.223339 (+1173924us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5400170a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:23.223462 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38488320}
I20260709 13:17:23.225479  8843 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:23.225584  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:23.226053  8843 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:23.226487  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 788) took 1358 ms. Trace:
I20260709 13:17:23.227018  7232 rpcz_store.cc:276] 0709 13:17:21.868509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.868633 (+   124us) service_pool.cc:224] Handling call
0709 13:17:23.226353 (+1357720us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b54005195d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:23.226438 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100388224}
I20260709 13:17:23.230816  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1721) took 2213 ms. Trace:
I20260709 13:17:23.231352  6514 rpcz_store.cc:276] 0709 13:17:21.017194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.023501 (+  6307us) service_pool.cc:224] Handling call
0709 13:17:21.023829 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.133170 (+1109341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.230485 (+1097315us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003f70d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:23.230712 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2549462912}
I20260709 13:17:23.232935  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.182500  7237 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:23.090410  8856 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:23.246738  6516 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.247498  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 560) took 1148 ms. Trace:
I20260709 13:17:23.248010  6592 rpcz_store.cc:276] 0709 13:17:22.098680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.098798 (+   118us) service_pool.cc:224] Handling call
0709 13:17:22.099068 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:22.266458 (+167390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.266504 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:22.266562 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:22.266607 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.247231 (+980624us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540023cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:23.247293 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.247428 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384401152}
I20260709 13:17:23.234911  8934 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.082819  8924 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.215262  8775 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:17:23.217402  6520 rpcz_store.cc:269] 0709 13:17:21.871733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.871826 (+    93us) service_pool.cc:224] Handling call
0709 13:17:23.216825 (+1344999us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400044554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:23.216912 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799564672}
I20260709 13:17:23.219153  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:23.223599  8935 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:23.144429  8810 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.089656  8923 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.111884  8918 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:23.246317  8620 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.234268  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.252662  6519 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
W20260709 13:17:23.256369  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 371353us
I20260709 13:17:23.286706  8773 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:17:23.312357  8933 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:23.286839  8917 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:17:23.246380  8915 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.873s	user 0.003s	sys 0.000s
I20260709 13:17:23.290414  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.318331  6520 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:23.089172  8899 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.294546  8787 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:23.295639  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 789) took 1426 ms. Trace:
I20260709 13:17:23.325058  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:23.325083  7234 rpcz_store.cc:276] 0709 13:17:21.868770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.868907 (+   137us) service_pool.cc:224] Handling call
0709 13:17:21.906902 (+ 37995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:22.050941 (+144039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.295321 (+1244380us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002e4ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:23.295565 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329615616}
I20260709 13:17:23.327782  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:23.328374  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.328747  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.328997  6591 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:23.329051  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:23.313741  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:23.330722  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.331283  6591 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:23.332610  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.335441  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.336891  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.338261  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:23.339457  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 120357us
I20260709 13:17:23.340409  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:23.343451  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:23.343950  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:23.344121  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.344661  6596 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:23.298259  8934 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:23.349305  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.351402  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:23.352056  6590 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:23.353668  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.354701  6522 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:23.354965  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:23.356987  6589 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:23.355329  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:23.314090  8913 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.356936  6668 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:23.328697  8892 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.335304  8775 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:23.329265  8787 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:23.329835  8845 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.342989  8762 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.330082  8835 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.296250  8810 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.312938  8933 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.358466  8872 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:23.671734  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.785269  6522 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:23.400034  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:23.392563  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:23.787081  6662 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:23.739014  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 409974us
I20260709 13:17:23.710574  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2162) took 1745 ms. Trace:
I20260709 13:17:23.787691  6516 rpcz_store.cc:276] 0709 13:17:21.965029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.965188 (+   159us) service_pool.cc:224] Handling call
0709 13:17:21.979376 (+ 14188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:23.247159 (+1267783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.710227 (+463068us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54003f75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:23.710488 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2914147072}
I20260709 13:17:23.392328  8563 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:23.371649  8917 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:23.354903  8620 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:23.798389  8918 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.799768  8945 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:23.801609  8946 raft_consensus.cc:476] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:17:23.370186  8858 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.372479  8836 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:23.810562  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 496840us
W20260709 13:17:23.713002  8942 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.371s	user 0.000s	sys 0.002s
I20260709 13:17:23.372097  8800 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.836483  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.840502  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:23.841976  6522 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:23.837456  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:23.842609  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:23.399477  8897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.015s	sys 0.048s
I20260709 13:17:23.843114  6516 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:23.709338  8860 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:23.712071  8864 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.380s	user 0.008s	sys 0.003s
I20260709 13:17:23.848673  8934 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.849509  8934 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:23.851486  8924 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.850418  8934 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:23.853150  8934 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:23.854491  8934 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.528800  8677 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.470693  8812 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.398834  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:23.808414  8615 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:23.773752  8861 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.824121  8620 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.804311  8563 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:23.808027  8919 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.831532  8872 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:23.871595  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:23.858762  8897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.051s	user 0.023s	sys 0.064s
I20260709 13:17:23.844063  8832 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.803608  8762 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.472678  8787 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.836501  8945 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:23.384418  8936 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.605s
W20260709 13:17:23.976186  6634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.044s	sys 0.114s
W20260709 13:17:24.047451  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 258746us
W20260709 13:17:24.133623  6634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.044s	sys 0.126s
W20260709 13:17:23.965009  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.934387  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2167) took 1498 ms. Trace:
I20260709 13:17:24.135597  6521 rpcz_store.cc:276] 0709 13:17:22.435758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.436142 (+   384us) service_pool.cc:224] Handling call
0709 13:17:23.934179 (+1498037us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540014e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:23.934310 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562112384}
W20260709 13:17:24.044229  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.068s	sys 0.095s
W20260709 13:17:24.114130  7228 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.314s	user 0.008s	sys 0.009s
W20260709 13:17:23.978365  8735 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.911s	user 0.009s	sys 0.004s
W20260709 13:17:24.021687  6636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.007s	sys 0.020s
I20260709 13:17:23.898118  6522 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:24.180023  6636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.013s	sys 0.023s
I20260709 13:17:23.873025  8856 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.921895  8824 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.180389  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1730) took 1308 ms. Trace:
I20260709 13:17:24.041067  8941 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:24.180946  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 336369us
W20260709 13:17:24.180826  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.072s	sys 0.107s
I20260709 13:17:23.930688  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 800) took 1289 ms. Trace:
I20260709 13:17:23.861867  8928 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.865741  8808 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:23.879944  8586 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:23.860531  8615 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:23.973057  8686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.011s	sys 0.028s
I20260709 13:17:23.875368  8563 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.183459  8827 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:24.191469  8911 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:23.902411  8923 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.873338  8857 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:23.928851  8843 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:24.193544  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 1945) took 2159 ms. Trace:
I20260709 13:17:24.231140  6663 rpcz_store.cc:276] 0709 13:17:22.034103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.034237 (+   134us) service_pool.cc:224] Handling call
0709 13:17:22.034550 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:22.221381 (+186831us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:22.325880 (+104499us) log.cc:844] Serialized 14 byte log entry
0709 13:17:22.379639 (+ 53759us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:22.379779 (+   140us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:22.379854 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.193314 (+1813460us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540011d5d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:24.193361 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.193466 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":147864,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":19981,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12134,"spinlock_wait_cycles":429168512,"thread_start_us":128990,"threads_started":2,"wal-append.queue_time_us":158198,"wal-append.run_cpu_time_us":2355,"wal-append.run_wall_time_us":1247319}
W20260709 13:17:24.194629  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.195123  8842 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:24.234431  8842 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:24.235404  8842 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.235903  8933 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:24.236435  8842 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:24.198026  8935 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.879s	user 0.005s	sys 0.002s
W20260709 13:17:23.930109  8660 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.004s	user 0.012s	sys 0.009s
I20260709 13:17:24.201357  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 797) took 1560 ms. Trace:
I20260709 13:17:24.246526  7232 rpcz_store.cc:276] 0709 13:17:22.640883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.227958 (+587075us) service_pool.cc:224] Handling call
0709 13:17:24.200406 (+972448us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:24.200676 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.200914 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:24.200956 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:24.201014 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:24.201062 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.201138 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.201285 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54081408}
I20260709 13:17:24.192938  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1739) took 1315 ms. Trace:
I20260709 13:17:24.190129  6522 rpcz_store.cc:276] 0709 13:17:22.872143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.857094 (+984951us) service_pool.cc:224] Handling call
0709 13:17:23.897916 (+ 40822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.180114 (+282198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.180333 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:24.210381  7233 rpcz_store.cc:276] 0709 13:17:22.641722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.888820 (+247098us) service_pool.cc:224] Handling call
0709 13:17:23.930400 (+1041580us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400684cd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:23.930582 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461518208}
W20260709 13:17:24.210603  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 859210us
I20260709 13:17:24.248517  6521 rpcz_store.cc:276] 0709 13:17:22.877517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.189190 (+1311673us) service_pool.cc:224] Handling call
0709 13:17:24.192468 (+  3278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.192647 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:24.192671 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:24.192709 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:24.192738 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.192793 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.192887 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:24.250357  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.198967  8850 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.859555  8917 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:24.209534  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:24.215514  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:24.255466  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46020.9us
I20260709 13:17:24.257078  6488 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:24.163583  8946 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:24.185663  8744 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.802s	user 0.014s	sys 0.011s
I20260709 13:17:23.859579  8864 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.227574  8945 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.212324  8856 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.919466  8804 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.214282  8586 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:24.227628  8619 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.274s	user 0.012s	sys 0.000s
I20260709 13:17:24.308463  8619 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.308796  8586 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.309912  8586 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:24.310948  8619 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.311703  8619 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:24.312731  8875 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.435s	user 0.009s	sys 0.015s
I20260709 13:17:23.865154  8620 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:24.191092  8735 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.189316  8913 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:24.227953  8762 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:24.232887  8926 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.239s	user 0.000s	sys 0.008s
I20260709 13:17:24.235787  8776 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.313527  8913 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.196727  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1734) took 1322 ms. Trace:
I20260709 13:17:24.314604  6519 rpcz_store.cc:276] 0709 13:17:22.874686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.210702 (+336016us) service_pool.cc:224] Handling call
0709 13:17:24.196578 (+985876us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5400045954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:24.196674 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78845184}
I20260709 13:17:24.315892  8913 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.316893  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1740) took 1164 ms. Trace:
I20260709 13:17:24.258553  8660 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.313540  8735 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.317400  6519 rpcz_store.cc:276] 0709 13:17:23.152743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.315894 (+1163151us) service_pool.cc:224] Handling call
0709 13:17:24.316311 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.316516 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:24.316542 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:24.316587 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:24.316631 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.316701 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.316826 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:24.197533  7237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 735) took 2140 ms (client timeout 2238 ms). Trace:
I20260709 13:17:24.214812  8936 consensus_queue.cc:1243] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:17:24.318787  7237 rpcz_store.cc:269] 0709 13:17:22.057281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.097651 (+ 40370us) service_pool.cc:224] Handling call
0709 13:17:22.793074 (+695423us) spinlock_profiling.cc:243] Waited 69.4 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:24.197359 (+1404285us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54006cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:24.197452 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034721408}
I20260709 13:17:24.243096  8810 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.213930  8845 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:24.224287  8686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.025s	sys 0.044s
I20260709 13:17:24.322310  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.259207  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.322836  8735 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.222998  8941 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.323588  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.323593  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.264346  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1847) took 1113 ms. Trace:
I20260709 13:17:24.266285  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.324760  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:24.265281  7236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 793) took 1625 ms (client timeout 2155 ms). Trace:
W20260709 13:17:24.265470  6513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2166) took 2272 ms (client timeout 1873 ms). Trace:
I20260709 13:17:24.265591  7234 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:24.306532  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46385.7us
W20260709 13:17:24.325734  7236 rpcz_store.cc:269] 0709 13:17:22.639716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.640601 (+   885us) service_pool.cc:224] Handling call
0709 13:17:24.265018 (+1624417us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b5400576a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:24.265205 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170051328}
W20260709 13:17:24.325835  6513 rpcz_store.cc:269] 0709 13:17:21.993332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.020962 (+ 27630us) service_pool.cc:224] Handling call
0709 13:17:24.265169 (+2244207us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540007dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:24.265334 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1759999360}
I20260709 13:17:24.299156  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:24.217410  8858 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.334214  8941 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.189711  8872 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:24.224155  7229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.017s	sys 0.031s
I20260709 13:17:24.325416  7237 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:17:24.243927  8827 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.324710  6588 rpcz_store.cc:276] 0709 13:17:23.151799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.151977 (+   178us) service_pool.cc:224] Handling call
0709 13:17:23.152373 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:23.333455 (+181082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.333491 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:23.333537 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:23.333568 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.264009 (+930441us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400093dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:24.264080 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.264265 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415980800}
I20260709 13:17:24.333899  8858 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.245030  8835 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:24.332983  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.339779  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.258776  8857 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:24.260807  8850 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:24.341571  8795 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.375s	user 0.001s	sys 0.013s
I20260709 13:17:24.264789  8805 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:24.325122  7232 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:24.241361  8806 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:24.352557  6515 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:24.358412  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:24.354820  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 220360us
I20260709 13:17:24.261408  8933 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:24.386653  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.421657  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:24.367522  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.332964  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.387822  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.254668  8935 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.410029  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.383373  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:24.423143  6671 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:24.301437  8744 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.421468  6666 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:24.387437  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:24.417313  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 89773us
I20260709 13:17:24.388274  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:24.370544  8839 consensus_meta.cc:287] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.145s	user 0.007s	sys 0.009s
I20260709 13:17:24.305135  8946 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.429152  7237 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:24.429698  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.430014  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:24.369236  8905 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.560s	user 0.015s	sys 0.009s
W20260709 13:17:24.346580  8701 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.308s	user 0.000s	sys 0.018s
I20260709 13:17:24.416079  7236 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:17:24.425736  6669 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:24.369719  7229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.017s	sys 0.032s
I20260709 13:17:24.429637  7233 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:24.261332  8917 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.348752  8660 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.378469  8866 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.444844  8701 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.454639  8935 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.455598  8701 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.456401  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:24.457765  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:24.443265  8905 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:24.371357  8908 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:24.436937  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:24.444192  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:24.420070  7198 connection.cc:256] RPC call timeout handler was delayed by -1.0888636219999626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25579999999999997s for a response
I20260709 13:17:24.387056  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.463146  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:24.324536  8845 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.464859  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.423455  8805 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:24.423802  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 728) took 2380 ms. Trace:
I20260709 13:17:24.320541  8856 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.349884  8563 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:24.473073  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.484189  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:24.465377  6517 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:24.356259  8875 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.477061  6595 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:24.477990  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:24.479676  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.440093  8941 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:24.442368  8806 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:24.484242  6668 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:24.485641  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:24.512185  6670 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:24.474165  8853 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:24.488509  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.441111  7231 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:24.437959  8795 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:24.472079  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 43840.1us
I20260709 13:17:24.495087  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.495772  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.491992  7232 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:24.515250  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.428277  8835 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.515252  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.521340  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.476938  8810 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:24.532599  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.533542  6669 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:24.537559  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.538913  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:24.534456  6665 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:24.532668  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.536077  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.484927  8839 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.541196  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.541363  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.542199  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.487174  8948 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:24.551527  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.490051  8933 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:24.560024  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.511237  8941 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.526113  8762 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:24.545640  6671 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:24.558063  6521 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:24.477340  8905 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:17:24.567545  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:24.522557  8806 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.549976  8795 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.548658  8947 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:24.536829  8810 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.567842  6522 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:24.558918  8933 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:24.545295  8945 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:24.550604  8906 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:24.572140  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:24.572810  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 313372us
I20260709 13:17:24.550415  8701 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:24.478724  8908 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:24.550928  7231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 734) took 2495 ms (client timeout 3305 ms). Trace:
I20260709 13:17:24.550904  8948 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:24.558570  8919 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:24.544870  8804 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:24.564117  8941 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:24.579823  6518 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 449) took 3012 ms (client timeout 1508 ms). Trace:
I20260709 13:17:24.540452  8853 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.565862  8762 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:24.614987  8762 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:24.616073  8845 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.617005  8853 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:24.613281  6518 rpcz_store.cc:269] 0709 13:17:21.567707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.567905 (+   198us) service_pool.cc:224] Handling call
0709 13:17:24.579666 (+3011761us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400355054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:24.579761 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3143065344}
I20260709 13:17:24.618289  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:24.613952  8804 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.619199  8804 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:24.619459  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:24.620549  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:24.621074  6518 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:24.577086  8810 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:24.583709  8906 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:24.624137  8906 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:24.581133  8945 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.625912  8945 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:24.556797  8839 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.626742  8947 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.480960  7228 rpcz_store.cc:276] 0709 13:17:22.038691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.042595 (+  3904us) service_pool.cc:224] Handling call
0709 13:17:22.219409 (+176814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:22.253895 (+ 34486us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:24.179355 (+1925460us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.354233 (+174878us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:24.364516 (+ 10283us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:24.368302 (+  3786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.376538 (+  8236us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.419299 (+ 42761us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124026,"raft.queue_time_us":91658,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":6870,"thread_start_us":76158,"threads_started":1,"wal-append.queue_time_us":375048}
I20260709 13:17:24.628378  8839 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.613294  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:24.629189  8947 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.562577  8805 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.629664  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.630290  8805 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:24.630296  7228 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:24.631640  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.632444  8905 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:24.632814  8905 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.633148  8905 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:24.637357  8906 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.585335  8912 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:24.638990  8912 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:24.639575  8906 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.639909  8912 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.640857  6667 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:24.573670  8806 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:24.577162  8864 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.583520  8701 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.656729  8701 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:24.571132  8620 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.584496  8899 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:24.654973  6637 connection.cc:256] RPC call timeout handler was delayed by -1.1386586829999032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
I20260709 13:17:24.614079  8919 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.613376  8948 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:24.587105  8781 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.558s	user 0.009s	sys 0.027s
I20260709 13:17:24.582279  8933 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:24.665418  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 174379us
I20260709 13:17:24.667824  6519 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:17:24.641095  8924 consensus_queue.cc:1175] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ceee54e49d041be9f7f48a2052709c5" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
I20260709 13:17:24.626371  8804 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:24.658000  8929 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.521s	user 0.019s	sys 0.003s
I20260709 13:17:24.657704  8912 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:24.672911  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:24.675422  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 799) took 2034 ms. Trace:
W20260709 13:17:24.671058  8944 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:17:24.678104  7234 rpcz_store.cc:276] 0709 13:17:22.641447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.328250 (+686803us) service_pool.cc:224] Handling call
0709 13:17:24.265384 (+937134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.351985 (+ 86601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.675230 (+323245us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400576a50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:24.675368 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099215616}
I20260709 13:17:24.681253  8880 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:24.682072  8961 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:24.682518  8961 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:24.682595  8778 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:24.683028  8961 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:24.683193  8778 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.685292  8778 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.685559  8959 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.681808  8880 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.664726  8899 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:24.668318  8935 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.668928  8919 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:24.689121  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:24.691762  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.690119  6664 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:24.672825  8805 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.681149  8908 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:24.702320  8851 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.596s	user 0.009s	sys 0.013s
I20260709 13:17:24.706676  8935 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.701247  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.702920  8875 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:24.739378  8734 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.815s	user 0.022s	sys 0.008s
I20260709 13:17:24.699380  8880 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:24.673242  8781 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:24.754683  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.702711  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 794) took 2062 ms. Trace:
I20260709 13:17:24.673532  8827 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.695572  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:24.757635  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 62127.2us
I20260709 13:17:24.758278  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:24.700690  8917 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.703991  8916 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.671489  8946 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:24.703493  8856 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.699509  8913 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:24.707079  8899 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.662206  8905 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:24.584830  7231 rpcz_store.cc:269] 0709 13:17:22.051841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.107390 (+ 55549us) service_pool.cc:224] Handling call
0709 13:17:23.225987 (+1118597us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:24.339619 (+1113632us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5ce9c 00007f04b5f5ca5b 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:24.547425 (+207806us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317964032}
I20260709 13:17:24.756222  7229 rpcz_store.cc:276] 0709 13:17:22.640008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.173481 (+533473us) service_pool.cc:224] Handling call
0709 13:17:23.275963 (+102482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:23.336764 (+ 60801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.346142 (+  9378us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:23.529513 (+183371us) log.cc:844] Serialized 26 byte log entry
0709 13:17:24.459396 (+929883us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:24.667494 (+208098us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.677067 (+  9573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.687529 (+ 10462us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:24.694080 (+  6551us) raft_consensus.cc:1685] finished
0709 13:17:24.694099 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.702633 (+  8534us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":273610,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":6849,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":21231,"thread_start_us":1063909,"threads_started":2,"wal-append.queue_time_us":820432}
I20260709 13:17:24.694505  8959 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:17:24.746869  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.746919  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.781560  7237 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:24.692695  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.750790  7234 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:24.701671  8810 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.707198  8850 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:24.744935  8956 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:17:24.743551  8775 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.688340  8929 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.759135  6563 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:24.762934  6670 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:24.764808  6667 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:17:24.749637  8805 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:24.800915  8865 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.845s	user 0.015s	sys 0.019s
I20260709 13:17:24.772866  8620 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.801379  8865 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.773829  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.749459  8851 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.802701  7228 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:24.803254  6514 raft_consensus.cc:2368] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:24.803596  8851 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:24.804258  6669 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:24.678563  8858 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.810657  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:24.762678  8880 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:24.783418  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:24.787475  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:24.763522  8972 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.755414  8734 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:24.797376  8775 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.824079  8762 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:24.787137  8778 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:24.795603  8949 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:24.818650  6666 raft_consensus.cc:2368] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:24.777069  8913 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:24.905719  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.064291  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 305195us
I20260709 13:17:25.065292  8880 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.811444  6594 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:25.065762  8880 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:25.066042  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 1842) took 2096 ms (client timeout 1939 ms). Trace:
I20260709 13:17:25.066279  8949 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:25.066725  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.066581  6594 rpcz_store.cc:269] 0709 13:17:22.969416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.036449 (+ 67033us) service_pool.cc:224] Handling call
0709 13:17:24.810451 (+1774002us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:25.065974 (+255523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55052544}
I20260709 13:17:25.066082  8778 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.067600  8949 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.068470  8778 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.069279  8949 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.071154  8913 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.072696  7232 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:25.075136  8762 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:25.081707  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 293108us
I20260709 13:17:25.064841  7237 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
W20260709 13:17:25.063941  8808 consensus_meta.cc:287] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.831s	user 0.011s	sys 0.000s
I20260709 13:17:24.802767  8810 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:24.796370  8899 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:25.106863  6520 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:24.826519  8781 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.064138  8858 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.831308  8804 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:24.914772  8851 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.089366  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:25.116062  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.110560  8850 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.101037  8805 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:25.110932  8620 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:25.132531  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 67501.8us
I20260709 13:17:25.202930  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:25.118875  8905 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.113377  8974 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:25.204715  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.119380  8916 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:25.110695  8954 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.220528  8954 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.120839  8929 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.218690  8974 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:25.219200  7229 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:25.223039  7229 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:25.219822  8961 consensus_meta.cc:287] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.000s
I20260709 13:17:25.223654  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 804) took 1024 ms. Trace:
I20260709 13:17:25.223999  8961 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.223618  8775 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:25.225239  8941 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:17:25.225759  8941 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:25.226720  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.227136  8701 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.568s	user 0.007s	sys 0.002s
I20260709 13:17:25.227535  8701 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.229425  8701 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:25.229558  8806 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.567s	user 0.004s	sys 0.004s
I20260709 13:17:25.229995  8701 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:25.229971  8806 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.224082  7229 rpcz_store.cc:276] 0709 13:17:24.199201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.222507 (+1023306us) service_pool.cc:224] Handling call
0709 13:17:25.222840 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.223407 (+   567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.223598 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:25.237633  8970 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:17:25.238523  7229 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:25.239173  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 807) took 1019 ms. Trace:
I20260709 13:17:25.239504  7229 rpcz_store.cc:276] 0709 13:17:24.219679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.237824 (+1018145us) service_pool.cc:224] Handling call
0709 13:17:25.238167 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.238907 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.239119 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.241794  7229 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:25.221596  8734 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.115598  8948 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.121450  8935 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.233175  6563 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.113322  8945 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.226382  8961 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:25.204943  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 421612us
I20260709 13:17:25.124388  8851 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.199431  8850 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.247112  8941 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.247367  8806 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:25.202210  8865 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.118793  8858 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.230516  8701 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.242935  8905 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.256775  8880 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.236711  8974 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.266857  6415 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 327) took 7840 ms. Trace:
I20260709 13:17:25.205575  8620 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.269431  8620 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.260916  8701 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.271839  8948 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:25.277029  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.280064  8850 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.280161  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 800) took 1098 ms. Trace:
I20260709 13:17:25.280754  7231 rpcz_store.cc:276] 0709 13:17:24.182039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.279280 (+1097241us) service_pool.cc:224] Handling call
0709 13:17:25.279628 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.279808 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:25.279839 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:25.279883 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:25.279918 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.279984 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:25.280092 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.281599  8771 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:25.281983  8771 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:25.282451  8851 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.282318  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.267298  6415 rpcz_store.cc:276] 0709 13:17:17.428300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.428684 (+   384us) service_pool.cc:224] Handling call
0709 13:17:25.266789 (+7838105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:22.022111 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:22.327642 (+305531us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:22.332900 (+  5258us) write_op.cc:435] Acquired schema lock
0709 13:17:22.335237 (+  2337us) tablet.cc:662] Decoding operations
0709 13:17:22.397267 (+ 62030us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:22.398539 (+  1272us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:22.400485 (+  1946us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:17:22.552984 (+152499us) tablet.cc:701] Row locks acquired
0709 13:17:22.591831 (+ 38847us) write_op.cc:260] PREPARE: finished
0709 13:17:22.662691 (+ 70860us) write_op.cc:270] Start()
0709 13:17:22.753704 (+ 91013us) write_op.cc:276] Timestamp: P: 1783603042617167 usec, L: 0
0709 13:17:22.753858 (+   154us) op_driver.cc:348] REPLICATION: starting
0709 13:17:22.796124 (+ 42266us) log.cc:844] Serialized 923 byte log entry
0709 13:17:24.234108 (+1437984us) op_driver.cc:464] REPLICATION: finished
0709 13:17:24.573613 (+339505us) write_op.cc:301] APPLY: starting
0709 13:17:24.576201 (+  2588us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:24.585055 (+  8854us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:24.660911 (+ 75856us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:24.680465 (+ 19554us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:24.680626 (+   161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:17:24.680668 (+    42us) write_op.cc:312] APPLY: finished
0709 13:17:24.680864 (+   196us) log.cc:844] Serialized 29 byte log entry
0709 13:17:24.681030 (+   166us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:24.681115 (+    85us) write_op.cc:454] Released schema lock
0709 13:17:24.681153 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:24.681508 (+   355us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:24.681537 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":18176,"child_traces":[["op",{"apply.queue_time_us":328527,"apply.run_cpu_time_us":2269,"apply.run_wall_time_us":115537,"num_ops":2,"prepare.queue_time_us":340378,"prepare.run_cpu_time_us":110978,"prepare.run_wall_time_us":2192259,"raft.queue_time_us":472520,"raft.run_cpu_time_us":49886,"raft.run_wall_time_us":132339,"replication_time_us":1472489,"thread_start_us":1868656,"threads_started":4,"wal-append.queue_time_us":962724}]]}
I20260709 13:17:25.282768  8771 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.200831  8972 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:17:25.280603  8701 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.285619  7231 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.291316  7229 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:25.287425  8976 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:25.245306  8916 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.296880  8975 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:25.307775  8771 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.306816  8972 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:25.308779  8976 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:25.315934  8975 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:25.326117  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 801) took 1142 ms. Trace:
I20260709 13:17:25.327117  8972 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:25.325645  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 807) took 1027 ms. Trace:
W20260709 13:17:25.356499  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.349578  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 95009.3us
W20260709 13:17:25.336722  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 103628us
I20260709 13:17:25.342839  8905 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:25.360145  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:25.366972  8880 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.369714  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.370321  6588 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:25.371420  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:25.328902  8976 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.361727  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:25.371966  6593 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.372262  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.346226  8858 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.332633  8701 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.375010  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.333303  8975 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.367596  8899 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:25.377386  8899 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:25.377588  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.377643  8979 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:25.378333  8979 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.378443  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.378773  8979 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.376312  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:25.333465  8778 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.377789  8899 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.375222  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.386343  6588 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:25.388254  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.388841  7232 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.389325  6593 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:25.390944  8960 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:25.390998  8971 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.315s	user 0.003s	sys 0.000s
I20260709 13:17:25.392469  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2173) took 1179 ms. Trace:
I20260709 13:17:25.391841  8960 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.383733  8824 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:25.393791  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 430) took 1008 ms. Trace:
W20260709 13:17:25.395291  8912 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.000s
I20260709 13:17:25.396377  8960 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.397413  8563 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.376698  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:25.399142  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.394213  6668 rpcz_store.cc:276] 0709 13:17:24.385153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.387203 (+  2050us) service_pool.cc:224] Handling call
0709 13:17:25.393644 (+1006441us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b540011dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.393740 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130687872}
I20260709 13:17:25.392832  6522 rpcz_store.cc:276] 0709 13:17:24.213271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.248892 (+ 35621us) service_pool.cc:224] Handling call
0709 13:17:25.392323 (+1143431us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b540007d0d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.392415 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551874432}
I20260709 13:17:25.393852  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.391033  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.400796  6518 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:25.389818  8945 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:25.401397  8919 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.003s
I20260709 13:17:25.406339  8912 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:25.423527  8912 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.424031  8912 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:25.426088  6595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 1837) took 2465 ms (client timeout 1618 ms). Trace:
I20260709 13:17:25.412868  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.412868  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:25.426486  6595 rpcz_store.cc:269] 0709 13:17:22.960249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.960368 (+   119us) service_pool.cc:224] Handling call
0709 13:17:25.425911 (+2465543us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54003bd3d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.426025 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3247795968}
I20260709 13:17:25.427029  6522 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:25.427167  6589 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:25.427520  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.427834  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.428256  6595 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:25.428337  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:25.428850  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.428915  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.429055  6589 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.429575  6595 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:25.429656  6522 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:25.398747  8975 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.430389  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.413546  8787 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.409830  8701 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:17:25.414644  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:25.402403  8660 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.452960  8660 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.436780  8615 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.455924  8615 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.415289  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.408618  8892 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:17:25.443060  8773 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:25.458945  8773 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:25.459244  8892 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:25.459370  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:25.459681  8892 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:25.459733  8773 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.460770  8773 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:25.460566  8892 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:25.417667  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.431751  8835 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.450040  8919 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:25.462870  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.434281  8954 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.462780  8744 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.461746  8892 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.467988  8916 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.463781  8660 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.418233  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.418931  8778 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.470894  6593 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:25.331605  7237 rpcz_store.cc:276] 0709 13:17:24.184034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.112362 (+928328us) service_pool.cc:224] Handling call
0709 13:17:25.113655 (+  1293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.114791 (+  1136us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:25.201061 (+ 86270us) log.cc:844] Serialized 14 byte log entry
0709 13:17:25.325788 (+124727us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:25.325837 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:25.325880 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.325957 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:25.326068 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86085,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":92,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2173,"thread_start_us":124327,"threads_started":1,"wal-append.queue_time_us":151898}
I20260709 13:17:25.474157  8960 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.419219  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:25.474871  6594 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.474766  8960 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.359148  7229 rpcz_store.cc:276] 0709 13:17:24.298672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.242538 (+943866us) service_pool.cc:224] Handling call
0709 13:17:25.258393 (+ 15855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.308596 (+ 50203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.324201 (+ 15605us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.420998  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.476043  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:25.476239  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 55313.8us
I20260709 13:17:25.476756  6662 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:25.420792  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.477558  6488 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.436136  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2157) took 3480 ms. Trace:
I20260709 13:17:25.478279  8787 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:25.478192  6515 rpcz_store.cc:276] 0709 13:17:21.955901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.956063 (+   162us) service_pool.cc:224] Handling call
0709 13:17:21.978517 (+ 22454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.420348 (+2441831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.435901 (+1015553us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b540007e4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:25.436064 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5459034880}
I20260709 13:17:25.478864  8787 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:25.479225  8835 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:25.479331  8787 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.479633  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.479802  8744 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:25.480266  8744 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:25.480636  8744 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.480682  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.480968  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.481478  6668 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
W20260709 13:17:25.441341  6520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2170) took 3003 ms (client timeout 3025 ms). Trace:
I20260709 13:17:25.430389  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:25.418948  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:25.443027  6590 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:25.413085  7232 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:25.435421  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.485285  6595 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:25.486014  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.486155  7236 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:25.486037  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2176) took 1271 ms. Trace:
I20260709 13:17:25.487166  6521 rpcz_store.cc:276] 0709 13:17:24.214679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.303895 (+ 89216us) service_pool.cc:224] Handling call
0709 13:17:25.485893 (+1181998us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54003cc154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.485988 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315606272}
I20260709 13:17:25.438351  6589 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:25.450704  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.452374  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:25.474362  8933 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.786s	user 0.010s	sys 0.000s
W20260709 13:17:25.477636  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56936.6us
I20260709 13:17:25.448066  6522 raft_consensus.cc:2368] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3.
I20260709 13:17:25.421182  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:25.482004  6520 rpcz_store.cc:269] 0709 13:17:22.437428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.328533 (+891105us) service_pool.cc:224] Handling call
0709 13:17:25.441156 (+2112623us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400111054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.441285 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4079679232}
I20260709 13:17:25.480705  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.483614  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.484391  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.482293  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.483253  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.491530  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 809) took 1269 ms. Trace:
I20260709 13:17:25.491802  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 466) took 1047 ms. Trace:
I20260709 13:17:25.482380  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.494292  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:25.535035  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:25.530354  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.527825  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.568921  7236 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:25.527586  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.527907  7233 rpcz_store.cc:276] 0709 13:17:24.222424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.332667 (+110243us) service_pool.cc:224] Handling call
0709 13:17:25.491390 (+1158723us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b54004e0554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.491480 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16414336}
I20260709 13:17:25.569890  7232 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:25.534019  6668 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.570166  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.536808  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.538177  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.570739  7233 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:25.539819  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.540196  6594 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
W20260709 13:17:25.572252  6561 connection.cc:256] RPC call timeout handler was delayed by -1.1651974009998867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
I20260709 13:17:25.543321  6667 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:25.572700  6561 connection.cc:256] RPC call timeout handler was delayed by -1.1498974010000893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 13:17:25.573128  6561 connection.cc:256] RPC call timeout handler was delayed by -1.1012974010000107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
I20260709 13:17:25.481820  8949 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.575491  8949 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.576100  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.577215  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.577581  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.577837  6662 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:25.578164  6667 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:25.579181  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:25.544399  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62089.4us
I20260709 13:17:25.579707  6667 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.580503  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.581224  7232 raft_consensus.cc:2368] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4.
W20260709 13:17:25.581768  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.535156  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 113589us
I20260709 13:17:25.479656  8835 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.546452  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.546517  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.548071  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.548203  8701 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:25.548812  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.549299  6597 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:25.550410  8976 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.551602  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:25.547448  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 53238us
I20260709 13:17:25.548079  8941 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.546343  8778 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:25.556701  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1657) took 1061 ms. Trace:
I20260709 13:17:25.420769  8972 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.532866  8660 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.559438  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.564150  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:25.566144  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:25.566007  6517 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 458) took 2801 ms (client timeout 2056 ms). Trace:
I20260709 13:17:25.565917  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 463) took 1130 ms. Trace:
I20260709 13:17:25.566602  6516 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:25.569792  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.528366  6518 rpcz_store.cc:276] 0709 13:17:24.444559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.452137 (+1007578us) service_pool.cc:224] Handling call
0709 13:17:25.491737 (+ 39600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.569063  6520 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:25.574419  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.574548  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.544013  8563 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:25.579321  8960 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.583840  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.561206  7231 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:25.695616  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.588469  6668 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:25.592603  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:25.651145  6517 rpcz_store.cc:269] 0709 13:17:22.764603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.040051 (+275448us) service_pool.cc:224] Handling call
0709 13:17:25.565787 (+2525736us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540019bed4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.565934 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3264500608}
I20260709 13:17:25.652000  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.602228  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.653146  6522 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:25.655043  7234 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:25.700152  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.700932  7234 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:25.701539  6520 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:25.652050  6519 rpcz_store.cc:276] 0709 13:17:24.435444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.457522 (+ 22078us) service_pool.cc:224] Handling call
0709 13:17:25.565736 (+1108214us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540014c8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.565863 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481403776}
W20260709 13:17:25.701944  6561 connection.cc:256] RPC call timeout handler was delayed by -1.0202974009998798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
I20260709 13:17:25.654807  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.656139  6515 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:17:25.656828  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.702668  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:25.703275  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.651176  7233 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:25.704528  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.705087  7233 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.590812  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1656) took 1095 ms. Trace:
I20260709 13:17:25.563333  8869 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.492885  8974 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.585798  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.699167  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.653877  6669 rpcz_store.cc:276] 0709 13:17:24.495698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.541942 (+ 46244us) service_pool.cc:224] Handling call
0709 13:17:25.556508 (+1014566us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540005b254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.556646 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589089024}
I20260709 13:17:25.685732  6513 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:25.660373  6663 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:25.653729  6665 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:25.710153  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:25.665330  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 124615us
I20260709 13:17:25.715440  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.589501  8955 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:25.544983  8933 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.653684  8778 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:25.761643  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1750) took 1200 ms. Trace:
I20260709 13:17:25.651160  7236 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:25.605795  8903 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.561497  8899 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:25.752102  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48899.9us
W20260709 13:17:25.707808  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 122037us
I20260709 13:17:25.766824  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.756049  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.768656  6517 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:25.660043  8835 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.708696  6662 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:25.772708  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.773456  6662 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:25.664422  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.757764  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1733) took 2883 ms. Trace:
W20260709 13:17:25.757766  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 47715.9us
I20260709 13:17:25.760520  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 797) took 1589 ms. Trace:
I20260709 13:17:25.701427  8660 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:25.590276  8919 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.762081  6515 rpcz_store.cc:276] 0709 13:17:24.561132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.481640 (+920508us) service_pool.cc:224] Handling call
0709 13:17:25.761573 (+279933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.694846  8563 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:25.762775  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.763619  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.812475  6522 raft_consensus.cc:2368] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4.
I20260709 13:17:25.593360  8949 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.756721  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:25.763197  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.766732  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.816991  6663 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.709581  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:25.609225  8787 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.768107  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.818907  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:25.707837  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 124068us
I20260709 13:17:25.820201  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:25.821821  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.770722  6596 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:17:25.765743  6514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1732) took 2888 ms (client timeout 2163 ms). Trace:
I20260709 13:17:25.685930  8701 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.705893  6671 rpcz_store.cc:276] 0709 13:17:24.495451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.495560 (+   109us) service_pool.cc:224] Handling call
0709 13:17:25.590654 (+1095094us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b540030ff54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.590761 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30238080}
I20260709 13:17:25.660530  6666 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:25.773947  6516 rpcz_store.cc:276] 0709 13:17:22.874039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.857248 (+983209us) service_pool.cc:224] Handling call
0709 13:17:25.566427 (+1709179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.667140 (+100713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.757563 (+ 90423us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b540019bed0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.757706 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3892564224}
I20260709 13:17:25.774915  7232 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:25.775099  6637 leader_election.cc:400] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3.
I20260709 13:17:25.832777  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.833299  6637 leader_election.cc:403] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:17:25.833581  7232 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
W20260709 13:17:25.833763  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 58866.8us
W20260709 13:17:25.834125  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.834372  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.834973  7232 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:25.835726  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.836176  7232 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.775794  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.794953  6669 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:25.751708  8892 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.694854  8952 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.843015  6669 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:17:25.843411  6669 pending_rounds.cc:85] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 5
I20260709 13:17:25.843668  6669 pending_rounds.cc:107] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:17:25.843889  6669 raft_consensus.cc:2889] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:25.845129  6669 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:25.801616  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.756987  8827 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:25.756049  8771 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:25.764397  8929 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:25.706979  6518 raft_consensus.cc:2368] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:25.853250  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.853907  6518 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:25.761691  8972 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.761916  8779 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:25.762198  8955 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.811345  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.831689  8919 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:25.765571  8913 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:25.765661  8778 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:25.818961  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.861449  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.862277  6515 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:25.714640  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.755246  6520 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:25.822592  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.822937  6664 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:25.862941  7234 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:25.822047  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.823597  6670 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:25.771709  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.864501  6665 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:25.695374  8941 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:25.865187  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.866271  7232 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:25.866386  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:25.840757  8804 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.868219  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:25.837281  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.868868  7232 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 1.
I20260709 13:17:25.868927  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 462) took 1434 ms. Trace:
I20260709 13:17:25.869387  6515 rpcz_store.cc:276] 0709 13:17:24.435068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.840195 (+1405127us) service_pool.cc:224] Handling call
0709 13:17:25.868871 (+ 28676us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:25.869743  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.841743  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1745) took 1282 ms. Trace:
I20260709 13:17:25.870126  6517 rpcz_store.cc:276] 0709 13:17:24.559809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.775497 (+1215688us) service_pool.cc:224] Handling call
0709 13:17:25.841646 (+ 66149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.870602  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:25.870982  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.871511  6517 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.871829  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:25.848169  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 81603.4us
I20260709 13:17:25.848483  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.873668  6522 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:25.874133  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1743) took 1315 ms. Trace:
I20260709 13:17:25.874511  6522 rpcz_store.cc:276] 0709 13:17:24.559256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.818555 (+1259299us) service_pool.cc:224] Handling call
0709 13:17:25.874060 (+ 55505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.816180  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.846971  6662 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:25.880719  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.841178  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.881575  6666 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:25.850910  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.807680  8773 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:17:25.816526  8858 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:25.882706  6516 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:25.846494  6669 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:25.830875  8615 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:25.855680  6519 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:25.813354  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2175) took 1599 ms. Trace:
I20260709 13:17:25.780757  8912 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.904502  7235 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:25.858522  8955 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.858286  8779 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.857473  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.905503  6518 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.831671  8911 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:17:25.820300  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 110810us
I20260709 13:17:25.912442  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:25.866664  8913 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:25.825141  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.840485  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:25.856343  8929 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.850818  8827 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:25.855638  8975 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.885054  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2179) took 1240 ms. Trace:
I20260709 13:17:25.926016  6519 rpcz_store.cc:276] 0709 13:17:24.644886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.766360 (+1121474us) service_pool.cc:224] Handling call
0709 13:17:25.885000 (+118640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:25.885573  8856 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.885174  6513 rpcz_store.cc:276] 0709 13:17:24.214298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.357996 (+143698us) service_pool.cc:224] Handling call
0709 13:17:24.375337 (+ 17341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.755156 (+1379819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.813142 (+ 57986us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400355f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:25.813298 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2961097088}
I20260709 13:17:25.884253  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.904624  8850 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.884826  8615 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.933923  8615 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.914260  8779 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.887770  8941 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.888455  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:25.883994  6487 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.531s (SENT)
I20260709 13:17:25.942595  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:25.889462  8787 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:25.889818  8865 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:25.890146  8858 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.882452  8916 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.899267  7231 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.910472  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.905035  8927 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.913534  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:25.966717  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53254.8us
I20260709 13:17:25.840935  8949 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.845521  8892 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.979725  8949 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.980425  8858 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.981148  8858 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:25.981302  8892 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.919207  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.854962  8952 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.982095  8891 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:25.982271  7236 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:25.983355  8787 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:25.983552  8952 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:25.983803  8787 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:25.924309  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:25.984251  8787 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.918393  8919 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:25.984591  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:25.967181  8981 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:17:25.985709  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:25.885314  6668 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:25.927896  7234 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:25.823870  6514 rpcz_store.cc:269] 0709 13:17:22.873480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.233988 (+360508us) service_pool.cc:224] Handling call
0709 13:17:25.699907 (+2465919us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540007dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.761351 (+ 61444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3434218880}
W20260709 13:17:25.989020  7235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 733) took 3937 ms (client timeout 1531 ms). Trace:
W20260709 13:17:25.989425  7235 rpcz_store.cc:269] 0709 13:17:22.051439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.096797 (+ 45358us) service_pool.cc:224] Handling call
0709 13:17:23.871392 (+1774595us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:25.988862 (+2117470us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b540051a254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:25.988960 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4803535872}
I20260709 13:17:25.990865  8933 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.991644  8974 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.992008  8916 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:25.992424  8899 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.992915  8916 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.993563  8701 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:25.883617  8771 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.865682  8979 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.887852  8563 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:25.929808  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.006877  6515 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.981725  8949 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:25.946305  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:25.945389  8976 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.017381  6519 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:25.924332  8929 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:25.919735  8913 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:25.949268  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 461) took 1715 ms. Trace:
I20260709 13:17:25.928040  8827 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:25.880712  8804 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.952670  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:25.963505  6522 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:25.922303  8911 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:25.967456  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:25.978772  8927 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.981880  6671 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:25.994340  8771 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:26.003748  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.029274  7236 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:26.034096  6591 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:17:26.034463  6591 pending_rounds.cc:85] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 2
I20260709 13:17:26.034715  6591 pending_rounds.cc:107] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:17:26.035005  6591 raft_consensus.cc:2889] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:26.035128  8979 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.036176  8771 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.037384  7233 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:26.037858  8840 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.038322  8840 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.039364  8911 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.043454  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.002481  6667 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:25.883173  8660 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:25.945732  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:25.943324  8850 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:26.018360  6516 rpcz_store.cc:276] 0709 13:17:24.233861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.948507 (+1714646us) service_pool.cc:224] Handling call
0709 13:17:25.948786 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.948958 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:25.948984 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:25.949022 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:25.949051 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.949113 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:25.949218 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.046689  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.003793  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.047926  7234 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.019405  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 796) took 3378 ms. Trace:
I20260709 13:17:26.022261  6518 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:26.024820  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:26.058501  6670 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:26.027554  7232 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.004253  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.060004  7237 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:26.043550  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76176.9us
I20260709 13:17:25.939805  8856 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.044979  7233 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.947980  8744 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.017202  8933 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.020197  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.072186  6561 leader_election.cc:400] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:26.079654  6561 leader_election.cc:403] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:26.081743  6561 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:26.082016  6561 leader_election.cc:403] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:26.061961  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2185) took 1287 ms. Trace:
I20260709 13:17:26.073087  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1748) took 1512 ms. Trace:
I20260709 13:17:26.082763  6518 rpcz_store.cc:276] 0709 13:17:24.774857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.857274 (+1082417us) service_pool.cc:224] Handling call
0709 13:17:26.061890 (+204616us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.082815  6519 rpcz_store.cc:276] 0709 13:17:24.560581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.017999 (+1457418us) service_pool.cc:224] Handling call
0709 13:17:26.072445 (+ 54446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.072679 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.072716 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.072772 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:26.072813 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.072885 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.073016 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.074357  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:26.084436  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2180) took 1439 ms. Trace:
I20260709 13:17:26.084764  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.085835  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:26.023983  8949 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.086194  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2177) took 1441 ms. Trace:
I20260709 13:17:26.086954  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:26.091735  6667 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:26.091516  6522 rpcz_store.cc:276] 0709 13:17:24.644380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.027292 (+1382912us) service_pool.cc:224] Handling call
0709 13:17:26.085746 (+ 58454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.085896 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.085924 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.085975 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:26.086007 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.086054 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.086142 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.092278  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:26.088351  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:26.084759  6518 rpcz_store.cc:276] 0709 13:17:24.645108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.083736 (+1438628us) service_pool.cc:224] Handling call
0709 13:17:26.084006 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.084157 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.084180 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.084220 (+    40us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:26.084248 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.084296 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.084386 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.026266  8858 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:26.107333  6516 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:26.107982  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2182) took 1449 ms. Trace:
I20260709 13:17:26.108276  6516 rpcz_store.cc:276] 0709 13:17:24.659025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.090512 (+1431487us) service_pool.cc:224] Handling call
0709 13:17:26.107171 (+ 16659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.107745 (+   574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.107918 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.046149  8976 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:26.056963  7230 rpcz_store.cc:276] 0709 13:17:22.640568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.225352 (+584784us) service_pool.cc:224] Handling call
0709 13:17:24.579633 (+1354281us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:24.579819 (+   186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:24.585173 (+  5354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:24.585204 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:24.619073 (+ 33869us) log.cc:844] Serialized 26 byte log entry
0709 13:17:25.578698 (+959625us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b180045e8c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:17:25.810907 (+232209us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:25.969208 (+158301us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.019112 (+ 49904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.019183 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:26.019222 (+    39us) raft_consensus.cc:1685] finished
0709 13:17:26.019235 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.019352 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48826,"prepare.queue_time_us":221421,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":52618,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":32730,"spinlock_wait_cycles":151516928,"thread_start_us":369029,"threads_started":2,"wal-append.queue_time_us":1265459}
I20260709 13:17:26.094041  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:26.089517  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:26.115928  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:26.117267  7232 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:26.119376  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.122217  6593 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:26.123554  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.124364  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.124904  6588 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:26.125440  7236 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:26.098551  6518 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:26.109253  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:26.089661  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 205689us
I20260709 13:17:25.800851  7228 rpcz_store.cc:276] 0709 13:17:24.171280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.396451 (+1225171us) service_pool.cc:224] Handling call
0709 13:17:25.396794 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:25.397169 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:25.397198 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:25.397644 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:17:25.523702 (+126058us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:25.533856 (+ 10154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:25.539327 (+  5471us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:25.760302 (+220975us) raft_consensus.cc:1685] finished
0709 13:17:25.760329 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:25.760463 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217,"raft.queue_time_us":1807,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":2620,"thread_start_us":66293,"threads_started":1,"wal-append.queue_time_us":169980}
W20260709 13:17:26.097766  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:26.127012  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.098325  8701 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.138953  8974 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:17:26.103037  8824 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.166463  8824 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.160044  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.177891  6663 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:26.133013  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.135226  6668 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:26.087724  8972 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:26.138230  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.100349  8839 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.138314  7233 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:26.131665  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:26.190042  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58497.9us
I20260709 13:17:26.192294  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.146274  7234 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:26.118470  8930 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:26.147854  6590 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:26.148618  6596 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:26.093498  8933 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.152885  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.137128  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:26.203756  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 66695us
I20260709 13:17:26.105207  8869 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:26.086122  8979 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:17:26.138471  8892 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:26.156080  8899 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:26.152267  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.222905  6516 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:26.203430  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.162858  8906 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:26.213388  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.228601  6522 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:26.214078  6636 leader_election.cc:400] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4.
I20260709 13:17:26.229600  6636 leader_election.cc:403] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:17:26.106500  8949 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.241372  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.216862  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.244411  6590 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 1.
I20260709 13:17:26.202697  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:26.136436  8987 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:26.201833  6589 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:26.224368  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:26.225594  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:26.188551  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 317977us
W20260709 13:17:26.265544  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.266139  6637 leader_election.cc:400] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4.
I20260709 13:17:26.267330  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.183015  8960 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3.
I20260709 13:17:26.162149  8912 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.216015  7237 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:26.260262  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.110430  8858 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:26.262355  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59722.6us
I20260709 13:17:26.274195  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:26.262883  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58414.1us
I20260709 13:17:26.275964  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.139717  8919 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:17:26.273020  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.285543  6516 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.218173  8972 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:26.298264  6513 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:26.218763  8892 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:26.221698  8990 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:26.291957  6637 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.284628  7234 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:26.289708  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:26.320247  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.274014  8941 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:26.273312  8808 raft_consensus.cc:2764] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:17:26.232307  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.274526  8773 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4.
I20260709 13:17:26.263566  8987 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:26.315299  8892 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:26.316766  8990 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:26.322525  8744 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.323212  8949 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.323437  7237 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:26.348220  8941 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:26.366065  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 812) took 1726 ms. Trace:
I20260709 13:17:26.366708  7237 rpcz_store.cc:276] 0709 13:17:24.639522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.277895 (+1638373us) service_pool.cc:224] Handling call
0709 13:17:26.323240 (+ 45345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.365749 (+ 42509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.365995 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.329427  8804 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.375339  6513 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:26.325194  7231 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:26.388592  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 393064us
W20260709 13:17:26.393033  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.394773  7230 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:26.397877  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 808) took 2090 ms. Trace:
I20260709 13:17:26.398348  7230 rpcz_store.cc:276] 0709 13:17:24.307578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.374735 (+2067157us) service_pool.cc:224] Handling call
0709 13:17:26.375093 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.397557 (+ 22464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.397812 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.401464  6516 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:26.327064  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.332123  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.409623  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.200240  8850 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:26.310918  8912 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:26.414842  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 138967us
I20260709 13:17:26.336658  7232 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:26.415933  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:26.337390  6517 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:26.339346  6520 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:26.307000  8858 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:26.343667  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.357590  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.419744  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.421202  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.401669  6515 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
W20260709 13:17:26.402582  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 293374us
I20260709 13:17:26.424175  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:26.346853  8987 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.349035  7228 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.409765  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.437037  6516 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.437444  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:26.438282  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 146599us
I20260709 13:17:26.411636  8916 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.448055  8916 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.448092  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.449764  6591 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:17:26.320752  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:26.406150  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.427263  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.451958  6520 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:26.418025  8804 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:26.452905  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.355044  8990 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:26.453873  6520 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:26.432323  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.454572  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.454916  6522 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:26.455169  6520 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:26.456094  6522 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:26.432701  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.457883  7235 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 13:17:26.458096  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.458455  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 736) took 1058 ms. Trace:
I20260709 13:17:26.450140  8916 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.458829  7235 rpcz_store.cc:276] 0709 13:17:25.399785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.457371 (+1057586us) service_pool.cc:224] Handling call
0709 13:17:26.457661 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:26.458211 (+   550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.458392 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.460656  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 738) took 1060 ms. Trace:
I20260709 13:17:26.461016  7235 rpcz_store.cc:276] 0709 13:17:25.400270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.459919 (+1059649us) service_pool.cc:224] Handling call
0709 13:17:26.460193 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.460366 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.460392 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.460434 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:26.460466 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.460518 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.460607 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.462301  7235 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 5. Ops: []
W20260709 13:17:26.463809  8778 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.581s	user 0.006s	sys 0.003s
I20260709 13:17:26.435216  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.464389  6590 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:26.464252  8778 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:26.465253  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.466266  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.373834  8929 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.411059  8563 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.467470  6592 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:26.467972  8563 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.468458  8929 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.442515  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:26.407480  8949 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:26.468924  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:26.469118  8929 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:26.469702  6663 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:26.469187  8949 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.469659  8929 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.470144  8949 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.470240  8929 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.470973  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:26.471737  6663 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:26.473825  6670 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:17:26.475085  8615 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.541s	user 0.003s	sys 0.005s
I20260709 13:17:26.438329  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.475164  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:26.475839  6517 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:26.443210  6597 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:26.475438  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.443233  8991 raft_consensus.cc:2764] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:26.452883  8804 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.463680  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:26.487914  6668 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.467422  6518 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:26.466898  6590 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:26.475551  8615 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.346346  8989 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:26.466411  8778 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:26.478597  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 739) took 1078 ms. Trace:
I20260709 13:17:26.503325  7235 rpcz_store.cc:276] 0709 13:17:25.400493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.461892 (+1061399us) service_pool.cc:224] Handling call
0709 13:17:26.462145 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.478358 (+ 16213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.478538 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.499590  8824 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:26.487267  8787 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.506548  8839 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:26.433310  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 88076.2us
I20260709 13:17:26.480919  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.508090  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:26.481617  7237 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:26.407656  8856 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.477841  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.453063  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.485685  6522 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:26.412853  8972 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:26.477036  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:26.494125  6520 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:26.492234  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:26.516148  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.516708  6520 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:26.405851  8773 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:17:26.435851  8906 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:26.444730  7200 connection.cc:256] RPC call timeout handler was delayed by -1.9325460140000814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 13:17:26.524178  7200 connection.cc:256] RPC call timeout handler was delayed by -1.8227460140001313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
I20260709 13:17:26.523904  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:26.525566  7200 connection.cc:256] RPC call timeout handler was delayed by -1.3661349559999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
I20260709 13:17:26.435869  8912 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:26.530730  6590 raft_consensus.cc:2368] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3.
I20260709 13:17:26.533232  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:26.534883  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:26.506894  8787 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.544842  8787 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:26.510283  6516 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:26.367095  8941 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.513283  7232 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:26.497041  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:26.548454  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 51495.3us
I20260709 13:17:26.549307  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:26.526654  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48874.9us
I20260709 13:17:26.527372  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 813) took 1866 ms. Trace:
I20260709 13:17:26.492398  8804 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.495864  8989 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.558836  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.511914  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 468) took 2025 ms. Trace:
I20260709 13:17:26.552402  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:26.507150  8994 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:17:26.575682  9006 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:26.576224  8744 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:26.576625  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 125010us
I20260709 13:17:26.556053  6668 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:26.577524  6563 leader_election.cc:400] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:26.577979  6563 leader_election.cc:403] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:26.560249  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 61410.8us
I20260709 13:17:26.548141  8991 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:26.550200  7237 rpcz_store.cc:276] 0709 13:17:24.661269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.422370 (+1761101us) service_pool.cc:224] Handling call
0709 13:17:26.481436 (+ 59066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.509041 (+ 27605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.527311 (+ 18270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.563886  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.563788  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.479413  8988 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:17:26.548969  8858 raft_consensus.cc:2764] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:26.546972  8992 raft_consensus.cc:2764] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:17:26.551187  8660 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:26.557657  8856 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:26.581188  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:26.574925  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:26.582376  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.584424  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 814) took 1127 ms. Trace:
I20260709 13:17:26.597601  7237 rpcz_store.cc:276] 0709 13:17:25.456806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.583411 (+1126605us) service_pool.cc:224] Handling call
0709 13:17:26.583894 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.584079 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.584116 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.584158 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:26.584192 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.584259 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.584364 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:26.583019  6591 raft_consensus.cc:2368] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:26.590945  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.591197  6590 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:26.557367  8933 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:26.526830  8911 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:17:26.560679  8989 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.601873  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.590852  8869 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:26.607465  9011 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:26.607041  8912 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.607887  9011 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.608301  9011 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.608493  8972 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.610423  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:26.611874  9003 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:17:26.564651  8778 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:17:26.612911  8992 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:26.614316  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.614862  8773 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:26.615223  8773 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.615567  8773 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.607079  8991 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:26.616894  8869 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.618116  8991 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.619675  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.621032  8911 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5.
I20260709 13:17:26.598389  8824 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:26.622993  8824 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.624979  8933 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:26.625489  8933 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:26.625953  8933 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.615499  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:26.629894  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.633929  6590 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:26.635476  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.633458  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.640347  9014 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:26.595464  9004 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:26.596400  8856 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.664132  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.648942  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.649075  8997 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:26.649366  9004 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:26.647806  9014 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:26.671248  9010 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:17:26.671859  9014 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:26.651654  8779 consensus_meta.cc:287] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.700s	user 0.007s	sys 0.003s
I20260709 13:17:26.672986  9004 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.673368  8779 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.674124  9004 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.674794  8927 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:26.654711  8903 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.884s	user 0.007s	sys 0.003s
I20260709 13:17:26.657608  8778 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:26.669142  8988 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:26.676251  8778 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:26.676630  8988 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:26.677106  8778 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.677763  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.677568  8988 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.658423  8990 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.602820  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 814) took 1911 ms. Trace:
I20260709 13:17:26.659648  8771 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:17:26.660653  8913 consensus_meta.cc:287] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.616s	user 0.005s	sys 0.004s
I20260709 13:17:26.647178  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.663074  8615 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.652458  9011 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.668756  8997 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:26.666896  8856 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.670886  8660 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:26.678285  8778 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.602589  9016 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:26.736912  8997 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.737870  8804 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.726948  9012 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.739037  9012 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.739435  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.740098  6594 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 1.
I20260709 13:17:26.740994  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.741119  9012 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.741556  6594 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:26.742199  8744 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:26.678992  8832 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.742928  8744 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:26.742856  8832 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.743333  8744 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:26.744151  8744 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:26.745337  6595 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:26.746130  7236 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:26.733980  8987 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:26.678388  6596 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:26.669752  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:26.565589  6514 rpcz_store.cc:276] 0709 13:17:24.475252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.988708 (+1513456us) service_pool.cc:224] Handling call
0709 13:17:25.988991 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:25.989199 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:26.217211 (+228012us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.432922 (+215711us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.440786 (+  7864us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:26.444965 (+  4179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.484810 (+ 39845us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.500611 (+ 15801us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":328904,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":41,"raft.queue_time_us":20686,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":38853,"thread_start_us":344993,"threads_started":2,"wal-append.queue_time_us":278926}
I20260709 13:17:26.753104  8865 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.728371  8999 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:26.685070  8771 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:26.746083  8660 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.736629  9011 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.667793  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.708724  6589 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:26.731400  8952 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:26.732214  7228 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:26.732888  8779 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.768257  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.680378  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.753433  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.756327  6596 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:26.745393  8832 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:26.837001  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 168282us
I20260709 13:17:26.742422  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.838689  6594 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:26.839905  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 810) took 2478 ms. Trace:
I20260709 13:17:26.840408  7234 rpcz_store.cc:276] 0709 13:17:24.361895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.203912 (+1842017us) service_pool.cc:224] Handling call
0709 13:17:26.225254 (+ 21342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.318914 (+ 93660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.839614 (+520700us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b5400567f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:26.839830 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135006080}
I20260709 13:17:26.803001  6591 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:26.847697  6593 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:26.848831  7230 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:26.735922  8835 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.928s	user 0.009s	sys 0.000s
I20260709 13:17:26.850471  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:26.851872  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:26.857023  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.860520  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:26.868021  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1672) took 1303 ms. Trace:
I20260709 13:17:26.869261  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1758) took 1285 ms. Trace:
I20260709 13:17:26.883303  6513 rpcz_store.cc:276] 0709 13:17:25.583479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.927722 (+344243us) service_pool.cc:224] Handling call
0709 13:17:26.869070 (+941348us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54003f7fd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:26.869194 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582907264}
I20260709 13:17:26.884662  6671 rpcz_store.cc:276] 0709 13:17:25.568149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.840228 (+272079us) service_pool.cc:224] Handling call
0709 13:17:26.867839 (+1027611us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b54000e60d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:26.867951 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148728704}
I20260709 13:17:26.886431  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:26.888365  7236 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:26.888422  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:26.890926  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.892457  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:26.872941  7235 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:26.895097  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 740) took 1491 ms. Trace:
I20260709 13:17:26.895619  7235 rpcz_store.cc:276] 0709 13:17:25.403343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.504066 (+1100723us) service_pool.cc:224] Handling call
0709 13:17:26.872437 (+368371us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:26.872694 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.894778 (+ 22084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.895032 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11387776}
W20260709 13:17:26.736013  8827 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.692s	user 0.007s	sys 0.003s
I20260709 13:17:26.866429  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 820) took 1439 ms. Trace:
W20260709 13:17:26.766420  8899 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.004s
I20260709 13:17:26.897305  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.732254  8824 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:26.899420  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49038.2us
I20260709 13:17:26.908767  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.901871  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.906781  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.903038  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:26.905790  7232 rpcz_store.cc:276] 0709 13:17:25.426690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.118205 (+691515us) service_pool.cc:224] Handling call
0709 13:17:26.866226 (+748021us) spinlock_profiling.cc:243] Waited 90.5 ms on lock 0x7b540049d254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:26.866358 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208264576}
I20260709 13:17:26.909206  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:26.905220  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.909942  7235 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:26.898797  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.766856  8979 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:26.759771  8865 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:26.846196  8701 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.000s
I20260709 13:17:26.905563  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:26.753142  9013 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.761070  8999 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:26.756884  8990 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:26.763015  8660 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.835439  9011 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.848193  8952 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:27.002363  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 93210.9us
I20260709 13:17:26.916227  8927 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:26.692677  8913 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.003903  6589 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
W20260709 13:17:26.972110  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 390949us
I20260709 13:17:27.004652  7230 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:27.021651  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 824) took 1165 ms. Trace:
I20260709 13:17:27.022116  7230 rpcz_store.cc:276] 0709 13:17:25.856154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.020884 (+1164730us) service_pool.cc:224] Handling call
0709 13:17:27.021171 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.021338 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:27.021363 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:27.021406 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:27.021436 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.021494 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.021595 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.009068  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:26.982275  8891 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:27.001986  8701 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:26.761768  8771 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:26.763571  8955 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.843s	user 0.009s	sys 0.000s
I20260709 13:17:26.920955  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:27.002985  6595 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:26.919729  8827 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.003054  6594 raft_consensus.cc:2368] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:27.009254  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:27.063639  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 142868us
I20260709 13:17:27.064123  6516 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.010569  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 823) took 1516 ms. Trace:
I20260709 13:17:27.016324  8927 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.003268  8824 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:17:26.899461  9016 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:26.971171  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 389776us
I20260709 13:17:26.917109  8975 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:26.983312  8899 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.064815  7235 rpcz_store.cc:276] 0709 13:17:25.494233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.009613 (+1515380us) service_pool.cc:224] Handling call
0709 13:17:27.009905 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.010242 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:27.010273 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:27.010317 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:27.010346 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.010404 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.010514 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7706}
I20260709 13:17:27.073140  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:27.074466  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:27.016395  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 505948us
I20260709 13:17:26.990640  8979 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:27.023386  6664 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:27.024892  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.096400  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:27.099303  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:26.971570  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 419944us
I20260709 13:17:26.991582  7234 raft_consensus.cc:2368] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:26.899777  8832 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:17:27.006714  8941 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:26.902757  8835 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:27.011484  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 500275us
I20260709 13:17:27.019630  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:27.100124  7200 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.887s (SENT)
I20260709 13:17:26.683876  7231 rpcz_store.cc:276] 0709 13:17:24.686596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.479600 (+1793004us) service_pool.cc:224] Handling call
0709 13:17:26.525830 (+ 46230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.558016 (+ 32186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.561329 (+  3313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.576920 (+ 15591us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:26.578571 (+  1651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.581988 (+  3417us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:26.598237 (+ 16249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.102706  6486 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:27.098464  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:27.058976  8891 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.107893  8987 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.064929  8927 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.013206  8773 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.016942  9011 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.009822  9013 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.056437  7228 raft_consensus.cc:2376] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 4. Re-sending same reply.
I20260709 13:17:27.252488  9018 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:27.067673  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:27.252621  9024 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:17:27.255733  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:27.252269  8840 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.256742  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 821) took 1762 ms. Trace:
I20260709 13:17:27.255293  9018 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:27.257098  7237 rpcz_store.cc:276] 0709 13:17:25.493917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.598605 (+1104688us) service_pool.cc:224] Handling call
0709 13:17:26.645758 (+ 47153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:26.646176 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.646205 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:26.646602 (+   397us) log.cc:844] Serialized 26 byte log entry
0709 13:17:26.747813 (+101211us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:26.747853 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.747911 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.256527 (+508616us) raft_consensus.cc:1685] finished
0709 13:17:27.256547 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.256688 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65,"raft.queue_time_us":15071,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":78663,"threads_started":1,"wal-append.queue_time_us":230814}
I20260709 13:17:27.258265  9018 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.109903  6519 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:27.263248  8832 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:27.254460  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 118622us
I20260709 13:17:27.104647  8991 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.163282  8787 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:27.268229  9004 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.000s
I20260709 13:17:27.267884  8991 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.268018  8787 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.268962  8827 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.134234  8701 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 13:17:27.270306  8991 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.133988  8952 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.267961  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:27.267261  8997 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.003s
W20260709 13:17:27.266887  8989 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.626s	user 0.003s	sys 0.009s
I20260709 13:17:27.194006  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:27.065408  8976 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:27.209862  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:27.273489  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 170849us
I20260709 13:17:27.283723  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:27.288177  6486 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:27.154877  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.286288  8997 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:27.133175  8949 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
I20260709 13:17:27.222071  6516 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
W20260709 13:17:27.166976  8906 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.619s	user 0.006s	sys 0.003s
W20260709 13:17:27.150456  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.096534  8903 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:27.303215  6589 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:27.303599  6589 pending_rounds.cc:85] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 6
I20260709 13:17:27.303822  6589 pending_rounds.cc:107] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:27.292069  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:27.297240  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 198827us
I20260709 13:17:27.304121  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:27.286841  8976 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:27.307334  6590 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:27.065428  9012 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.230s
I20260709 13:17:27.296260  8987 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:17:27.319414  8987 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:27.319563  9028 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:17:27.296307  8832 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.323570  8987 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:27.323796  8997 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:27.283058  8999 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.306152  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:27.326004  8892 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.963s	user 0.005s	sys 0.004s
I20260709 13:17:27.299743  7232 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:27.307530  8827 raft_consensus.cc:2764] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:27.327387  7231 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:27.327807  8827 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:27.328199  8827 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:27.328262  8773 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.328653  8827 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:27.305559  8899 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:27.329018  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.329337  8899 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:27.329761  7234 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:27.330596  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.331724  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.331966  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.322242  9011 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.304054  6589 raft_consensus.cc:2889] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.333212  8832 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:27.333650  8832 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:27.307045  9004 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:27.334118  8832 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:27.334556  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:27.335156  8933 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.000s
I20260709 13:17:27.335551  8933 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.337244  8933 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.286432  8989 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.324486  8987 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.332608  7237 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:27.339933  8989 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:27.340278  9013 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.300086  8941 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:27.340747  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.341404  8804 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.341316  8941 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.341825  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.330199  8899 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:27.343801  8929 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.849s	user 0.008s	sys 0.000s
I20260709 13:17:27.287484  8979 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.344383  8929 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.345019  8979 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.345341  7237 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:27.332692  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.333710  7235 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:17:27.346486  7237 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:27.311941  8988 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:27.346483  8929 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.349224  8744 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:27.319682  8778 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.001s
I20260709 13:17:27.287001  8850 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:27.306313  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:27.311373  8913 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:27.353430  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 47179.2us
I20260709 13:17:27.353760  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.354120  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:27.328564  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.354631  7231 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:27.355293  9022 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:27.075181  8955 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.357219  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.357792  7230 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.359136  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:27.360986  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.361963  6668 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:27.316232  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:27.364818  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:27.365468  6663 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
W20260709 13:17:27.365746  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49596.6us
I20260709 13:17:27.365949  7234 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:27.366271  6671 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:27.368099  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:27.369321  7234 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:27.355819  7236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:47822
I20260709 13:17:27.370977  7236 raft_consensus.cc:606] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:27.371308  7236 raft_consensus.cc:614] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:27.307324  8906 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:27.372284  8563 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.306839  8949 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.375895  8840 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:27.376153  9031 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:27.376319  8840 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:27.377151  8955 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.377367  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:27.377919  7233 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:27.379123  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 829) took 1187 ms. Trace:
I20260709 13:17:27.379509  7233 rpcz_store.cc:276] 0709 13:17:26.191659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.192072 (+   413us) service_pool.cc:224] Handling call
0709 13:17:27.378914 (+1186842us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540025c454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:27.379059 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724810112}
I20260709 13:17:27.379853  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:27.380489  6670 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:27.370393  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:27.381330  6669 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:27.377188  8869 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.382902  8976 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.384027  8976 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.384896  8832 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.377394  8840 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.387214  7231 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:27.387275  8840 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:27.387620  8891 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.367013  8778 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:27.326402  8892 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.391158  8778 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:27.392148  8778 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.393060  8892 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:27.393893  9027 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:17:27.367722  8856 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.617s	user 0.003s	sys 0.006s
I20260709 13:17:27.394994  8856 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.395489  9014 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.396910  8856 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.399919  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:27.401580  6637 leader_election.cc:400] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 2 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3.
I20260709 13:17:27.402483  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:27.404976  7236 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:27.303301  8903 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.406414  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.407620  8913 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:27.355897  9022 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:27.408105  8960 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:27.409124  7234 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:27.454365  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.364931  9011 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.438380  8913 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.439329  8903 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.474298  8835 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.471408  8913 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.466169  8927 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:27.506342  8865 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:17:27.506994  8865 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.522742  9022 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.528375  8927 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:27.301029  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:27.599442  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 298221us
W20260709 13:17:27.600454  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.601325  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:27.287580  6513 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:27.306901  6521 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
W20260709 13:17:27.614490  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.617307  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:27.619714  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:27.621556  6594 raft_consensus.cc:2368] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5.
I20260709 13:17:27.621465  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.620023  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.622853  6593 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:27.623786  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:27.623996  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:27.624691  6592 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:27.625437  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.627995  6597 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:27.630438  6597 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:17:27.639479  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:27.641315  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.644737  6597 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:17:27.647811  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 244308us
I20260709 13:17:27.653376  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:27.653755  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 444) took 1809 ms. Trace:
W20260709 13:17:27.655179  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 252636us
I20260709 13:17:27.657070  6637 leader_election.cc:400] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:27.657666  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:27.417675  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 317534us
I20260709 13:17:27.654376  6664 rpcz_store.cc:276] 0709 13:17:25.844072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.863426 (+ 19354us) service_pool.cc:224] Handling call
0709 13:17:25.863689 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.095536 (+1231847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.652904 (+557368us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000b3454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:27.653401 (+   497us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666591104}
W20260709 13:17:27.689807  6521 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1752) took 2226 ms (client timeout 1572 ms). Trace:
W20260709 13:17:27.690666  6521 rpcz_store.cc:269] 0709 13:17:25.463534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.576991 (+113457us) service_pool.cc:224] Handling call
0709 13:17:27.689086 (+2112095us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b540007dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:27.689418 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3976293120}
I20260709 13:17:27.696031  7200 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:27.660415  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.661921  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:27.697944  6597 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:27.698082  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.662570  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:27.668511  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2170) took 1518 ms. Trace:
I20260709 13:17:27.676479  6589 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:27.678601  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.460376  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.684828  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 823) took 1866 ms. Trace:
I20260709 13:17:27.659399  6593 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:27.696277  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:27.642467  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.709954  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.710242  8804 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:27.712098  8856 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:27.699829  6590 rpcz_store.cc:276] 0709 13:17:26.149422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.686801 (+537379us) service_pool.cc:224] Handling call
0709 13:17:27.667922 (+981121us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540019e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:27.668187 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550232320}
I20260709 13:17:27.715215  9002 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:27.699290  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.701670  7232 rpcz_store.cc:276] 0709 13:17:25.817736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.004757 (+1187021us) service_pool.cc:224] Handling call
0709 13:17:27.005039 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.326754 (+321715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.684119 (+357365us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54005a3554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:27.684537 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677318528}
I20260709 13:17:27.660416  6636 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:27.716269  9002 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:17:27.699167  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.723315  8787 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.510097  8865 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.715260  8563 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:27.726382  8563 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:27.728050  8563 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.481145  8827 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.735227  7234 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:27.736608  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.737768  7234 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:27.739097  7230 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:27.740429  8976 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.741282  8976 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:27.742286  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46229.8us
W20260709 13:17:27.739118  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76480.8us
I20260709 13:17:27.743618  8835 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.746747  8899 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:27.747062  9022 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:27.747289  9006 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:27.749244  9006 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:27.749782  9006 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:27.751206  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.738415  8989 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.752985  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.753914  7236 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:27.754088  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:27.734891  8960 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:27.755314  7237 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:27.756721  9016 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.760577  6592 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:27.762295  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.761904  8960 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.747424  8899 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:27.776769  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.774549  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.746102  9004 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:17:27.777097  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:27.783319  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 123164us
I20260709 13:17:27.783330  7234 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:27.786577  6590 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:27.790315  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.737648  7229 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:27.814414  7232 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.817288  8787 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:27.817845  8787 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.818279  8787 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.813762  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:27.822804  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68908.8us
I20260709 13:17:27.823992  6597 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:27.824100  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.825132  7236 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:27.825070  9013 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:27.826730  8840 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:17:27.826753  8660 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.753s	user 0.013s	sys 0.001s
I20260709 13:17:27.827888  8660 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:27.829545  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 167285us
I20260709 13:17:27.830276  8660 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.830498  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:27.831146  7228 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:27.831497  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.832526  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.826314  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.833834  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.833834  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.833947  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.835287  6590 raft_consensus.cc:2376] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3. Re-sending same reply.
I20260709 13:17:27.835495  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.835983  8827 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.778239  8899 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.764715  9016 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.839619  6597 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:27.841665  6595 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:17:27.842048  6595 pending_rounds.cc:85] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 6
I20260709 13:17:27.842274  6595 pending_rounds.cc:107] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:17:27.842597  6595 raft_consensus.cc:2889] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.836027  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.847358  7236 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 1.
I20260709 13:17:27.847884  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.848555  7234 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:27.850339  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:27.851455  9013 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:27.851874  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:27.852823  8840 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:27.853251  8840 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.853411  9013 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:27.853814  8840 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:27.853952  8927 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:27.854475  8927 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.854836  8927 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.857453  9013 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:27.857807  9013 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:27.858776  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.017s	sys 0.003s
W20260709 13:17:27.859182  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.018s	sys 0.003s
I20260709 13:17:27.859131  7236 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:27.858942  9013 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:27.859699  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 555611us
I20260709 13:17:27.860448  9013 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.860561  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:27.860939  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:27.861725  9036 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:27.862254  9036 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:27.862738  9036 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.864060  8976 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.864686  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.865438  7235 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:27.868002  9020 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:27.868530  9006 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.869367  9034 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:27.869822  9034 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:27.870148  9006 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:27.871071  9034 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.872575  9034 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.873991  8771 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:27.877055  8911 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:27.879032  6592 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:27.879387  8987 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.879195  8840 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:27.880079  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 184510us
I20260709 13:17:27.884510  8960 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.884563  8787 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.887185  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2181) took 3241 ms. Trace:
I20260709 13:17:27.887246  8787 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.887655  6519 rpcz_store.cc:276] 0709 13:17:24.645330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.083971 (+1438641us) service_pool.cc:224] Handling call
0709 13:17:26.113040 (+ 29069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.259844 (+1146804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.886751 (+626907us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b54002157d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:27.887014 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2292110720}
I20260709 13:17:27.889639  8840 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.813333  8804 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:27.836251  6596 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:27.852999  8744 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.905642  7232 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:27.906476  7234 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
W20260709 13:17:27.906925  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.913923  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:27.916996  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 86663.5us
I20260709 13:17:27.918711  7232 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:27.923895  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:27.926265  8856 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:27.928646  8913 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:27.869736  8949 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:27.931073  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:27.942749  6561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.301s (SENT) [suppressed 36 similar messages]
I20260709 13:17:27.943333  8787 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:27.943800  8787 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.944178  8787 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.945039  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:27.947088  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.948957  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.950313  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.951141  6670 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:27.951628  8827 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.952026  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.952832  6663 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:27.953823  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.954591  6668 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:27.955138  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.972635  8832 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:27.973147  6667 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:27.973515  7229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 741) took 2569 ms (client timeout 1647 ms). Trace:
W20260709 13:17:27.973914  7229 rpcz_store.cc:269] 0709 13:17:25.403536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.992628 (+589092us) service_pool.cc:224] Handling call
0709 13:17:27.056446 (+1063818us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:27.967106 (+910660us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400575dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:27.973428 (+  6322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279096832}
I20260709 13:17:27.957417  6596 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:17:27.933027  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:27.958950  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:27.963059  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.919554  8804 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.976625  7237 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:27.968286  7234 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:17:27.959126  7200 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:27.959738  7230 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:27.980172  6667 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:17:27.969023  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:27.969139  6488 leader_election.cc:400] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5.
I20260709 13:17:27.984582  6488 leader_election.cc:403] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:17:27.983079  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:27.968027  9015 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:27.967680  8913 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:27.956248  8952 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.994410  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.991868  9015 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:27.957676  8927 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:28.002950  8999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.040s	sys 0.052s
I20260709 13:17:28.005507  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:27.960714  8787 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:27.965790  8990 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:17:28.000258  8563 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.007949  8949 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:28.016368  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47435.9us
I20260709 13:17:28.000411  8832 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:17:28.014470  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 82770.5us
I20260709 13:17:28.008819  9015 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:28.015934  8992 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:28.014494  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:28.011060  8999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.040s	sys 0.053s
I20260709 13:17:28.014901  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.022356  6667 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:28.030810  8563 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.031601  8827 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:28.033219  8563 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.034864  8979 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:17:28.035910  6518 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:28.037209  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:28.038718  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:28.038367  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:28.039543  6516 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:28.041095  6514 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:28.042840  7230 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:28.025147  8976 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:28.044955  8976 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.045327  8976 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:28.053649  6667 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:28.055572  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:28.060480  6670 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:28.060626  6521 raft_consensus.cc:2368] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7.
I20260709 13:17:28.061230  6518 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:28.061566  8999 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:28.055513  8979 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 13:17:28.079360  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 55150.3us
W20260709 13:17:28.082401  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.092782  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:28.103664  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:28.114348  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 187820us
I20260709 13:17:28.118053  6670 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:28.122449  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:28.123212  8563 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:28.123602  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:28.124380  8563 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:28.124818  8744 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:28.125464  8744 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:28.125478  8827 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:28.126436  8827 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:28.126852  8744 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.127427  8827 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.129201  8744 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:28.124914  8563 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:28.133231  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 78990.3us
I20260709 13:17:28.078161  8999 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:17:28.135583  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:28.119925  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:28.134549  9017 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.138024  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:28.138538  8913 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:28.139037  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:28.140686  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:28.141940  6664 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:28.143590  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.119722  8832 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:28.145291  8832 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:28.145787  8832 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:28.097740  8903 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.143406  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.137372  9017 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.154076  6667 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:28.181169  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.139079  8913 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:28.193100  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79597.6us
W20260709 13:17:28.197490  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.192514  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:28.186757  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:28.192122  6521 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:28.200510  6670 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:28.177467  9006 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:28.202908  6662 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:28.174785  8804 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.215488  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:28.181178  8960 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.215636  7237 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:28.152441  6415 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: config changed from index 3 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:28.180160  8999 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.222020  6514 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:28.219640  8987 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:28.226869  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.192829  8990 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:28.228586  6637 leader_election.cc:400] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:28.229995  7234 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:28.216447  8927 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:28.236471  6637 leader_election.cc:403] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:17:28.213850  8787 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.229144  8903 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:28.237187  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:28.229074  8773 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:28.238996  7231 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:28.239274  8773 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.239703  8773 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:28.240893  9048 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:17:28.232957  8987 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.244726  8987 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:28.247452  8773 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.230700  8960 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.194733  8856 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 13:17:28.262655  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 81155.5us
I20260709 13:17:28.233470  9006 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:28.277261  9011 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:28.282907  6415 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 5, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.285226  6415 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:17:28.297935  6415 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 5, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.304675  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.305469  6664 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:28.306051  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 90928.7us
W20260709 13:17:28.307549  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 70460.8us
I20260709 13:17:28.307245  6662 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:28.261826  8999 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.311102  6669 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:17:28.311440  6520 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:28.311955  8778 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:28.312629  9058 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:17:28.312553  8778 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.314406  7233 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:28.314906  8778 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.315898  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:28.316151  6666 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:28.316896  9052 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:17:28.274395  8990 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:28.319543  8990 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.320574  8778 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:28.321385  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:28.322232  8778 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:28.322144  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:28.322443  8979 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:28.322887  8778 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.323069  6521 raft_consensus.cc:606] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:28.323280  8778 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:28.324203  6670 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:17:28.325251  9047 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:17:28.325492  9006 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.330317  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:28.332914  9021 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:28.333024  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.335776  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:28.335942  9011 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:28.336555  9011 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:28.337028  8992 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5.
I20260709 13:17:28.337522  8927 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:28.339427  6518 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:28.312889  8999 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:28.342209  6663 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:28.318418  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:28.233031  8563 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.311460  6669 pending_rounds.cc:85] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 1
I20260709 13:17:28.347440  6669 pending_rounds.cc:107] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:28.347708  6669 raft_consensus.cc:2889] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:28.286293  8832 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.338501  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.355464  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.355726  7231 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:28.355999  8865 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.358690  9020 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:28.360210  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:28.361145  6670 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:28.363026  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:28.363780  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:28.331993  9015 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.365052  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:28.265976  8804 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.297055  8911 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:28.370159  6668 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:28.342173  6563 leader_election.cc:400] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7.
I20260709 13:17:28.390201  6563 leader_election.cc:403] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:17:28.342934  6415 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 4, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.342532  8927 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:28.374022  8827 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:17:28.375901  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57340.6us
I20260709 13:17:28.379272  6664 raft_consensus.cc:2368] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7.
W20260709 13:17:28.382431  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54564.7us
I20260709 13:17:28.385735  7237 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:28.364233  9034 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:28.309509  8903 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:28.337581  9011 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.387991  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.366786  9013 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:17:28.392596  7236 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:28.389339  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:28.411979  7230 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:28.334797  8778 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.393337  6415 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 4, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.299340  8989 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:17:28.393921  8773 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.405934  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.408052  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.414767  7233 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 2: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:28.388870  6514 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:28.413416  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.413547  8927 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.413995  8975 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:17:28.404836  9006 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.402997  6671 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:28.350173  9016 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.462970  8778 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.367973  8913 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.438318  6667 raft_consensus.cc:2368] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
W20260709 13:17:28.429381  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.414687  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.475319  6663 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:28.476797  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 102038us
W20260709 13:17:28.418010  9036 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.000s
I20260709 13:17:28.400441  9017 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:28.384770  8987 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.462277  8804 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:28.438243  8899 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.003s
W20260709 13:17:28.436885  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44275.7us
I20260709 13:17:28.447465  6415 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 3, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.413988  8903 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:28.464200  8999 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:17:28.487130  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.443784  9013 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:17:28.499282  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.532193  6663 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:28.499449  8778 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7.
I20260709 13:17:28.501626  9064 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:28.523044  9060 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.523298  8840 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.530450  7237 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:28.405774  8865 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.500344  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.476765  8832 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:28.443565  8911 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.537388  6663 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:17:28.464216  9002 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:17:28.475692  8992 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
W20260709 13:17:28.544039  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 49165us
I20260709 13:17:28.470134  9006 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.560626  6415 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 4, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.566920  6667 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:28.526729  8804 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:17:28.527014  8899 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.504551  9036 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.559854  8960 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:28.553917  8976 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.006s
W20260709 13:17:28.569100  8949 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.544s	user 0.005s	sys 0.005s
W20260709 13:17:28.589339  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49089.7us
W20260709 13:17:28.590353  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 267758us
W20260709 13:17:28.589962  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 240879us
I20260709 13:17:28.590564  6415 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 6, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 6 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:28.551543  9013 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:28.586556  6664 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:17:28.577222  7228 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:28.631943  6664 pending_rounds.cc:85] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 5
I20260709 13:17:28.632251  6664 pending_rounds.cc:107] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:17:28.632474  6664 raft_consensus.cc:2889] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:28.600431  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 153496us
W20260709 13:17:28.633011  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.633581  6667 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:28.633766  6636 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:28.585259  8999 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.645539  6589 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:28.646194  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.643579  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:28.648924  6667 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:28.582713  8778 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:28.653411  6670 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:28.551331  9060 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.641620  8949 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:28.665462  7236 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:28.668313  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:28.647588  8832 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.671132  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:28.651968  8960 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:28.680423  8960 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:28.681563  9036 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.681468  8960 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.682097  8975 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:28.683339  9020 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:28.599284  8840 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.647787  9015 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.683691  8975 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:28.713493  9060 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:28.703712  8832 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.660912  8999 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:28.632560  8976 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.745080  6667 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:17:28.749117  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 78061.4us
I20260709 13:17:28.751232  7236 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:17:28.751574  9065 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:17:28.751878  9080 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:28.747303  9071 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.753273  8840 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:28.753849  9050 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:17:28.770968  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.673959  9013 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.780337  9013 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.703907  8778 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:28.764117  9063 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:17:28.759580  8865 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.657737  8903 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.783280  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:28.787712  9056 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:17:28.764880  9080 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:28.778398  8987 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.790357  6513 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
W20260709 13:17:28.868453  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 86975us
W20260709 13:17:28.788045  9061 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 13:17:28.886981  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 108615us
W20260709 13:17:28.891813  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:28.894524  6487 leader_election.cc:336] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.588s (SENT)
I20260709 13:17:28.857528  8865 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.841300  8832 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:28.857245  6514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
W20260709 13:17:28.808350  9068 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 13:17:28.884286  9020 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.963264  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:28.929822  6487 leader_election.cc:336] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.603s (SENT)
I20260709 13:17:28.967749  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:28.784415  9015 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 13:17:28.789738  9071 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:28.977913  9082 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
W20260709 13:17:28.914408  9072 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:17:28.917158  6514 raft_consensus.cc:606] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:28.909789  8771 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.985306  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 1993) took 1043 ms. Trace:
I20260709 13:17:28.987383  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.927412  8899 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:28.926367  8999 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:28.975328  9058 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.004s
I20260709 13:17:28.974498  9021 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.987785  6669 rpcz_store.cc:276] 0709 13:17:27.936844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.948112 (+ 11268us) service_pool.cc:224] Handling call
0709 13:17:27.948419 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:28.404677 (+456258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.980427 (+575750us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540011e9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:28.980602 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":795636608}
I20260709 13:17:28.915650  8912 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:17:28.980526  9020 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.999352  9021 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.000949  9020 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:29.001506  9021 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:29.002244  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72473.3us
I20260709 13:17:29.002705  9089 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:29.002978  6487 leader_election.cc:336] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.647s (SENT)
I20260709 13:17:29.003404  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.004010  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:29.004644  6487 leader_election.cc:336] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.683s (SENT)
I20260709 13:17:29.005154  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:29.005584  6487 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.748s (SENT)
I20260709 13:17:29.006075  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.006856  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:29.007476  6667 raft_consensus.cc:2376] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4. Re-sending same reply.
I20260709 13:17:29.008350  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:29.009065  6667 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:29.010360  6667 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: [3.6-4.7]
I20260709 13:17:29.011080  8976 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:29.011674  9054 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.645s	user 0.007s	sys 0.000s
I20260709 13:17:29.011847  8912 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:29.012168  9071 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.006304  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:28.926569  8865 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:28.988579  6514 raft_consensus.cc:614] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:29.016775  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 843) took 1145 ms. Trace:
W20260709 13:17:29.018148  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50329.9us
I20260709 13:17:28.914197  8778 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.019881  9089 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:29.024151  6664 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:29.032948  8865 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:29.034171  8744 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.904s	user 0.014s	sys 0.001s
I20260709 13:17:29.015463  8912 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:29.035933  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 280389us
I20260709 13:17:29.035718  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:29.037045  7232 rpcz_store.cc:276] 0709 13:17:27.871176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.931314 (+ 60138us) service_pool.cc:224] Handling call
0709 13:17:27.985557 (+ 54243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:28.003196 (+ 17639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:28.003231 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:28.021116 (+ 17885us) log.cc:844] Serialized 26 byte log entry
0709 13:17:28.184226 (+163110us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:28.184281 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.184521 (+   240us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:29.010980 (+826459us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:29.014172 (+  3192us) raft_consensus.cc:1685] finished
0709 13:17:29.014196 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:29.016684 (+  2488us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5247,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":81268480,"thread_start_us":150185,"threads_started":1,"wal-append.queue_time_us":306620}
I20260709 13:17:29.021030  9021 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:29.073376  8903 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:29.074306  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:29.076421  8912 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.077045  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:29.077832  8912 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:28.992892  8771 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.079620  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:28.915127  9080 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.085309  8832 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:29.079049  8778 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.091905  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.093552  6669 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:29.094666  6594 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:29.104168  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.087308  9020 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:29.110594  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:29.114151  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.114471  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.115309  6596 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:29.119189  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 463) took 1070 ms. Trace:
I20260709 13:17:29.119596  6671 rpcz_store.cc:276] 0709 13:17:28.048589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.048704 (+   115us) service_pool.cc:224] Handling call
0709 13:17:28.057489 (+  8785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:28.462717 (+405228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:29.118962 (+656245us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540012a054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:29.119126 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793772288}
I20260709 13:17:29.120740  6519 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:29.122146  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.118377  9016 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.127502  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:29.128118  6663 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:29.129055  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:29.130015  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:29.130931  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.131624  6593 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:29.111150  9015 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.132453  9015 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.132846  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:29.107219  8960 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:17:29.112290  8903 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:17:29.129302  6669 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:29.140874  6670 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:29.144443  6663 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:17:29.134426  9015 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:29.110647  9020 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:29.108788  9103 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:29.152804  6663 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:29.111126  8778 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:29.156605  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49494.1us
I20260709 13:17:29.139076  6514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:29.158388  6670 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:29.160226  6487 leader_election.cc:336] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.112s (SENT)
W20260709 13:17:29.146106  8979 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.012s	sys 0.012s
I20260709 13:17:29.151758  9021 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.161193  8976 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:29.161898  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.157673  8899 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:17:29.149412  9020 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.157846  8744 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:29.163559  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.150334  9103 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.169253  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.151459  9098 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:17:29.191903  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1777) took 1010 ms. Trace:
I20260709 13:17:29.192257  6519 rpcz_store.cc:276] 0709 13:17:28.181752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.181878 (+   126us) service_pool.cc:224] Handling call
0709 13:17:29.191745 (+1009867us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54002161d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:29.191839 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001175936}
I20260709 13:17:29.196878  8913 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:29.199892  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.202807  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:29.203596  7228 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 2: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:29.173195  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.206593  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:29.158828  9080 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:29.173489  9021 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.175033  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:29.175544  8744 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.360400  8744 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.177593  9078 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.361255  9078 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.363397  9078 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.178037  9097 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.364104  9097 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.178959  9011 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.366817  9011 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:29.367676  9011 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.368703  9011 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.179245  8771 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.179594  9103 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:29.181171  9018 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:29.374768  9018 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:29.375734  9018 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.376713  9018 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:29.182536  9013 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:29.378782  9013 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:29.184958  9006 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:29.187542  8778 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:29.190609  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.366339  9097 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:29.380498  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:29.380996  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.157507  9071 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:17:29.191222  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1768) took 1156 ms. Trace:
I20260709 13:17:29.381675  6513 rpcz_store.cc:276] 0709 13:17:28.035222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.036880 (+  1658us) service_pool.cc:224] Handling call
0709 13:17:29.191047 (+1154167us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54003f6bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:29.191147 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771446144}
I20260709 13:17:29.382798  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.383955  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.313500  6666 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:29.385810  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1683) took 1053 ms. Trace:
I20260709 13:17:29.386137  6666 rpcz_store.cc:276] 0709 13:17:28.332042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.338249 (+  6207us) service_pool.cc:224] Handling call
0709 13:17:29.385627 (+1047378us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54000e60d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:29.385733 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2000677120}
I20260709 13:17:29.310266  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.387274  6596 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.304800  6593 raft_consensus.cc:2410] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:29.392025  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.393332  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.395457  6664 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:29.315214  6670 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:29.315425  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.314177  9060 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.397563  9060 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.397939  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.316437  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.398638  6594 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:29.398888  6590 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.399889  6594 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:29.310688  8856 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:29.402915  8856 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:29.402993  9097 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.403501  9097 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.403836  8856 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.405071  8856 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:29.317400  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:29.317437  8911 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.311223  6520 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:29.407028  6597 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:29.407164  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1782) took 1091 ms. Trace:
I20260709 13:17:29.407486  6520 rpcz_store.cc:276] 0709 13:17:28.315383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.348425 (+ 33042us) service_pool.cc:224] Handling call
0709 13:17:29.215944 (+867519us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b540007de28. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee5e924 00007f04be047227 00007f04be046ab8 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:29.407110 (+191166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14790400}
I20260709 13:17:29.408458  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.408974  6520 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:29.409812  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.411037  6520 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:29.411219  9097 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.410341  6594 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:29.306878  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 459) took 1332 ms. Trace:
I20260709 13:17:29.317662  9020 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.165594  9015 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:29.170421  6514 raft_consensus.cc:606] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:29.412487  6668 rpcz_store.cc:276] 0709 13:17:27.975096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.978809 (+  3713us) service_pool.cc:224] Handling call
0709 13:17:29.216307 (+1237498us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b540005a854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:29.306789 (+ 90482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25248128}
I20260709 13:17:29.178543  8660 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:17:29.186360  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 601) took 1353 ms (client timeout 1736 ms). Trace:
I20260709 13:17:29.221067  8773 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:29.414994  8773 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:29.414916  6589 rpcz_store.cc:269] 0709 13:17:27.832552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.832682 (+   130us) service_pool.cc:224] Handling call
0709 13:17:29.186148 (+1353466us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54002f5dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:29.186270 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861105024}
I20260709 13:17:29.319243  7231 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:29.415997  8773 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.417157  8773 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:29.418426  9020 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.320734  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:29.419056  7228 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:29.320809  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:29.419993  7234 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:29.321285  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:29.422935  7232 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:29.321904  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:29.321671  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:29.414386  8660 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:29.322822  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.304577  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:29.425024  8660 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:29.425720  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 253562us
I20260709 13:17:29.426170  8660 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:29.426414  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:29.329624  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.330528  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.331123  6595 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:29.331753  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.332917  6515 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:29.333977  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.335063  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.335738  9036 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:29.336687  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.394449  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.337082  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:29.337739  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 134064us
I20260709 13:17:29.339064  8899 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:29.339248  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.341670  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.342492  9069 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.342779  8912 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.307713  8976 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:29.304152  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 152206us
I20260709 13:17:29.343359  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.344488  6588 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:29.355581  6665 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:29.307065  9021 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.412734  9060 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.320149  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:29.423892  7236 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.427083  6519 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.428577  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:29.429463  6518 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:29.433465  6590 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
W20260709 13:17:29.438558  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101572us
I20260709 13:17:29.573959  6520 raft_consensus.cc:1217] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Deduplicated request from leader. Original: 1.1->[3.2-3.2]   Dedup: 3.2->[]
I20260709 13:17:29.559877  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:29.166018  8979 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.013s	sys 0.012s
W20260709 13:17:29.505079  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.601747  7230 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:29.501644  6521 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
W20260709 13:17:29.505162  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 330210us
I20260709 13:17:29.504782  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:29.508152  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 81814.7us
I20260709 13:17:29.501303  8744 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:29.510676  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:29.512732  6516 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:29.509442  9015 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:29.504729  9077 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:29.506021  8563 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.508409  9020 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.518148  7232 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:29.507977  8840 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:17:29.519062  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.559003  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:29.601183  6520 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:29.634493  7230 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 13:17:29.520749  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 181562us
I20260709 13:17:29.441306  7234 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:29.603824  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:29.556619  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.499747  6596 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
W20260709 13:17:29.606436  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.556357  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:29.637267  7229 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:29.602850  6589 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:29.638118  8913 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:29.638775  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1684) took 1306 ms. Trace:
I20260709 13:17:29.633447  8563 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.560642  7235 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:17:29.633042  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 74135.7us
I20260709 13:17:29.509056  9103 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.600824  6591 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:29.622958  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 194752us
W20260709 13:17:29.609251  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.630363  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:17:29.628507  6487 leader_election.cc:336] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.113s (SENT)
I20260709 13:17:29.519644  9069 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:29.607352  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:29.636893  8903 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:29.641223  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:29.639245  6665 rpcz_store.cc:276] 0709 13:17:28.332386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.332593 (+   207us) service_pool.cc:224] Handling call
0709 13:17:29.638610 (+1306017us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000cbf54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:29.638723 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338544896}
I20260709 13:17:29.631099  8992 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:29.521412  8899 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:29.644578  7237 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:29.559484  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:29.656064  6519 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:29.646982  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.649724  7232 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:29.607553  8744 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.584161  9036 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:29.653188  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45902.2us
I20260709 13:17:29.633822  8840 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.639262  8913 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:29.655932  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:29.546514  8912 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.660768  9021 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:29.646076  8856 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:17:29.560464  9013 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:29.680366  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51904.6us
I20260709 13:17:29.686707  9036 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.687515  8773 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:29.699774  8773 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:29.652122  8992 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.608201  8976 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.679796  8563 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:29.697551  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.667441  8989 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:29.667730  6669 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:17:29.660923  8744 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:29.702309  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46491.2us
I20260709 13:17:29.639675  9020 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.666921  8913 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:29.666843  8840 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.709707  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.675953  8949 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.694218  8899 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.696364  9021 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.717666  9013 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:29.717957  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.654666  9002 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:17:29.705379  8773 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.709259  8992 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:29.707088  9080 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:29.740379  8899 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:29.743808  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 102636us
I20260709 13:17:29.740264  9021 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:29.763835  8976 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.758706  8949 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.769142  9036 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.763654  8912 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.767980  9020 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.715617  8660 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:29.752079  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 148322us
W20260709 13:17:29.794698  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 125947us
I20260709 13:17:29.776145  7236 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:29.781406  9103 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.783135  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:29.737970  8563 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:17:29.785110  6513 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:17:29.787310  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 77714.4us
W20260709 13:17:29.803735  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:29.805472  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.817308  9121 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:29.817922  9121 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.818307  9121 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:29.818578  9002 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.819200  9002 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.821444  8949 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.784168  8911 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:29.825632  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 171989us
W20260709 13:17:29.835420  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.826399  6563 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3.
I20260709 13:17:29.740989  9069 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.795040  9036 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:29.859990  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:29.827760  8912 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:29.860307  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:29.856722  8911 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:29.826825  8903 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:29.821866  9020 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:17:29.867733  6520 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:29.868940  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.822584  8773 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.866840  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.878413  6521 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:29.883869  6518 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:29.885262  9126 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:17:29.885787  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.886510  9002 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:29.887108  9002 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:29.887498  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:29.888433  9002 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:29.888824  9002 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.888957  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.889240  9002 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:29.890833  9015 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:29.861076  8979 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:17:29.891315  9015 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.885506  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:29.892874  6520 raft_consensus.cc:2376] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4. Re-sending same reply.
W20260709 13:17:29.907269  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 108955us
I20260709 13:17:29.914150  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.879995  9069 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.920706  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.891726  9015 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:29.938659  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 142291us
I20260709 13:17:29.884498  8912 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:29.946122  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.949509  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.875672  8827 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:17:29.874974  9123 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 13:17:29.954105  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52792.5us
I20260709 13:17:29.914898  8949 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:29.956791  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:29.958261  8949 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:29.913179  9128 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:29.883107  9036 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.959414  8949 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.960240  8773 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.950341  6589 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:29.961767  9036 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:29.958408  6588 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:29.957233  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:29.964308  8840 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.965163  8979 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.948093  8911 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.968547  9004 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.969280  9004 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.969883  8744 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:29.971638  9004 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:29.974104  7200 connection.cc:256] RPC call timeout handler was delayed by -1.2210280889999012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 13:17:29.974890  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.977624  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.981570  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:29.982650  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.014649  6519 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:29.983274  6514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:29.984753  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:29.986253  8979 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:29.986855  8903 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:29.988938  9018 raft_consensus.cc:993] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:29.991438  6522 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:29.993377  8856 raft_consensus.cc:993] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:29.994439  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:29.996050  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.005340  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.059367  6513 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:30.005774  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.006419  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:29.941622  9126 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:30.007870  6518 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
W20260709 13:17:30.017012  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.030660  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:29.978145  8992 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:30.050163  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65422.5us
I20260709 13:17:29.955374  9020 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:30.041183  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.054975  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.055919  6515 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:17:30.052789  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 200778us
I20260709 13:17:30.060237  6668 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:30.060295  6664 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:30.066305  6670 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:30.062001  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.068651  6520 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:30.070174  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:30.073783  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.042894  9078 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:30.074517  6519 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:30.073122  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.071089  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.067893  6663 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:30.051322  8773 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.074859  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.076522  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.087281  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.088433  6520 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:30.088439  6513 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.090183  6518 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:30.091969  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:30.097450  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:30.069666  9134 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:30.099514  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 220722us
I20260709 13:17:30.056384  6514 raft_consensus.cc:606] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:17:30.102877  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.067098  8563 raft_consensus.cc:1081] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:30.104164  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.104951  6518 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.100088  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:30.105389  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.097594  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.106256  6517 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:30.067363  9006 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.107914  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:30.095486  9002 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.067579  9020 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:30.109365  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.125222  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.125847  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.111271  6519 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 3: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:30.129287  6522 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:30.110276  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 128765us
I20260709 13:17:30.072649  9018 raft_consensus.cc:1081] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:17:30.092265  8827 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.053732  9121 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.102125  9134 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.102420  6514 raft_consensus.cc:614] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:30.139258  6594 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:30.136409  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.140481  6517 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:30.141321  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.109084  8912 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.096128  8773 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.131549  9078 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:30.147876  6519 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:30.096978  8989 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:17:30.078581  8913 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.182150  6518 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:30.183805  8840 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:30.184855  9121 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.187070  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.188050  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:30.138778  9134 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:30.194108  6518 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:30.130836  9006 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.163061  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:30.118952  8660 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:30.103120  9103 raft_consensus.cc:2764] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:17:30.169436  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.169533  6513 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:30.177309  8979 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:30.202783  8856 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:30.205715  8899 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:30.206822  6588 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:17:30.324792  6588 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:30.325162  6588 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:30.208180  9002 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:30.143307  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 129350us
I20260709 13:17:30.140714  8827 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.304745  6522 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:30.306340  8840 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.144104  9128 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:17:30.192090  9139 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:30.310791  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 100575us
I20260709 13:17:30.342499  6515 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:30.210572  9004 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.315965  6513 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:30.214699  9078 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:30.320387  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.326982  6518 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:30.328650  6520 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:30.301352  9138 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:30.211647  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:30.372759  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 161200us
I20260709 13:17:30.335587  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:30.311748  9077 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:30.322851  9132 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 13:17:30.261798  9021 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.317126  9123 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.327807  9060 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.370719  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 270970us
W20260709 13:17:30.380824  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 45342.3us
I20260709 13:17:30.317494  8660 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.321966  8771 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:17:30.392199  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:30.393677  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.369665  9144 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.395573  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.371117  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:30.398484  6513 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:30.399273  6520 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:30.374818  9103 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.372151  9121 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:30.368880  8992 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.377648  9002 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:30.308642  9020 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.415475  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.379870  9071 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:30.410020  6521 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:30.419854  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:30.343743  8840 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.416095  6522 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:30.408985  8899 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:30.478993  6513 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:17:30.370177  9139 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.379631  9004 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.492767  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:30.440115  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 277133us
I20260709 13:17:30.377434  9138 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.397225  8744 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.512291  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:30.513418  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.515882  6522 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:30.415715  9002 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.409181  9013 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 13:17:30.418195  6514 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:30.466957  9103 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.514549  8563 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.419214  9121 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.475410  9060 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:30.529834  9015 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.003s
I20260709 13:17:30.564023  6513 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:30.405341  9123 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.409411  9144 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.531411  9071 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:30.555500  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.539197  9002 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.555843  8773 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:30.594342  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.568645  9020 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.595690  6520 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:30.597644  6518 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:30.558734  9126 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:30.599308  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 205247us
I20260709 13:17:30.582018  8827 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.601933  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:30.591738  9071 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:30.606814  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 103510us
I20260709 13:17:30.607609  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:30.591543  9006 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:30.610054  6591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.006s	sys 0.009s
W20260709 13:17:30.612363  6591 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.006s	sys 0.009s
W20260709 13:17:30.613212  9141 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:17:30.613867  9060 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.614646  8660 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:30.615082  9140 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:17:30.615433  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:30.616004  8840 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:30.616447  9150 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:17:30.622233  9077 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:30.622813  9077 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:30.623114  9146 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:17:30.623837  9123 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.605805  8773 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:17:30.627455  9138 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:30.611074  9071 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.627933  8992 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:30.628643  9004 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:30.628906  9021 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.629884  9103 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.609862  9155 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.631258  9139 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:30.631270  9155 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:30.634300  9069 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.685s	user 0.006s	sys 0.003s
I20260709 13:17:30.635191  9155 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.635440  9002 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:30.628994  9142 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.237s	user 0.000s	sys 0.003s
I20260709 13:17:30.637818  9071 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.643152  9126 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.645287  8979 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:30.645747  8979 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.646286  9071 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:30.647228  9144 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.647675  9148 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:30.654670  8949 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:30.654981  9148 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.655144  8949 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:30.655615  9148 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:30.656106  8949 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.657060  8949 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.648316  9002 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:30.679759  6487 leader_election.cc:336] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.630s (SENT)
I20260709 13:17:30.681001  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:30.648710  9006 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:30.692591  9168 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:17:30.697474  9021 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:30.683331  9148 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.648885  9077 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.557605  9078 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.697526  9006 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:30.704698  9006 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.588770  9015 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.708276  9015 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:30.567111  8563 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.710435  8563 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:30.651508  9121 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:30.710882  9137 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.211s	user 0.002s	sys 0.000s
W20260709 13:17:30.713539  9147 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:17:30.651262  6588 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.714993  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.715621  6518 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:30.716912  6588 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:30.675457  8913 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:30.655036  8911 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:30.692942  8899 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:30.696307  9134 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:30.723706  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.701087  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 281273us
I20260709 13:17:30.717063  6520 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:30.593529  9122 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:30.741689  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.742503  6522 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:30.742579  6513 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:30.743000  6522 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:30.758035  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:30.758885  6518 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:30.760221  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.765856  6591 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:17:30.744143  6522 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.766597  6591 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:30.769299  6591 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.766917  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:30.770463  6516 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.768699  6522 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:30.738282  9149 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:30.744454  9122 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.780496  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.781262  6596 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.763996  9036 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.758925  8911 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:30.803526  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:30.804365  6594 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:30.796404  6522 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:30.797622  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.808966  6588 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:30.763651  9172 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:30.675535  8979 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:30.827625  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.833707  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.838158  6596 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:30.789459  9148 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.843123  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.843961  6595 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:30.850241  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:30.851248  6596 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
W20260709 13:17:30.859217  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 366507us
I20260709 13:17:30.859484  6594 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:30.858240  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.789273  9149 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.864125  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:30.869426  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:30.862214  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 250208us
I20260709 13:17:30.870908  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:30.804828  9036 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:30.875913  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.878544  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:30.879650  6665 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:30.879988  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:30.880829  6596 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
W20260709 13:17:30.883230  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 175251us
I20260709 13:17:30.883477  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.863454  8563 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.802037  9122 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.886471  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.884415  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:30.886694  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.859948  8976 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:17:30.865113  9167 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.893901  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:30.900352  6666 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:30.892601  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:30.900995  6667 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:30.894481  6487 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.786s (SENT)
I20260709 13:17:30.892031  6594 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.896312  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:30.901988  7236 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:30.897944  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:30.899756  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:30.898880  6588 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:30.901746  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:30.904284  6663 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:30.883104  9148 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:30.888497  9015 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:30.887086  9121 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:30.850749  9172 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.907095  9148 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:30.905827  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:30.832839  8899 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:30.907086  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:30.908903  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:30.925081  6670 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.909432  9160 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:17:30.906764  8989 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:17:30.904325  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:30.889712  8911 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.894307  8563 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:30.925268  7233 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:30.976359  9103 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:30.977067  9103 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:30.916723  9149 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:30.975099  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 70843.6us
I20260709 13:17:30.912006  6589 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:30.915405  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.927208  9148 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:30.927595  9126 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:17:30.970248  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.919587  6665 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:30.906989  9077 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:30.962554  6666 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
W20260709 13:17:30.983574  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49140.9us
I20260709 13:17:30.984714  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:30.974047  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79594.7us
W20260709 13:17:30.973913  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 72224.2us
I20260709 13:17:31.011411  6563 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.011786  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:30.988734  6667 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:30.995925  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.997900  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.012902  6588 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:31.013146  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:30.916842  9134 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:30.940510  9159 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:17:31.024362  6518 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:30.964879  9015 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:31.026341  6513 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:31.026222  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.030747  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.911309  9121 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:31.033631  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.031144  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:31.043164  6593 raft_consensus.cc:2368] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6.
I20260709 13:17:31.036300  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.030149  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:30.964637  9122 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:30.917721  9173 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:30.904052  9167 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.046172  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.041160  6595 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:31.051020  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:31.049213  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 150337us
I20260709 13:17:31.054965  6596 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:31.051532  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:30.938024  9128 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:17:31.025170  9078 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:30.962842  9004 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:17:30.978190  9103 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:31.070199  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.071826  6666 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:31.075749  6670 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:31.097915  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.075512  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.076104  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.071123  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:31.043536  9121 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.075783  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.037271  9123 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:17:31.085739  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.043352  9177 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:31.113226  6597 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:31.042711  9013 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:31.082912  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:31.093335  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 66688us
I20260709 13:17:31.100064  6667 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:31.100219  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 74093.8us
I20260709 13:17:31.075676  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.101924  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.102715  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.106688  7235 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:31.028229  9071 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:17:31.110688  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:31.086314  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 67999.5us
I20260709 13:17:31.137466  7236 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:31.137712  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.078167  8979 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:31.128139  9151 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.106050  9126 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:31.104790  9173 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:31.129225  9013 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.169378  9013 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:31.090858  9162 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:31.128047  9178 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.113205  9123 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.130447  9177 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:31.132228  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.130509  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.134891  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.135409  9158 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:17:31.137800  6667 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:31.071017  9122 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:31.120298  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.142060  6596 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:31.116979  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.088040  9078 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:31.149382  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50315.4us
W20260709 13:17:31.160498  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.135895  9155 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:17:31.161804  9174 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:31.151115  9128 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:17:31.163236  9167 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:31.164013  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.165200  6665 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:31.166205  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.105911  9134 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:31.161859  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.178809  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:31.069746  9175 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:17:31.076924  8912 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:17:31.187505  6516 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:31.192535  6591 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:31.239714  7236 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:31.213205  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:31.265281  6588 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.265501  6597 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:31.264667  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.266053  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1792) took 1543 ms. Trace:
I20260709 13:17:31.264319  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:31.257774  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:31.272562  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.318810  6590 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:31.257579  9123 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:31.286942  6669 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:31.253155  8903 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.180248  9162 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:31.339479  6591 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:31.340519  6515 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:31.299437  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.272943  9078 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.319468  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.346108  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.346758  6591 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:31.279337  6664 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:31.347633  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:31.297905  9184 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:17:31.357645  6596 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
W20260709 13:17:31.338523  8913 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.000s
I20260709 13:17:31.359651  6662 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:31.324604  6594 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:31.318383  6518 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:31.363073  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.254352  9036 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:31.337321  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.212687  9160 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.272606  7229 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.341351  6593 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:31.318308  6519 rpcz_store.cc:276] 0709 13:17:29.722193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.188057 (+465864us) service_pool.cc:224] Handling call
0709 13:17:30.283547 (+ 95490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.283783 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.283806 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.310169 (+ 26363us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.600047 (+289878us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:30.600095 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.600175 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:31.265751 (+665576us) raft_consensus.cc:1685] finished
0709 13:17:31.265781 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.265977 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":258625,"threads_started":1,"wal-append.queue_time_us":456510}
I20260709 13:17:31.288096  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:31.379914  6595 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.342338  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1794) took 1600 ms. Trace:
I20260709 13:17:31.344401  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.380482  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.381299  6597 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:31.344740  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:31.298657  9179 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 13:17:31.385401  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.348649  6591 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:31.276643  9174 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.288858  9004 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.387957  6588 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
W20260709 13:17:31.350127  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.273077  9122 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.296532  9176 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:31.377121  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.365093  9175 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:31.383850  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:31.398677  6516 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:31.399097  6516 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:31.400326  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.337769  9134 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:31.295329  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.345666  9172 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:31.178680  9015 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.274082  9178 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.392721  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:31.182394  9151 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.380545  6517 rpcz_store.cc:276] 0709 13:17:29.742375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.190185 (+447810us) service_pool.cc:224] Handling call
0709 13:17:30.285889 (+ 95704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.316709 (+ 30820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.316754 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.337810 (+ 21056us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.720925 (+383115us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:31.092470 (+371545us) log.cc:844] Serialized 14 byte log entry
0709 13:17:31.340125 (+247655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.340198 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:31.340240 (+    42us) raft_consensus.cc:1685] finished
0709 13:17:31.340252 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.342266 (+  2014us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97474,"prepare.queue_time_us":400835,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3645,"raft.queue_time_us":226442,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":6620,"thread_start_us":925398,"threads_started":3,"wal-append.queue_time_us":492801}
I20260709 13:17:31.407014  6591 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:31.409315  6595 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:31.390951  6596 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:31.412708  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:31.414179  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 304379us
I20260709 13:17:31.414644  6593 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:31.387158  9174 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.399966  6516 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:31.471758  6516 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.344923  8903 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:31.419993  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:31.466707  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.389654  9036 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:31.478900  9036 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.479286  9036 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:31.480635  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.401468  8563 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:31.485105  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.487741  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:31.488318  9071 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.387696  9160 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.502912  6589 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
I20260709 13:17:31.505285  6667 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:17:31.505501  6562 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:31.506038  6593 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:31.472339  8979 raft_consensus.cc:2764] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:17:31.496048  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 487) took 1508 ms. Trace:
W20260709 13:17:31.489403  9189 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:17:31.416935  9175 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:31.493978  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 222310us
I20260709 13:17:31.506404  6593 pending_rounds.cc:85] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 6
I20260709 13:17:31.515420  6593 pending_rounds.cc:107] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 5.7
W20260709 13:17:31.515691  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 375373us
I20260709 13:17:31.512382  6516 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:31.509748  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.515717  6593 raft_consensus.cc:2889] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:31.508136  6515 rpcz_store.cc:276] 0709 13:17:29.987211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.471016 (+483805us) service_pool.cc:224] Handling call
0709 13:17:31.340364 (+869348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.345144 (+  4780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.495792 (+150648us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54003bbad0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:31.495965 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930363136}
I20260709 13:17:31.494038  8913 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.400599  9004 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.501362  8912 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:31.508848  8744 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.010s	sys 0.001s
I20260709 13:17:31.536105  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.537103  7235 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:31.401307  9176 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.545913  6562 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:31.516990  9155 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:31.559438  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.549366  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.569482  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:31.533138  8744 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.010s	sys 0.001s
I20260709 13:17:31.516868  9170 raft_consensus.cc:2764] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:31.504941  9071 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.533679  8912 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.573647  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:31.578423  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 62810.2us
W20260709 13:17:31.571550  8949 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.914s	user 0.012s	sys 0.004s
I20260709 13:17:31.572264  9077 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:31.585875  7235 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:31.588943  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.491285  9177 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:31.598093  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 277169us
W20260709 13:17:31.644585  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 55695.7us
I20260709 13:17:31.645272  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:31.645509  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.568491  9155 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:31.645553  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:31.535257  8913 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:31.646432  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.655215  7232 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:31.655352  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.582355  9097 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:31.655972  6562 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:31.599326  9148 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
I20260709 13:17:31.580530  8912 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:31.584519  9126 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:31.667765  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 71885.3us
I20260709 13:17:31.575726  9199 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:31.670691  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:31.542744  9134 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.683718  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.686357  7232 raft_consensus.cc:2376] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 6. Re-sending same reply.
I20260709 13:17:31.653498  9018 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:31.672642  9199 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.687677  9199 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.688172  9126 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.690371  9176 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:31.692478  9176 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:31.693768  7237 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:31.673043  9148 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.695243  7236 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:31.696753  9173 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:31.726383  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 187074us
I20260709 13:17:31.673590  9171 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:31.658643  9204 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:31.667474  9097 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.736836  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:31.733448  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.652885  9077 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:31.760818  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.762687  6670 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:31.764066  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.708894  9176 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:31.769016  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.774959  6668 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:31.691325  8992 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:31.672640  9217 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:17:31.783084  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.726248  9006 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:31.684952  8827 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:17:31.679375  9060 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.098s
W20260709 13:17:31.798336  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61598.8us
I20260709 13:17:31.733755  8949 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.730098  9018 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:31.732414  9122 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:31.760684  9097 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:31.775113  9140 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:17:31.781419  9176 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.769809  9171 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:31.837957  6663 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:31.652760  9175 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:31.788865  9202 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 13:17:31.842854  9128 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.784247  8903 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:31.786213  9158 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:31.781437  9204 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:31.847796  9176 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:31.857299  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:31.848506  9208 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 13:17:31.849944  9013 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.002s
I20260709 13:17:31.837227  8744 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.871873  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:31.856104  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48377.4us
I20260709 13:17:31.855140  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.806468  9006 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.877698  9193 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:31.892285  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:31.761536  9155 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.852546  9151 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:31.843669  8949 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.883916  6588 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:31.813186  9170 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 13:17:31.891390  9078 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.001s
I20260709 13:17:31.896389  9018 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:31.898125  8949 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:31.897650  9204 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:31.898531  9158 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:31.894997  8744 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.900382  6668 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:31.874761  9160 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:31.909205  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:31.899744  9078 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:31.963799  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.973176  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:31.978251  7232 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:31.932745  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:31.948444  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:31.981667  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:31.982780  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:31.908965  9205 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:17:31.872074  9128 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:31.891944  9013 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:31.913683  9203 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
W20260709 13:17:31.991267  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58604.6us
I20260709 13:17:31.994447  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:31.905342  9193 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:31.907971  9217 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:31.973780  9123 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.629s	user 0.002s	sys 0.009s
I20260709 13:17:31.999884  6597 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:32.007272  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.936327  8949 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:32.013561  6664 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:31.952507  9004 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.014904  6589 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:31.927263  9178 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.018802  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.019909  7232 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:31.948324  8744 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:31.999835  8913 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:32.002734  9151 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:32.002005  9160 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:32.005256  9232 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:32.033457  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.023216  9077 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:32.006829  9123 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:32.176698  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.978269  9171 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.178354  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.186304  7232 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:32.182566  7235 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.001s
I20260709 13:17:32.186856  7235 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:32.187821  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 792) took 1018 ms. Trace:
I20260709 13:17:32.049211  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.193073  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 779) took 1057 ms. Trace:
I20260709 13:17:32.193743  7232 rpcz_store.cc:276] 0709 13:17:31.135458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.047972 (+912514us) service_pool.cc:224] Handling call
0709 13:17:32.192999 (+145027us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.188230  7235 rpcz_store.cc:276] 0709 13:17:31.168991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.569172 (+400181us) service_pool.cc:224] Handling call
0709 13:17:32.187378 (+618206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.195341  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.199533  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.200574  7235 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:32.056628  9151 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:32.179086  9126 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.208935  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.209825  7235 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:32.206547  9206 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
W20260709 13:17:32.225270  9225 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:17:32.227103  9151 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.196350  8992 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:32.235432  8992 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:32.237478  9199 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.003s
I20260709 13:17:32.237943  9199 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:32.229102  9174 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.727s	user 0.008s	sys 0.000s
I20260709 13:17:32.233858  9097 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.203116  7228 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.233814  7235 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:32.245873  9036 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.766s	user 0.009s	sys 0.002s
W20260709 13:17:32.252741  9121 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.130s	user 0.008s	sys 0.006s
I20260709 13:17:32.122004  9232 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.254745  9128 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.255398  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:31.687798  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:32.257454  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 483) took 1095 ms. Trace:
I20260709 13:17:32.288113  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 868) took 1156 ms. Trace:
W20260709 13:17:32.258876  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.289717  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:32.290654  6516 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:32.290822  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:32.291472  6513 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:32.291990  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:32.292742  6518 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:32.288565  7232 rpcz_store.cc:276] 0709 13:17:31.131873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.214448 (+1082575us) service_pool.cc:224] Handling call
0709 13:17:32.288054 (+ 73606us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.293649  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.294123  7232 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.193416  9069 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:32.284925  6637 connection.cc:256] RPC call timeout handler was delayed by -1.0136857790000704s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 13:17:32.279109  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 786) took 1111 ms. Trace:
I20260709 13:17:32.287961  6663 rpcz_store.cc:276] 0709 13:17:31.162482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.165565 (+  3083us) service_pool.cc:224] Handling call
0709 13:17:31.165832 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.854936 (+689104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.257208 (+402272us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54001cacd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:32.257393 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545139968}
I20260709 13:17:32.295068  7235 rpcz_store.cc:276] 0709 13:17:31.167264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.208453 (+1041189us) service_pool.cc:224] Handling call
0709 13:17:32.279040 (+ 70587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.296173  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.296532  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.296767  6663 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
W20260709 13:17:32.286063  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 295553us
I20260709 13:17:32.286643  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 787) took 1119 ms. Trace:
I20260709 13:17:32.297539  7236 rpcz_store.cc:276] 0709 13:17:31.167564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.387944 (+220380us) service_pool.cc:224] Handling call
0709 13:17:32.286433 (+898489us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54004de254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.286570 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676338560}
I20260709 13:17:32.298936  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:32.292896  7229 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.900s	user 0.012s	sys 0.000s
I20260709 13:17:32.309134  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:32.269395  9168 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:32.308456  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:32.316926  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.317370  6521 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:32.317720  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.311973  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:32.216425  9126 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.315225  6519 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.315676  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.295889  9069 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:32.313753  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:32.318305  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:32.320775  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 791) took 1152 ms. Trace:
I20260709 13:17:32.328693  7236 rpcz_store.cc:276] 0709 13:17:31.168693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.298895 (+1130202us) service_pool.cc:224] Handling call
0709 13:17:32.320719 (+ 21824us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.298713  8903 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:32.328251  6517 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:32.323036  7235 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:32.319190  6520 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:32.200201  9193 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.279100  9174 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.308697  9078 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.279819  9036 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.340278  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:32.341094  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 796) took 1171 ms. Trace:
I20260709 13:17:32.289297  9121 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.358204  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:32.350706  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.359160  7236 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:32.359094  7232 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:32.359702  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 877) took 1225 ms. Trace:
I20260709 13:17:32.351027  6521 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2.
W20260709 13:17:32.352862  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.324558  9013 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.316851  7229 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:32.350378  7235 rpcz_store.cc:276] 0709 13:17:31.170153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.311859 (+1141706us) service_pool.cc:224] Handling call
0709 13:17:32.341009 (+ 29150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.360201  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.360013  7232 rpcz_store.cc:276] 0709 13:17:31.134320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.293412 (+1159092us) service_pool.cc:224] Handling call
0709 13:17:32.359649 (+ 66237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.362383  6520 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.364382  7236 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:32.365235  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.365849  7236 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 3: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:32.362455  6519 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:32.338650  9123 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.369398  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.324394  9168 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.371673  7233 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:32.372112  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.367450  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:32.373220  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.374478  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.371767  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.369102  9218 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.378224  6663 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:32.378229  7236 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:32.378814  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.380007  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.380187  7235 raft_consensus.cc:2376] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 4. Re-sending same reply.
I20260709 13:17:32.381212  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.381597  9121 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:32.382256  9097 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.383841  9071 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.385237  9199 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.385706  9151 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:32.386449  9151 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:32.386754  9176 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:32.387324  9213 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.617s	user 0.002s	sys 0.000s
I20260709 13:17:32.377444  9069 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.388497  9069 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.388967  9176 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.389106  6520 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:32.389853  9151 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:32.390368  9180 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.415s	user 0.004s	sys 0.008s
I20260709 13:17:32.369846  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 888) took 1236 ms. Trace:
I20260709 13:17:32.346819  8903 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.390857  7229 rpcz_store.cc:276] 0709 13:17:31.130096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.130223 (+   127us) service_pool.cc:224] Handling call
0709 13:17:32.366834 (+1236611us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.381384  9232 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.383899  9036 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.387181  9174 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:32.387974  7235 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:32.394090  7236 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:32.391805  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.397958  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 865) took 1267 ms. Trace:
I20260709 13:17:32.398332  7236 rpcz_store.cc:276] 0709 13:17:31.130992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.393685 (+1262693us) service_pool.cc:224] Handling call
0709 13:17:32.393960 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.397690 (+  3730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.397903 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.399394  7232 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:32.400010  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 870) took 1267 ms. Trace:
I20260709 13:17:32.400362  7232 rpcz_store.cc:276] 0709 13:17:31.132408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.398936 (+1266528us) service_pool.cc:224] Handling call
0709 13:17:32.399211 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.399786 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.399950 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.407996  7232 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:32.219386  9217 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.408749  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 873) took 1275 ms. Trace:
I20260709 13:17:32.409297  7232 rpcz_store.cc:276] 0709 13:17:31.133185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.401348 (+1268163us) service_pool.cc:224] Handling call
0709 13:17:32.407801 (+  6453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.408515 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.408697 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.413369  7232 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:32.417337  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.418011  6663 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:32.418805  8903 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.418960  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.419343  8903 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:32.419662  6663 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:32.395233  9218 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.420512  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.421562  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.422076  6663 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
W20260709 13:17:32.422147  6487 connection.cc:256] RPC call timeout handler was delayed by -1.0032786350000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 13:17:32.423204  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.423965  6671 raft_consensus.cc:2368] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7.
I20260709 13:17:32.406028  9168 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.409601  8949 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:17:32.425863  8949 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:32.426172  8949 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:32.427062  8949 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.406275  9174 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:17:32.430336  9174 consensus_queue.cc:1243] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:32.431680  9126 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.401368  7229 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:32.432478  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 798) took 1261 ms. Trace:
W20260709 13:17:32.432894  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.432786  7229 rpcz_store.cc:276] 0709 13:17:31.192038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.391511 (+1199473us) service_pool.cc:224] Handling call
0709 13:17:32.432419 (+ 40908us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.428215  8949 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:32.434059  6487 leader_election.cc:336] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.871s (SENT)
I20260709 13:17:32.442433  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 874) took 1309 ms. Trace:
I20260709 13:17:32.447052  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.447153  7232 rpcz_store.cc:276] 0709 13:17:31.133484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.412972 (+1279488us) service_pool.cc:224] Handling call
0709 13:17:32.413193 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.413678 (+   485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.415401 (+  1723us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.449672  7232 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:32.451156  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.452342  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:32.433161  8912 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.754s	user 0.011s	sys 0.000s
I20260709 13:17:32.466522  6671 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:32.457774  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.419795  8903 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.466356  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 876) took 1332 ms. Trace:
I20260709 13:17:32.451947  9217 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.468248  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.474602  7232 rpcz_store.cc:276] 0709 13:17:31.134046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.449089 (+1315043us) service_pool.cc:224] Handling call
0709 13:17:32.449455 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.465606 (+ 16151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.466288 (+   682us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.476208  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.477546  7232 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:32.479322  6671 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.483762  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.484730  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 780) took 1343 ms. Trace:
I20260709 13:17:32.485061  7232 rpcz_store.cc:276] 0709 13:17:31.135685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.477063 (+1341378us) service_pool.cc:224] Handling call
0709 13:17:32.477357 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.479354 (+  1997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.479548 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.485934  7232 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:32.486369  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 782) took 1347 ms. Trace:
I20260709 13:17:32.486644  7232 rpcz_store.cc:276] 0709 13:17:31.139614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.485599 (+1345985us) service_pool.cc:224] Handling call
0709 13:17:32.485813 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.486216 (+   403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.486333 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.470450  9018 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:32.488577  7232 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:32.489440  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 784) took 1337 ms. Trace:
W20260709 13:17:32.600297  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.487718  9217 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.489813  7232 rpcz_store.cc:276] 0709 13:17:31.152849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.487158 (+1334309us) service_pool.cc:224] Handling call
0709 13:17:32.487342 (+   184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.489203 (+  1861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.489387 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:32.600900  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 117201us
I20260709 13:17:32.602458  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.604514  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.607153  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:32.613325  9218 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.619181  7232 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:32.475692  8912 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.601725  9193 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:32.619983  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 785) took 1453 ms. Trace:
I20260709 13:17:32.620963  7232 rpcz_store.cc:276] 0709 13:17:31.166966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.601485 (+1434519us) service_pool.cc:224] Handling call
0709 13:17:32.602598 (+  1113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.619709 (+ 17111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.619924 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.607089  9204 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:32.609652  9199 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:32.614567  9232 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:32.624038  9232 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:32.636004  9232 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.643219  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.643532  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.643939  6665 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:32.644619  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.644891  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.645268  6663 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:32.646003  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.646092  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.647136  6665 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.638872  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 877) took 1611 ms. Trace:
I20260709 13:17:32.651347  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.652159  6663 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:32.653234  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.653968  6663 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:32.656244  6665 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:32.657480  9162 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.658241  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 875) took 1524 ms. Trace:
I20260709 13:17:32.658790  6665 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:32.661278  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 871) took 1528 ms. Trace:
I20260709 13:17:32.662849  7236 rpcz_store.cc:276] 0709 13:17:31.132675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.399340 (+1266665us) service_pool.cc:224] Handling call
0709 13:17:32.399724 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:32.400085 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.400115 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:32.400687 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:17:32.448970 (+ 48283us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:32.449017 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.449108 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.661092 (+211984us) raft_consensus.cc:1685] finished
0709 13:17:32.661111 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.661225 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2546,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":3058,"thread_start_us":47896,"threads_started":1,"wal-append.queue_time_us":146192}
I20260709 13:17:32.664426  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.665367  6671 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
W20260709 13:17:32.666990  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.667737  7236 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:32.668937  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 794) took 1499 ms. Trace:
I20260709 13:17:32.669358  7236 rpcz_store.cc:276] 0709 13:17:31.169552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.665314 (+1495762us) service_pool.cc:224] Handling call
0709 13:17:32.665575 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.668611 (+  3036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.668872 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.671013  7236 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:32.673065  9011 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:32.673502  9011 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:32.674260  9011 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.675242  9011 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.676911  7236 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:32.677383  7236 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:32.658740  9162 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.658820  8903 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:32.687853  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.689497  7236 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.681797  7228 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:32.699991  7236 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:32.700981  7232 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:32.658653  7229 rpcz_store.cc:276] 0709 13:17:31.133749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.433865 (+1300116us) service_pool.cc:224] Handling call
0709 13:17:32.434240 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:32.434609 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.441427 (+  6818us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:32.477613 (+ 36186us) log.cc:844] Serialized 26 byte log entry
0709 13:17:32.623806 (+146193us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:32.624216 (+   410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.631279 (+  7063us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:32.640976 (+  9697us) raft_consensus.cc:1685] finished
0709 13:17:32.642309 (+  1333us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.658176 (+ 15867us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3989,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":11006,"thread_start_us":142171,"threads_started":1,"wal-append.queue_time_us":158196}
W20260709 13:17:32.709499  9253 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:17:32.721462  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2207) took 1552 ms. Trace:
I20260709 13:17:32.722095  6597 rpcz_store.cc:276] 0709 13:17:31.169441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.382227 (+212786us) service_pool.cc:224] Handling call
0709 13:17:32.721195 (+1338968us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54003bbfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.721379 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223524096}
W20260709 13:17:32.723778  7228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 878) took 1589 ms (client timeout 2103 ms). Trace:
W20260709 13:17:32.724385  7228 rpcz_store.cc:269] 0709 13:17:31.134577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.001412 (+866835us) service_pool.cc:224] Handling call
0709 13:17:32.723689 (+722277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.727188  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2203) took 1634 ms. Trace:
I20260709 13:17:32.727596  6593 rpcz_store.cc:276] 0709 13:17:31.092856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.471682 (+378826us) service_pool.cc:224] Handling call
0709 13:17:31.471975 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.520524 (+ 48549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.726921 (+1206397us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b540034fb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:32.727104 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77619584}
I20260709 13:17:32.729592  9209 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:32.730003  9209 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:32.730862  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 70126.5us
W20260709 13:17:32.733217  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.734964  9209 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.736064  9209 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.743115  7236 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:17:32.732316  8912 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.650810  7231 rpcz_store.cc:276] 0709 13:17:31.027822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.045293 (+ 17471us) service_pool.cc:224] Handling call
0709 13:17:32.416260 (+1370967us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:32.416463 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.416676 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:32.467558 (+ 50882us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.626904 (+159346us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.631340 (+  4436us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:32.632870 (+  1530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.638006 (+  5136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.638809 (+   803us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":190209,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":671,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":4735,"spinlock_wait_cycles":13121408,"thread_start_us":190594,"threads_started":2,"wal-append.queue_time_us":162148}
I20260709 13:17:32.747205  9148 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.749490  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2204) took 1630 ms. Trace:
I20260709 13:17:32.749876  6589 rpcz_store.cc:276] 0709 13:17:31.119594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.508640 (+389046us) service_pool.cc:224] Handling call
0709 13:17:31.509024 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.015393 (+506369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.748931 (+733538us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540019e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:32.749424 (+   493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1162838912}
I20260709 13:17:32.744760  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:32.750901  7236 pending_rounds.cc:85] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 6
I20260709 13:17:32.751219  7236 pending_rounds.cc:107] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:17:32.751523  7236 raft_consensus.cc:2889] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:32.722219  9211 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:32.737454  9232 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.727200  9002 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.784633  9002 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.760128  9211 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:32.789088  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.796900  9183 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.543s	user 0.004s	sys 0.000s
I20260709 13:17:32.765690  8912 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:32.803929  8912 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.804493  8912 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:32.806141  9245 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.381s	user 0.000s	sys 0.005s
I20260709 13:17:32.807742  9211 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.809237  9211 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.810906  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2043) took 1081 ms. Trace:
I20260709 13:17:32.811336  6670 rpcz_store.cc:276] 0709 13:17:31.729539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.763708 (+ 34169us) service_pool.cc:224] Handling call
0709 13:17:31.784929 (+ 21221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.900856 (+115927us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:32.188441 (+287585us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.286635 (+ 98194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.286691 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:32.286740 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.810644 (+523904us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54001900d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:32.810699 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.810847 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6922,"prepare.queue_time_us":413146,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":4082,"raft.queue_time_us":7038,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":12396,"spinlock_wait_cycles":265859456,"thread_start_us":351396,"threads_started":2,"wal-append.queue_time_us":138800}
I20260709 13:17:32.751634  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:32.819213  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:32.819994  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 165376us
W20260709 13:17:32.822161  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 70613.8us
I20260709 13:17:32.821660  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:32.823578  9148 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:32.824003  6637 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.558s (SENT)
I20260709 13:17:32.824152  8949 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:32.824945  6637 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.564s (SENT)
I20260709 13:17:32.823976  9148 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.825358  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:32.825845  9148 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:32.826803  9151 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.824628  8949 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.827979  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.829196  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:32.829496  9151 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:32.829898  9151 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:32.830013  9128 raft_consensus.cc:2764] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:32.830302  9151 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.831296  6669 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:32.832835  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.833412  6669 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:32.834672  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.835359  6669 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:32.836404  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.836894  6669 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:32.846640  6487 leader_election.cc:336] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.033s (SENT)
I20260709 13:17:32.849699  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:32.850852  6487 leader_election.cc:336] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.046s (SENT)
I20260709 13:17:32.847663  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:32.851506  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.853494  6663 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:32.853490  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.852353  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:32.855934  6669 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:32.856597  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:32.857933  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:32.859587  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:32.859848  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.860591  6665 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:32.860651  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:32.847952  8771 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:32.848299  9218 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:17:32.863454  6487 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.120s (SENT)
I20260709 13:17:32.844792  9021 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:32.870216  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.852916  9217 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:32.856074  8903 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:32.862494  9011 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:17:32.853345  8911 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:17:32.860767  9126 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:32.861814  9121 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:32.875337  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.855849  9176 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:32.905601  6665 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:32.884052  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.864297  9220 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:32.908568  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 45140.6us
I20260709 13:17:32.897019  9151 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.885224  9002 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.900197  8912 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.871196  8992 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:32.871153  9162 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:17:32.870528  9218 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:32.875397  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.877223  9229 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:32.894811  8903 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:32.907442  9121 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.923707  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:32.927878  6665 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.862789  9148 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.907210  9176 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:32.912084  9193 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:32.913167  9217 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:32.887004  9168 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:17:32.872262  9021 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:32.935248  7198 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.774s (SENT) [suppressed 44 similar messages]
W20260709 13:17:32.934468  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76322.6us
I20260709 13:17:32.936990  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2032) took 2054 ms. Trace:
I20260709 13:17:32.916483  9220 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:32.937565  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.927810  8903 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.924558  9218 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.940954  9006 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:32.929435  9011 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:32.945214  7230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 884) took 1898 ms (client timeout 1877 ms). Trace:
I20260709 13:17:32.929559  9121 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.926971  8771 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:32.939149  6662 rpcz_store.cc:276] 0709 13:17:30.882710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.886227 (+  3517us) service_pool.cc:224] Handling call
0709 13:17:32.936764 (+2050537us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b540023ffd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.936917 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1084237184}
I20260709 13:17:32.935016  9176 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:32.948664  7230 rpcz_store.cc:269] 0709 13:17:31.047241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.047358 (+   117us) service_pool.cc:224] Handling call
0709 13:17:32.945015 (+1897657us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54005a3554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.945145 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108044032}
I20260709 13:17:32.948876  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:32.933537  9229 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:32.959205  6665 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:32.958657  9263 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.962394  9263 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.964726  9263 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.965057  9021 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:32.966856  6667 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 489) took 1802 ms (client timeout 1774 ms). Trace:
I20260709 13:17:32.966966  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 490) took 1160 ms. Trace:
I20260709 13:17:32.968248  6668 rpcz_store.cc:276] 0709 13:17:31.806979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.807146 (+   167us) service_pool.cc:224] Handling call
0709 13:17:32.966809 (+1159663us) spinlock_profiling.cc:243] Waited 8.68 ms on lock 0x7b54000e60d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.966911 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19960064}
W20260709 13:17:32.972939  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.966423  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 105822us
W20260709 13:17:32.967338  6667 rpcz_store.cc:269] 0709 13:17:31.164173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.274265 (+110092us) service_pool.cc:224] Handling call
0709 13:17:32.966662 (+1692397us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54000e6ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.966795 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291363072}
I20260709 13:17:32.942337  9217 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.975188  8771 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.977053  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.977490  9217 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:32.978000  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 788) took 1810 ms. Trace:
I20260709 13:17:32.978052  9217 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.978587  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:32.978483  7232 rpcz_store.cc:276] 0709 13:17:31.167863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.623857 (+1455994us) service_pool.cc:224] Handling call
0709 13:17:32.700793 (+ 76936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.701386 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.977812 (+276426us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b5400684cd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.977936 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174996224}
I20260709 13:17:32.979194  9220 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.981297  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:32.982317  9193 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:32.982847  9193 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:32.982918  9229 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.983717  9193 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:32.985667  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:32.986017  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.986141  8912 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:32.986510  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.986864  8912 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:32.950359  9011 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:32.987108  8949 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:32.986929  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.988286  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 615) took 1982 ms. Trace:
I20260709 13:17:32.988003  9011 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.988685  6588 rpcz_store.cc:276] 0709 13:17:31.005982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.391400 (+385418us) service_pool.cc:224] Handling call
0709 13:17:31.391737 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:31.895736 (+503999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.988059 (+1092323us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54003a1954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:32.988234 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131267968}
I20260709 13:17:32.989300  9011 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.987471  8912 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:32.954784  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.991099  7228 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:17:32.991693  9260 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:17:32.994015  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 866) took 1862 ms. Trace:
W20260709 13:17:32.988283  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.986302  6519 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.995327  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:32.999043  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:32.994403  7235 rpcz_store.cc:276] 0709 13:17:31.131283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.380879 (+1249596us) service_pool.cc:224] Handling call
0709 13:17:32.993734 (+612855us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b54006cc354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:32.993959 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13474048}
I20260709 13:17:33.002707  8979 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.003247  8979 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.003921  6519 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:33.005295  8979 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.008308  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.008945  6515 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 2.
I20260709 13:17:33.009289  6517 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:33.013478  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:33.014357  6517 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:33.016149  6521 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:33.018405  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:33.020707  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:33.021423  6517 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:32.991525  8912 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.023931  9263 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:33.006979  8992 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:33.027760  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.027627  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.030501  6521 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:17:33.037736  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.038424  6514 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:33.038497  9263 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:33.039501  9263 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.042433  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.048089  6517 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:33.048677  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.023804  6519 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:33.058135  6522 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:17:33.058720  6522 pending_rounds.cc:85] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 4
I20260709 13:17:33.058218  6515 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:33.059667  6522 pending_rounds.cc:107] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:17:33.060030  6522 raft_consensus.cc:2889] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:33.061363  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.062937  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.069993  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.070907  6521 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:33.074311  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.076704  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:33.073092  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.064270  9168 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.084169  6519 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 2: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:33.058910  9071 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:33.091544  6563 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.090268  6522 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:33.092175  6521 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:33.094308  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:33.066759  9078 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:33.064975  9013 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:33.115206  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:33.074359  9151 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.117967  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.137310  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:33.138211  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:33.139767  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:33.118993  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.119503  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.121259  8979 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:33.123828  9259 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:17:33.116262  9002 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:17:33.127157  8911 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:33.127563  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.154834  7237 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:33.128962  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.155807  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.156486  7237 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:33.156937  7230 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:33.129235  9148 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.159966  6662 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:33.097384  9264 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.154304  7235 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:33.164721  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:33.157933  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.102327  9266 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.158890  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:33.103058  9162 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.165555  9266 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.165817  9162 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.168030  9162 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:33.160179  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.148187  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.168327  9266 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.160948  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:33.168874  7230 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:33.169580  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:33.169687  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.170123  6662 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:33.170501  7230 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:33.172086  6663 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:33.172348  6662 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:17:33.173822  6662 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:33.117347  9209 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:33.111063  9078 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.175885  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:33.176573  6668 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:33.176780  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.177086  6561 leader_election.cc:400] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7.
I20260709 13:17:33.177870  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:33.178503  7236 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:33.174706  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:33.180038  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:33.178532  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.132303  8771 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:33.179229  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.180859  6667 raft_consensus.cc:2410] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:33.180899  7235 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:33.152184  9217 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:33.189117  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.157207  9204 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:33.157377  9018 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:33.198150  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 83051.4us
I20260709 13:17:33.192998  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.182000  6662 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:33.154758  8912 raft_consensus.cc:2764] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:17:33.105875  9071 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.187094  9172 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:33.116813  9211 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:17:33.131362  9220 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:17:33.199362  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:33.103438  9168 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.195416  9204 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.177664  8911 consensus_queue.cc:1243] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:33.201916  6668 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:33.131589  9013 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.200066  6662 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:33.182368  9209 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:33.200785  9018 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.182194  9078 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:33.205015  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.132360  9176 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.188589  8771 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.194342  9217 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:33.195575  8979 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:33.288050  9128 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:33.288879  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 99114.5us
I20260709 13:17:33.311101  9204 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:33.289228  9071 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.289678  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:33.345052  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55466.5us
I20260709 13:17:33.290660  7235 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:33.291358  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 494) took 1448 ms. Trace:
I20260709 13:17:33.288056  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 876) took 2264 ms. Trace:
I20260709 13:17:33.163411  9011 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:17:33.235487  9193 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:33.292559  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.174734  7231 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
W20260709 13:17:33.293184  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 1.62137e+06us
I20260709 13:17:33.304219  6663 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:33.309533  6668 pending_rounds.cc:85] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
I20260709 13:17:33.313470  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.314560  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 894) took 1364 ms. Trace:
I20260709 13:17:33.306351  9272 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:33.322917  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.295591  8912 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.187129  9167 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:33.334803  9263 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:33.335728  9128 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.334504  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.340082  9176 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:33.341713  9168 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.350497  7233 rpcz_store.cc:276] 0709 13:17:31.023645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.027399 (+  3754us) service_pool.cc:224] Handling call
0709 13:17:31.051505 (+ 24106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.375189 (+1323684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.218057 (+842868us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54006cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:33.287965 (+ 69908us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3035515136}
W20260709 13:17:33.392177  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57775.6us
I20260709 13:17:33.392624  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:33.394945  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.396198  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:17:33.398984  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 286565us
I20260709 13:17:33.351432  6664 rpcz_store.cc:276] 0709 13:17:31.842869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.982552 (+139683us) service_pool.cc:224] Handling call
0709 13:17:33.217792 (+1235240us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b54001cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:33.291302 (+ 73510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69172096}
I20260709 13:17:33.328795  9018 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:33.352773  9232 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.330439  8771 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.353330  9002 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:33.352733  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 209965us
I20260709 13:17:33.389209  6668 pending_rounds.cc:107] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:33.420127  6668 raft_consensus.cc:2889] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:33.419965  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.372692  7237 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:33.326869  9209 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.397097  9148 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:33.427717  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.397854  9069 raft_consensus.cc:2764] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:33.428035  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:33.390667  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77283.5us
I20260709 13:17:33.429152  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.429847  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:33.427819  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.430954  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.431176  6588 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:33.432274  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:33.432991  6589 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:33.433360  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.434099  6594 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 2.
I20260709 13:17:33.434430  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.401109  9168 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:33.326399  9121 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.330588  9217 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:33.405980  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.407573  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.445428  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.408967  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.446372  6588 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:17:33.389030  7234 rpcz_store.cc:276] 0709 13:17:31.949873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.670923 (+721050us) service_pool.cc:224] Handling call
0709 13:17:32.671213 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:32.671574 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.671600 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:32.672066 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:17:32.847922 (+175856us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:32.847962 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.848023 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:33.123188 (+275165us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:33.310191 (+187003us) raft_consensus.cc:1685] finished
0709 13:17:33.310220 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:33.314487 (+  4267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3223,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":84,"spinlock_wait_cycles":8766848,"thread_start_us":175518,"threads_started":1,"wal-append.queue_time_us":242011}
I20260709 13:17:33.432583  9266 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:33.398495  9272 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:33.449007  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.449714  6597 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:33.417034  9071 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:33.411966  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.445339  9232 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:17:33.451026  6596 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:33.450935  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.451782  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.452035  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.452755  6588 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:33.452842  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.453271  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.453512  6596 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:33.453855  6593 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:33.454308  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.455312  7236 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:33.449577  9272 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.455350  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.457697  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.458535  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.460264  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.420198  9002 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.437189  9176 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:33.448575  9266 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:33.465553  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.465655  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.462947  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.468822  6595 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:33.464699  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.469383  6594 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:33.450488  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.396778  8912 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.453482  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.479231  6597 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:33.479771  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:33.473286  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.487334  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.488343  6589 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:33.467260  9002 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:33.487047  9071 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:33.499868  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.500326  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.468080  9266 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:33.501847  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.507051  6592 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:33.508672  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.491204  9193 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.482255  9176 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:33.537308  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.542320  6563 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.544672  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:33.547420  6594 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:33.502027  9121 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.476665  9265 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 13:17:33.581836  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 163815us
W20260709 13:17:33.565104  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 190514us
W20260709 13:17:33.626377  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 81821us
I20260709 13:17:33.590341  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.626792  6486 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:33.606879  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 142661us
I20260709 13:17:33.631737  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:33.633760  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.588308  6597 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:33.636723  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:33.550611  9218 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.011s
W20260709 13:17:33.548707  8903 consensus_meta.cc:287] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.000s
I20260709 13:17:33.642339  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.655468  6594 raft_consensus.cc:2376] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4. Re-sending same reply.
I20260709 13:17:33.546512  9172 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:33.548156  7228 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.008s
I20260709 13:17:33.632970  9167 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:33.606259  9011 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:33.587059  9171 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:33.779410  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.780166  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.638048  9288 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:17:33.795840  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 120288us
I20260709 13:17:33.802474  6594 raft_consensus.cc:2376] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 2. Re-sending same reply.
I20260709 13:17:33.803182  6593 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
W20260709 13:17:33.781016  9278 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
I20260709 13:17:33.545833  9193 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:33.808635  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 165301us
I20260709 13:17:33.811883  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.636897  9263 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.166s
I20260709 13:17:33.812801  6593 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:33.818017  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.819705  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:33.820595  9282 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:17:33.820907  9283 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.273s	user 0.000s	sys 0.000s
I20260709 13:17:33.633160  9071 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:33.823786  9281 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:17:33.825206  9002 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.678229  9218 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.827605  9002 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.828554  9218 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.829515  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:33.829972  9002 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:33.829890  9209 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.830456  9002 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:33.832540  6594 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:33.774700  8903 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.831350  9299 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:33.819373  9167 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.842846  9299 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:33.801182  9069 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:17:33.851483  8912 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.851531  9299 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:33.853302  9193 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.809836  9128 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:33.854761  6595 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:33.856153  9209 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.807962  9078 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.858659  9078 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:33.858923  6589 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:33.806938  9288 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:33.836321  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 172004us
I20260709 13:17:33.839268  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.805365  9295 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.644111  9280 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.178s
I20260709 13:17:33.646962  9176 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.804574  7228 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:33.794797  9011 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:33.863721  6593 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:33.843809  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.840021  9002 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:33.840744  9171 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:33.865523  9288 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:33.876128  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 244468us
I20260709 13:17:33.875278  9303 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:17:33.880178  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:33.883950  8903 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:33.886981  9264 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
I20260709 13:17:33.887416  9264 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:33.888098  6590 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
W20260709 13:17:33.888360  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.891410  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.892026  7232 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:33.893493  7228 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:33.876791  9296 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:33.905603  7200 leader_election.cc:400] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:33.905896  7200 leader_election.cc:403] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:33.916462  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.916917  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:33.917744  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.918326  6594 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:33.919673  9128 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:33.920164  9097 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:33.920580  9097 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:33.920418  9217 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.920157  9128 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.921315  9128 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.921414  9097 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.922413  9300 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:33.867723  9292 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:17:33.926117  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:33.926491  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:33.926687  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:33.930200  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:33.935530  6515 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:33.935882  6518 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:33.935658  6521 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:33.924407  9177 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4.
I20260709 13:17:33.924173  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:33.938203  6519 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:33.903558  9229 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:33.876595  9171 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.922606  9097 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:33.927889  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.941207  6665 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:33.940337  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:33.926745  6514 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:33.944651  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.880223  9018 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.926127  8992 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.934545  9217 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.870355  9295 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.948704  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.942785  9300 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:17:33.952682  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.940699  9229 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:33.954872  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:33.953933  6518 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
W20260709 13:17:33.959504  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.927487  9002 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.961107  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:33.971763  9155 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:33.972998  9288 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.976042  9295 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:33.976477  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:33.937098  9011 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.977946  9011 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:33.979671  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.979518  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:33.980674  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:33.981230  9071 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:33.981745  8771 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.000s
I20260709 13:17:33.981714  9071 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:33.982797  9298 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:33.983378  9298 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:33.984190  9298 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.984663  9071 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.986923  9218 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:33.987462  9218 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:33.988376  9218 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.988850  9018 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.990703  9122 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:33.991724  9126 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:17:33.947014  9296 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:33.996973  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:33.994272  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:33.963178  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:34.004094  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:34.008180  9286 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:17:34.008677  7234 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:34.009084  9264 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:34.009893  9264 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:34.010236  9264 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:33.945835  9299 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.016032  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:33.956928  9300 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.013365  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:34.021567  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1730) took 1177 ms. Trace:
I20260709 13:17:34.022205  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:34.022491  9171 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:33.962741  9229 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:34.023248  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:34.010550  9284 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.463s	user 0.001s	sys 0.000s
I20260709 13:17:33.948454  8992 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.963063  9289 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:34.081233  6516 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:34.019340  8949 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:34.003945  9291 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:34.027436  9300 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:34.029760  6517 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:17:34.044473  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.058657  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:34.059854  6519 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:34.084919  9171 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:17:34.089447  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.090824  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.091881  6519 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:34.060019  6520 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:34.092463  6521 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:34.057626  7236 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
W20260709 13:17:33.993680  9204 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.640s	user 0.006s	sys 0.002s
I20260709 13:17:33.982333  8771 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:33.963936  9265 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:34.108806  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:33.955235  9167 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:34.016258  9177 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:34.008679  9217 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:34.019245  9293 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:34.083994  6518 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:34.024694  6666 rpcz_store.cc:276] 0709 13:17:32.844395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.844506 (+   111us) service_pool.cc:224] Handling call
0709 13:17:32.966941 (+122435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:32.967325 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.967355 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:32.997114 (+ 29759us) log.cc:844] Serialized 26 byte log entry
0709 13:17:33.348070 (+350956us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:33.370095 (+ 22025us) log.cc:844] Serialized 14 byte log entry
0709 13:17:33.604762 (+234667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.604836 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:33.825168 (+220332us) raft_consensus.cc:1685] finished
0709 13:17:33.825193 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.021400 (+196207us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000e60d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:34.021509 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5342,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":245148,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":22295,"spinlock_wait_cycles":280775552,"thread_start_us":572536,"threads_started":2,"wal-append.queue_time_us":459658}
I20260709 13:17:34.058212  9304 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:17:34.078706  9289 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:17:33.964143  9273 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:17:34.078969  9218 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:17:34.060398  6513 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:17:34.094871  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:34.060873  9309 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:34.124081  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 108140us
I20260709 13:17:34.125705  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.125968  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.096624  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.070365  9168 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:34.070403  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:34.082782  9296 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:34.138043  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.138607  6670 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:17:34.117090  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 91339us
I20260709 13:17:34.119721  6588 raft_consensus.cc:2368] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3.
I20260709 13:17:34.093454  6520 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:34.122231  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.142001  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.083494  9172 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:17:34.096794  9288 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:34.111073  9011 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:34.110065  8949 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.126514  6663 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:34.127213  6522 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:34.127641  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.180429  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.180687  6668 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:34.180858  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2049) took 1025 ms. Trace:
I20260709 13:17:34.181012  6670 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:34.181454  6665 rpcz_store.cc:276] 0709 13:17:33.155636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.155750 (+   114us) service_pool.cc:224] Handling call
0709 13:17:33.179694 (+ 23944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:33.943665 (+763971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.180598 (+236933us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b540017d4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:34.180769 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1750856064}
I20260709 13:17:34.129426  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.187220  6671 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:34.130139  6669 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:34.189422  6522 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:34.130832  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.190274  6521 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:34.190862  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.132339  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.132946  6662 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:34.192626  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.133874  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:34.135725  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.118223  9291 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.117913  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:34.115208  9293 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:34.121845  9217 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:34.146857  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.173683  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:34.144315  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.106819  9204 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.167253  6520 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:34.213583  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:34.155066  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.214367  7234 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.128022  9121 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:17:34.231287  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.232358  6594 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:34.131649  9168 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.178287  6518 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:34.178439  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.128564  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.241818  6516 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:34.130928  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.190438  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.194896  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:34.196045  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.199112  6590 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:34.107270  9264 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.139016  9296 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.202775  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.209721  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.210803  6513 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:34.266002  6668 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.176203  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:34.212865  9293 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.174912  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 903) took 1227 ms. Trace:
I20260709 13:17:34.225819  9204 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.232998  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.244666  9168 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:34.239894  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 66293.6us
I20260709 13:17:34.280323  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.136368  9263 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:34.271996  9299 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:34.268150  9296 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:34.248951  9316 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:17:34.249480  9167 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:34.210325  9291 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.244782  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.249981  8992 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.250020  9303 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.133345  9309 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.143476  9128 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.307598  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:34.253247  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.259270  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.292621  6515 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:34.310060  6594 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.198922  9211 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:34.317772  9211 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:34.318660  9211 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.319644  9211 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:34.320291  7230 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
W20260709 13:17:34.267570  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 72777.7us
I20260709 13:17:34.322824  9261 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.323134  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:34.323338  9261 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.323989  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:34.324179  9316 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.324653  9316 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:34.132972  9289 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:17:34.215719  6514 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.270125  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.337208  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:34.337787  6592 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:34.271852  9210 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:34.251667  9002 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.229832  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.341243  6520 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:34.342322  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.274231  6597 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:34.343986  6597 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:34.284274  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:34.245705  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.349845  6595 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:34.208369  8949 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.354372  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:34.355252  6520 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:34.283435  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:34.216099  9011 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:34.287521  6522 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:34.291240  9320 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:34.295874  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:34.365759  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.365648  6519 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:34.366012  6595 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:17:34.366422  6595 pending_rounds.cc:85] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 3
I20260709 13:17:34.366652  6595 pending_rounds.cc:107] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:17:34.269966  7229 rpcz_store.cc:276] 0709 13:17:32.945248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.948013 (+  2765us) service_pool.cc:224] Handling call
0709 13:17:32.973907 (+ 25894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:33.096573 (+122666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:33.096636 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:33.173606 (+ 76970us) log.cc:844] Serialized 26 byte log entry
0709 13:17:33.388037 (+214431us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:33.390997 (+  2960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.401941 (+ 10944us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:34.010021 (+608080us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:34.120244 (+110223us) raft_consensus.cc:1685] finished
0709 13:17:34.121535 (+  1291us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.172410 (+ 50875us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21803,"raft.queue_time_us":100327,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":13718,"spinlock_wait_cycles":19400960,"thread_start_us":231098,"threads_started":2,"wal-append.queue_time_us":296343}
I20260709 13:17:34.293696  9167 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.340128  9280 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.294934  9291 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.304662  9128 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.315670  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.300917  9309 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.317363  6594 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:34.282262  9263 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:34.268451  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.323920  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.329515  7234 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:34.336856  6668 raft_consensus.cc:2410] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:34.338436  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.389693  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.285374  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.383373  6594 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:34.391986  6591 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:17:34.358711  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 75340.6us
I20260709 13:17:34.392976  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:34.388689  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 64828.9us
I20260709 13:17:34.360812  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.294791  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.361495  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.362068  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:34.366840  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.394460  6592 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:34.366815  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.336483  9264 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.295358  9303 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.340304  9295 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:34.340560  9210 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.380239  6520 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:34.339969  9177 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:17:34.322387  9300 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.363763  8912 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:34.366974  6595 raft_consensus.cc:2889] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:34.395253  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:34.396703  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:34.392990  9291 raft_consensus.cc:2764] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:34.346740  9299 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:17:34.379299  9002 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.359632  9310 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:17:34.272194  9217 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:34.412844  9002 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.403872  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.408599  7234 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.397622  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:34.391552  9263 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.419018  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:34.438000  6518 raft_consensus.cc:2376] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5. Re-sending same reply.
I20260709 13:17:34.421674  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:34.339977  9296 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.380227  9167 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:34.406458  9210 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:34.400974  9320 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:34.439502  7234 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
W20260709 13:17:34.460219  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 62363.6us
I20260709 13:17:34.461372  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:34.462821  7234 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:17:34.462690  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.404831  9155 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:17:34.464970  7234 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:17:34.475277  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 65399.2us
I20260709 13:17:34.415591  9176 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:34.415079  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:34.439867  8979 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.469846  9069 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:34.383358  9280 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.476579  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:34.460526  9320 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.460127  9326 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:34.484225  9266 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.942s	user 0.007s	sys 0.004s
I20260709 13:17:34.487018  9069 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:34.506304  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 90694.5us
I20260709 13:17:34.510987  6597 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:34.465937  9291 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:17:34.486526  9128 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.476706  8912 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:17:34.481112  9318 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:17:34.506969  7228 raft_consensus.cc:2368] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:34.508667  9319 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:34.496577  8979 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:34.528231  9322 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:17:34.528908  8979 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:34.531075  8949 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.531538  9071 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:34.531720  8949 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:34.533071  7236 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:34.519846  9167 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:34.533883  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2227) took 1120 ms. Trace:
W20260709 13:17:34.534804  9317 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:17:34.455657  9011 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:34.535490  9270 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.131s	user 0.001s	sys 0.002s
I20260709 13:17:34.534191  9265 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:34.536576  9265 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:34.537444  9265 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.538815  9265 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.540457  9168 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.485988  9176 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:34.534206  7236 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:34.543727  9296 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:34.543439  9176 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.544965  9303 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.524591  7232 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:34.546249  8771 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.510394  9266 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.549978  9266 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:34.497997  9316 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:34.550197  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 638) took 1071 ms. Trace:
I20260709 13:17:34.513921  9274 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:17:34.526660  9002 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.550664  6588 rpcz_store.cc:276] 0709 13:17:33.478761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.479444 (+   683us) service_pool.cc:224] Handling call
0709 13:17:34.550006 (+1070562us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b54002fe9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:34.550140 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1421859584}
I20260709 13:17:34.526715  9313 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:34.552717  9333 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:34.553766  9300 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.556293  9280 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.556563  9297 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:34.557103  9297 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:34.558755  8912 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:34.559129  8912 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:34.559912  8912 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.560963  8912 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.561781  9229 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.517840  9293 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.502368  9320 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:34.566174  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 92519.4us
I20260709 13:17:34.521638  9332 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:34.511169  9069 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.531991  9071 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:34.534265  6596 rpcz_store.cc:276] 0709 13:17:33.413686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.458323 (+ 44637us) service_pool.cc:224] Handling call
0709 13:17:34.533693 (+1075370us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54002fbcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:34.533823 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147713792}
I20260709 13:17:34.568544  7237 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:34.569015  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.570724  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.572434  9291 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:34.565095  9319 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.573967  9319 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:34.575127  6593 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:34.574366  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.575975  9291 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:34.577108  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.577800  9291 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:34.558187  9297 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.579375  9297 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.581344  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.582700  6415 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 330) took 6724 ms. Trace:
I20260709 13:17:34.582847  9300 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:34.583767  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.584041  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.584595  6517 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:34.585306  9263 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.586308  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.587718  9263 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.587922  9293 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.589272  6517 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:34.583186  6415 rpcz_store.cc:276] 0709 13:17:27.843400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.843729 (+   329us) service_pool.cc:224] Handling call
0709 13:17:34.567438 (+6723709us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:29.115879 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:29.116602 (+   723us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:29.116655 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:17:29.116691 (+    36us) tablet.cc:662] Decoding operations
0709 13:17:29.117744 (+  1053us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:29.117832 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:29.117870 (+    38us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:17:29.146887 (+ 29017us) tablet.cc:701] Row locks acquired
0709 13:17:29.163189 (+ 16302us) write_op.cc:260] PREPARE: finished
0709 13:17:29.172551 (+  9362us) write_op.cc:270] Start()
0709 13:17:29.172710 (+   159us) write_op.cc:276] Timestamp: P: 1783603049172474 usec, L: 0
0709 13:17:29.172736 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:17:29.173361 (+   625us) log.cc:844] Serialized 3646 byte log entry
0709 13:17:30.188947 (+1015586us) op_driver.cc:464] REPLICATION: finished
0709 13:17:30.903207 (+714260us) write_op.cc:301] APPLY: starting
0709 13:17:30.903301 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:30.908408 (+  5107us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:30.910091 (+  1683us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:30.915817 (+  5726us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:30.916176 (+   359us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:17:30.916224 (+    48us) write_op.cc:312] APPLY: finished
0709 13:17:30.916581 (+   357us) log.cc:844] Serialized 71 byte log entry
0709 13:17:30.916780 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:30.916903 (+   123us) write_op.cc:454] Released schema lock
0709 13:17:30.916970 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:30.917397 (+   427us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:30.917430 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9532,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":672458,"apply.run_cpu_time_us":4016,"apply.run_wall_time_us":22181,"num_ops":9,"prepare.queue_time_us":135659,"prepare.run_cpu_time_us":142003,"prepare.run_wall_time_us":543587,"raft.queue_time_us":314569,"raft.run_cpu_time_us":25607,"raft.run_wall_time_us":613188,"replication_time_us":962326,"thread_start_us":1112602,"threads_started":4,"wal-append.queue_time_us":568551,"wal-append.run_cpu_time_us":30646,"wal-append.run_wall_time_us":2183107}]]}
I20260709 13:17:34.589859  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:34.590698  9293 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:34.591063  9293 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.591441  9293 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:34.592670  9300 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:34.525512  9128 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.595407  9333 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:34.596390  9333 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.542423  9168 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.601089  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:34.604009  6563 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:34.688956  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 48575.7us
I20260709 13:17:34.731813  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:34.727648  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 443361us
I20260709 13:17:34.733844  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.642202  9021 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:34.745731  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 257925us
I20260709 13:17:34.640852  9211 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:34.723755  9264 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:34.755434  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.747288  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:34.725698  9168 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:34.752650  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.754542  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.755206  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:34.756820  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.747112  9167 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.758148  9313 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:17:34.761857  9167 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:34.763833  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.764253  6521 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:34.766252  6590 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:34.766546  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.767287  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.638571  8979 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:17:34.769493  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:34.770218  6669 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:34.776551  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.771001  6488 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:34.777535  6591 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:34.776640  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 379068us
I20260709 13:17:34.781214  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.781354  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.783789  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:34.784653  6663 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:34.747697  9002 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:34.751825  9316 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.789017  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:34.789366  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.759970  9264 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.735373  9293 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.768555  9297 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:34.796058  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:34.801920  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.797423  6590 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:17:34.802486  6663 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.802655  6590 pending_rounds.cc:85] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 1
I20260709 13:17:34.802940  6590 pending_rounds.cc:107] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:34.803162  6590 raft_consensus.cc:2889] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:34.800194  6669 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:34.771838  9232 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:34.780175  9204 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:34.779850  9309 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:34.807461  7230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 887) took 1630 ms (client timeout 1574 ms). Trace:
I20260709 13:17:34.804567  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.810521  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:34.815694  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:34.823195  7230 rpcz_store.cc:269] 0709 13:17:33.177154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.177656 (+   502us) service_pool.cc:224] Handling call
0709 13:17:34.807258 (+1629602us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400571554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:34.807389 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576418688}
I20260709 13:17:34.765319  9211 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.799006  9264 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.825462  7233 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:34.768126  9303 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:17:34.828075  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:34.892012  6513 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:34.831211  6666 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:34.793388  9011 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:34.764976  9021 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.834820  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.833369  6591 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:34.799189  9168 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:34.788777  9217 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:17:34.804421  9128 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.834405  6665 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:34.833434  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.888557  6520 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:17:34.970165  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 136803us
I20260709 13:17:34.888015  9229 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:34.894889  6663 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:34.895187  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:34.824967  9126 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:34.974056  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 78946.2us
I20260709 13:17:34.974118  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:34.974821  6637 leader_election.cc:336] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.107s (SENT)
I20260709 13:17:34.975236  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:34.779978  9018 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:17:34.786458  9285 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:17:34.824927  9167 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:34.771374  9071 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:34.945113  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:34.834820  9013 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:34.831890  9297 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:34.991312  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:34.973739  9299 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:34.879204  9263 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:34.899989  8992 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.981585  9128 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:34.982056  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:35.023067  6665 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:34.987680  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:34.831393  9342 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.991878  6663 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:34.991089  9341 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:17:34.907213  9011 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:34.943574  9309 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:17:35.014238  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.981331  9291 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:34.887375  9265 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:35.031237  9346 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:17:35.031680  9261 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.040613  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:35.018110  9168 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.036811  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:35.037065  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:35.014559  9071 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:35.048923  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.059878  6663 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:35.067478  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:35.094961  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:35.016481  9263 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:35.036132  9342 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.070384  9151 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.154814  8911 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:35.155783  9332 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.556s	user 0.013s	sys 0.013s
I20260709 13:17:35.155442  8911 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.155973  7228 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:35.156203  9332 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:35.157188  6636 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.574s (SENT)
I20260709 13:17:35.158072  8911 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.013031  9126 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:17:35.143656  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:34.998572  9340 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 13:17:35.148105  9263 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:35.179733  9126 consensus_queue.cc:1243] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:35.152631  9297 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:35.186856  9349 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:17:35.186949  9293 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.188421  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:35.154454  9151 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.167614  6513 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:35.033761  9155 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:35.166713  9329 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.200469  9329 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:35.201145  9151 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.174549  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:35.202481  9021 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.150406  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:35.203193  6663 raft_consensus.cc:2410] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:35.145572  9320 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.205384  6663 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:17:35.183669  9069 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.605s	user 0.007s	sys 0.002s
I20260709 13:17:35.176397  6665 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:17:35.172716  9319 consensus_meta.cc:287] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.598s	user 0.000s	sys 0.013s
I20260709 13:17:35.033501  8992 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.219625  9069 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:35.220541  9342 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:35.220800  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.221294  8992 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.222054  9069 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:35.172571  9210 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.692s	user 0.009s	sys 0.003s
I20260709 13:17:35.055824  9291 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:35.206348  8912 consensus_meta.cc:287] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.641s	user 0.008s	sys 0.001s
I20260709 13:17:35.215389  9211 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.215543  9011 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:35.222163  9326 consensus_meta.cc:287] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.004s
W20260709 13:17:35.172268  6514 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.807s	user 0.010s	sys 0.002s
I20260709 13:17:35.205547  9263 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:35.229714  6663 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:17:35.239857  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:35.206070  9320 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.262385  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:35.280746  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:35.285166  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51134.3us
I20260709 13:17:35.225073  9291 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:35.381548  6665 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:35.422905  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:35.423833  6665 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:35.224895  9319 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:35.225778  9210 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:35.562992  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.281880  9155 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:35.251456  9326 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:35.433672  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 191758us
I20260709 13:17:35.649652  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:35.596180  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:35.652575  6663 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:35.653909  6663 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:35.632622  6515 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.018s	sys 0.031s
I20260709 13:17:35.251680  6514 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:35.624457  8912 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.668915  6518 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:17:35.663268  6515 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.019s	sys 0.032s
I20260709 13:17:35.309201  9329 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.667331  9265 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:35.673622  6668 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:17:35.639269  9006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.021s	sys 0.025s
I20260709 13:17:35.607890  9126 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.589519  9011 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:35.676301  6521 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.008s	sys 0.048s
I20260709 13:17:35.679710  6663 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:35.707973  6521 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.008s	sys 0.048s
I20260709 13:17:35.708184  6665 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:17:35.607717  9220 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:35.685381  6671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.012s	sys 0.032s
W20260709 13:17:35.710191  6671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.012s	sys 0.033s
I20260709 13:17:35.286599  9069 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.363s
I20260709 13:17:35.550981  8911 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:35.711462  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.000s	sys 0.027s
I20260709 13:17:35.667351  9261 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:17:35.714092  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.000s	sys 0.027s
W20260709 13:17:35.716208  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 936354us
W20260709 13:17:35.682235  9006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.021s	sys 0.025s
I20260709 13:17:35.734242  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:35.761937  6669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.052s	sys 0.090s
W20260709 13:17:35.763283  6669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.052s	sys 0.090s
I20260709 13:17:35.701413  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2234) took 1655 ms. Trace:
I20260709 13:17:35.718420  9263 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.680078  8912 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:35.726028  9339 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.765290  9078 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:35.796200  7200 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.034s
I20260709 13:17:35.716549  9304 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
W20260709 13:17:35.798132  7200 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.000s	sys 0.035s
W20260709 13:17:35.799450  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 656021us
I20260709 13:17:35.711447  9126 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.787286  9262 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:35.797981  9078 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:35.811916  9267 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.324s
W20260709 13:17:35.812546  9348 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.403s	user 0.001s	sys 0.001s
I20260709 13:17:35.787905  9261 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:35.790359  9069 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:35.720125  9220 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.791214  9319 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.816689  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:35.759754  9329 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:17:35.771185  8771 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:17:35.714354  9011 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.734920  8911 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:35.821800  6670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.034s	sys 0.055s
W20260709 13:17:35.713030  9162 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.011s	sys 0.040s
W20260709 13:17:35.814430  9297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.001s	sys 0.029s
I20260709 13:17:35.812633  9262 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:35.774493  6514 rpcz_store.cc:276] 0709 13:17:34.026373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.071488 (+ 45115us) service_pool.cc:224] Handling call
0709 13:17:35.681283 (+1609795us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:35.840771  6516 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:35.830295  6670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.034s	sys 0.055s
I20260709 13:17:35.817145  9353 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:35.816856  9261 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:35.847344  6662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.013s	sys 0.011s
I20260709 13:17:35.836560  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:35.871202  6662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.014s	sys 0.011s
I20260709 13:17:35.871452  6517 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:35.805215  9339 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:35.836421  9162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.011s	sys 0.040s
I20260709 13:17:35.824635  9326 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:35.842892  9297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.001s	sys 0.029s
I20260709 13:17:35.829027  9267 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:17:35.875192  9316 consensus_meta.cc:287] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.078s	user 0.010s	sys 0.000s
I20260709 13:17:35.876919  9121 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:35.885908  6663 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:35.888099  6663 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:35.816684  9078 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.889204  6663 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.886066  9210 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.887995  9121 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:35.897897  9316 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:35.919468  6486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.031s	sys 0.073s
W20260709 13:17:35.926469  6486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.031s	sys 0.073s
I20260709 13:17:35.918313  9297 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:17:35.911362  9078 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:35.922919  9280 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.011s	sys 0.028s
I20260709 13:17:35.885818  9326 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:35.933679  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.915215  9267 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:35.884719  9353 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:35.938417  6520 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.000s	sys 0.024s
W20260709 13:17:35.942739  6520 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.000s	sys 0.025s
I20260709 13:17:35.952487  9355 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.953248  9326 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:35.954284  9360 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:17:35.955076  9210 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:35.955473  9210 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:35.956297  9210 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.957463  9210 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:35.953022  9355 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.896880  9262 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.962949  6664 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
W20260709 13:17:35.933657  9280 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.011s	sys 0.028s
W20260709 13:17:35.963974  6667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.017s	sys 0.053s
W20260709 13:17:35.958437  7228 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.802s	user 0.007s	sys 0.003s
W20260709 13:17:35.966385  6667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.018s	sys 0.053s
I20260709 13:17:35.937561  9297 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.965752  7228 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
W20260709 13:17:35.971226  9356 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:17:35.976694  9297 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:35.977393  9358 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.096s	user 0.000s	sys 0.002s
I20260709 13:17:35.978073  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 528) took 1191 ms. Trace:
I20260709 13:17:35.913666  9261 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.979066  8911 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.979803  9261 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:35.980391  9347 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.953s	user 0.021s	sys 0.000s
I20260709 13:17:35.972014  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2065) took 1064 ms. Trace:
W20260709 13:17:35.965107  9264 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.036s	user 0.000s	sys 0.012s
I20260709 13:17:35.978506  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2063) took 1098 ms. Trace:
I20260709 13:17:35.983539  9264 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:35.983831  6671 rpcz_store.cc:276] 0709 13:17:34.880031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.880416 (+   385us) service_pool.cc:224] Handling call
0709 13:17:34.880767 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.890890 (+ 10123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.890928 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.901282 (+ 10354us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.711164 (+809882us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:35.711204 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.711270 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:35.976847 (+265577us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:35.978232 (+  1385us) raft_consensus.cc:1685] finished
0709 13:17:35.978252 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.978451 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9697,"raft.queue_time_us":24272,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":54165,"spinlock_wait_cycles":2514688,"thread_start_us":734001,"threads_started":1,"wal-append.queue_time_us":866289}
I20260709 13:17:35.985957  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:35.986315  6671 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:35.972281  9126 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.988133  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:35.988296  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:35.988843  6514 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:35.978488  6515 rpcz_store.cc:276] 0709 13:17:34.787099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.787272 (+   173us) service_pool.cc:224] Handling call
0709 13:17:34.787639 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.945605 (+157966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.945651 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.946263 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.670283 (+724020us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:35.670394 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.670578 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:35.977880 (+307302us) raft_consensus.cc:1685] finished
0709 13:17:35.977898 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.978018 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3141,"raft.queue_time_us":37515,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":49453,"thread_start_us":813312,"threads_started":2,"wal-append.queue_time_us":853624}
I20260709 13:17:35.982788  6669 rpcz_store.cc:276] 0709 13:17:34.907647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.952015 (+ 44368us) service_pool.cc:224] Handling call
0709 13:17:34.952432 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.952821 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.952856 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.968643 (+ 15787us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.765016 (+796373us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:35.766321 (+  1305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.766397 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:35.971786 (+205389us) raft_consensus.cc:1685] finished
0709 13:17:35.971808 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.971953 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":487,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10380,"thread_start_us":779839,"threads_started":1,"wal-append.queue_time_us":954651}
I20260709 13:17:35.966087  8912 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:35.990999  9264 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:35.951277  9018 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:35.976099  9071 consensus_meta.cc:287] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.000s
I20260709 13:17:35.998633  9220 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:35.999114  7234 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:35.999559  9011 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:36.001782  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 510) took 1233 ms. Trace:
I20260709 13:17:36.107555  6666 rpcz_store.cc:276] 0709 13:17:34.768588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.768691 (+   103us) service_pool.cc:224] Handling call
0709 13:17:34.768933 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:34.892806 (+123873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.001561 (+1108755us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b54000b3454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:36.001729 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142660992}
I20260709 13:17:36.003286  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 918) took 1913 ms. Trace:
I20260709 13:17:36.007107  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 892) took 2700 ms. Trace:
W20260709 13:17:36.048286  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.109668  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.064850  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.101794  7234 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:36.109354  7233 rpcz_store.cc:276] 0709 13:17:33.307095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.392254 (+ 85159us) service_pool.cc:224] Handling call
0709 13:17:34.807391 (+1415137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:34.891710 (+ 84319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.006873 (+1115163us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400571550. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:36.007041 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3253580928}
I20260709 13:17:36.110695  7234 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.109215  7237 rpcz_store.cc:276] 0709 13:17:34.090258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.090382 (+   124us) service_pool.cc:224] Handling call
0709 13:17:34.113955 (+ 23573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:34.568943 (+454988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.002391 (+1433448us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b5400576a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:36.002535 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1044939136}
W20260709 13:17:36.111992  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.967505  9326 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.098992  9172 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:36.161728  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.158391  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 507001us
I20260709 13:17:36.220939  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:36.222160  7232 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:17:36.228986  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 79339.5us
I20260709 13:17:36.218855  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:35.955113  9121 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.107149  9262 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:36.238067  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:36.239372  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:36.241219  6634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.020s	sys 0.030s
I20260709 13:17:36.246953  6665 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:36.248683  6664 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 505) took 1494 ms (client timeout 1572 ms). Trace:
W20260709 13:17:36.249310  6664 rpcz_store.cc:269] 0709 13:17:34.755048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.755171 (+   123us) service_pool.cc:224] Handling call
0709 13:17:36.246425 (+1491254us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:36.248603 (+  2178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2700200832}
I20260709 13:17:36.251307  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.252488  7237 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:36.106701  9018 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:36.237011  9168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.000s	sys 0.021s
I20260709 13:17:36.151189  9297 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:36.205422  9264 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.333015  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:36.338183  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.347512  7233 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:36.348658  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:36.349365  7233 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:36.295856  9168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.001s	sys 0.022s
I20260709 13:17:36.258291  9289 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.298797  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.244393  8949 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.128088  9326 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.239482  9303 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:36.188380  9071 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:17:36.333918  7237 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:17:36.298197  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.280252  9339 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:36.342494  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.025s	sys 0.042s
W20260709 13:17:36.427356  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.025s	sys 0.042s
W20260709 13:17:36.427682  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 666471us
I20260709 13:17:36.222986  9172 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:36.359608  6521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.011s	sys 0.053s
W20260709 13:17:36.429538  6521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.012s	sys 0.053s
W20260709 13:17:36.339634  9265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.018s	sys 0.043s
I20260709 13:17:36.354588  7236 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:36.442317  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.365715  9303 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:36.490048  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 523) took 1256 ms. Trace:
I20260709 13:17:36.490450  6665 rpcz_store.cc:276] 0709 13:17:35.233458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.708837 (+475379us) service_pool.cc:224] Handling call
0709 13:17:36.246543 (+537706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:36.247451 (+   908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.489835 (+242384us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54001cb6d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:36.489978 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235854720}
W20260709 13:17:36.497522  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.367655  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:36.415908  6517 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.000s
I20260709 13:17:36.504498  6517 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:17:36.353065  9006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.011s	sys 0.051s
I20260709 13:17:36.351572  9204 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:36.352610  9011 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:36.366799  9155 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
I20260709 13:17:36.428376  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:36.431806  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2256) took 2133 ms. Trace:
I20260709 13:17:36.503921  7236 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:36.358142  9264 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.517066  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:36.517880  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.519398  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 91113us
W20260709 13:17:36.520231  7200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.027s	sys 0.032s
W20260709 13:17:36.575778  7200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.027s	sys 0.033s
W20260709 13:17:36.576333  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.576422  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:36.522384  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 317450us
I20260709 13:17:36.519572  6518 rpcz_store.cc:276] 0709 13:17:34.298418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.438674 (+140256us) service_pool.cc:224] Handling call
0709 13:17:35.668597 (+1229923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:35.669268 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.431557 (+762289us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003f75d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:36.431732 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2827825408}
W20260709 13:17:36.514273  9006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.011s	sys 0.051s
I20260709 13:17:36.430214  9002 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:36.423892  9353 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:36.397501  8949 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:36.520793  7197 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.020s	sys 0.039s
W20260709 13:17:36.587126  7197 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.020s	sys 0.040s
W20260709 13:17:36.513432  9265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.018s	sys 0.043s
I20260709 13:17:36.563630  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:36.524142  6486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.002s	sys 0.029s
W20260709 13:17:36.588419  6522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.017s	sys 0.023s
I20260709 13:17:36.487677  7231 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:36.515532  9362 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:36.592329  6486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.006s	sys 0.037s
W20260709 13:17:36.592468  6522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.018s	sys 0.023s
I20260709 13:17:36.366142  9289 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.592758  7236 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:17:36.593350  6485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.023s	sys 0.033s
W20260709 13:17:36.594269  6488 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.006s	sys 0.031s
I20260709 13:17:36.579385  9355 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:36.601785  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 642005us
I20260709 13:17:36.580672  9297 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.518653  9155 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:36.604563  6485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.023s	sys 0.033s
I20260709 13:17:36.586229  9274 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:36.518242  9011 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:36.607955  9372 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.583036  9204 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:36.609514  9280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.013s	sys 0.024s
I20260709 13:17:36.623644  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:36.631781  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:36.632129  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.632452  6515 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:17:36.616590  9361 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 13:17:36.625599  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:36.605144  6488 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.007s	sys 0.031s
I20260709 13:17:36.622848  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:36.627709  6517 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:36.635640  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 647771us
W20260709 13:17:36.625941  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.632714  7237 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:36.591480  9265 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:36.638702  7231 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:36.639330  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2064) took 1741 ms. Trace:
I20260709 13:17:36.639484  9162 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:36.639249  9265 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.635016  6515 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:36.602905  9374 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.640084  9162 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:36.640553  9374 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:36.633968  9372 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.639837  6670 rpcz_store.cc:276] 0709 13:17:34.897932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.898099 (+   167us) service_pool.cc:224] Handling call
0709 13:17:34.960946 (+ 62847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.961342 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.961381 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.969441 (+  8060us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.847106 (+877665us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:35.847159 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.847244 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:36.639132 (+791888us) raft_consensus.cc:1685] finished
0709 13:17:36.639156 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:36.639274 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6238,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":11286,"thread_start_us":830995,"threads_started":1,"wal-append.queue_time_us":1129138}
I20260709 13:17:36.594051  9002 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.642410  7234 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:36.630434  7236 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
W20260709 13:17:36.632066  9280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.013s	sys 0.025s
I20260709 13:17:36.643978  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:36.627348  9171 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:36.648351  9374 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:36.648907  9171 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.649241  9372 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:36.649559  9171 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:36.650921  9280 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:36.651721  9171 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:36.648036  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.653864  7236 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:36.626569  9220 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:36.654754  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:36.637665  9365 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.451s	user 0.000s	sys 0.001s
I20260709 13:17:36.624006  9297 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.658470  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.642331  9162 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:36.636771  9261 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:36.647357  9368 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 13:17:36.664844  7236 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:36.667107  7234 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
W20260709 13:17:36.676885  9291 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.157s	user 0.007s	sys 0.002s
I20260709 13:17:36.677594  7233 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
W20260709 13:17:36.678232  9367 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:17:36.678754  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 527) took 1895 ms. Trace:
I20260709 13:17:36.604487  9303 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.686717  6521 rpcz_store.cc:276] 0709 13:17:34.782914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.783056 (+   142us) service_pool.cc:224] Handling call
0709 13:17:34.783369 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:35.708381 (+925012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:35.708411 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:35.713640 (+  5229us) log.cc:844] Serialized 26 byte log entry
0709 13:17:36.429904 (+716264us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:36.429951 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.430027 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:36.678530 (+248503us) raft_consensus.cc:1685] finished
0709 13:17:36.678551 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:36.678692 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1082,"raft.queue_time_us":23551,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":7844,"thread_start_us":1567477,"threads_started":2,"wal-append.queue_time_us":870811}
I20260709 13:17:36.605068  9362 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.630141  9274 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.658862  9375 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.685099  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:36.685186  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.695648  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:36.696460  7236 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:36.690200  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.015s	sys 0.012s
I20260709 13:17:36.678325  7231 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:36.657306  9353 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 13:17:36.698989  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.016s	sys 0.012s
I20260709 13:17:36.656196  9379 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:17:36.682803  7234 pending_rounds.cc:85] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 6
I20260709 13:17:36.679633  9374 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:36.679411  9371 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:36.679767  9376 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:36.706382  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:36.704612  9303 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:36.709324  7234 pending_rounds.cc:107] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:36.743912  7234 raft_consensus.cc:2889] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:36.744580  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 919) took 2516 ms. Trace:
I20260709 13:17:36.704892  9372 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:36.744947  7234 rpcz_store.cc:276] 0709 13:17:34.228569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.483505 (+254936us) service_pool.cc:224] Handling call
0709 13:17:34.483855 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:35.999022 (+1515167us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b54001705a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:17:36.642246 (+643224us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b580005b7d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fdd0aa 00007f04b905f447 00007f04b9077fe7 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4
0709 13:17:36.744357 (+102111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.744528 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225542272}
W20260709 13:17:36.702381  7198 connection.cc:256] RPC call timeout handler was delayed by -1.4007414060001793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 13:17:36.667579  9220 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.692427  9382 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:36.720485  7236 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:36.740942  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 529) took 1576 ms. Trace:
I20260709 13:17:36.706809  9326 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:36.750883  7236 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:36.750214  6520 rpcz_store.cc:276] 0709 13:17:35.164245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.165130 (+   885us) service_pool.cc:224] Handling call
0709 13:17:35.165521 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:35.165910 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:35.165951 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:35.166512 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.943202 (+776690us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:35.943257 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.943353 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:36.740611 (+797258us) raft_consensus.cc:1685] finished
0709 13:17:36.740640 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:36.740844 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33526,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":10943,"thread_start_us":776306,"threads_started":1,"wal-append.queue_time_us":1172469}
I20260709 13:17:36.746538  7234 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:36.742117  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 509) took 1974 ms. Trace:
I20260709 13:17:36.708573  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:36.683895  9261 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.703208  9384 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:36.748180  6668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 503) took 1996 ms (client timeout 2496 ms). Trace:
I20260709 13:17:36.686417  9291 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:36.749596  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.749483  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:36.751959  6662 rpcz_store.cc:276] 0709 13:17:34.767293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.769635 (+  2342us) service_pool.cc:224] Handling call
0709 13:17:34.770004 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.821034 (+ 51030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.821208 (+   174us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.821938 (+   730us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.871631 (+1049693us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:35.871675 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.871746 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:36.738288 (+866542us) raft_consensus.cc:1685] finished
0709 13:17:36.738310 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:36.742038 (+  3728us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3000,"raft.queue_time_us":51610,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":61492,"thread_start_us":1020372,"threads_started":2,"wal-append.queue_time_us":1259081}
I20260709 13:17:36.751959  7236 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:36.757527  7234 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:36.757748  6668 rpcz_store.cc:269] 0709 13:17:34.751807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.752394 (+   587us) service_pool.cc:224] Handling call
0709 13:17:36.743630 (+1991236us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54000e6ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:36.748103 (+  4473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2102613888}
W20260709 13:17:36.758344  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 134750us
W20260709 13:17:36.758776  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.703130  9375 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:36.755507  9318 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:17:36.764880  7236 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:36.783061  7233 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:36.789562  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 213194us
I20260709 13:17:36.790287  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:36.742854  9374 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:36.744203  9371 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:36.779348  9363 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:17:36.759325  9382 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.745731  9376 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.780838  9172 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:36.972824  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.813735  9383 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:17:36.819149  9297 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:36.794435  9388 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.819223  9204 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:37.010918  9388 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.012723  9374 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:36.764320  7234 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:37.013392  9388 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:37.013733  9262 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.000s
W20260709 13:17:36.844264  9380 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:17:36.791501  9381 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.799839  9353 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:37.023435  7231 raft_consensus.cc:2410] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:37.023756  9353 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:37.024516  9376 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:37.016040  9289 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:37.025158  9078 consensus_meta.cc:287] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.091s	user 0.008s	sys 0.003s
I20260709 13:17:36.904534  9168 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:37.025714  9168 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:37.026674  9382 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:37.026561  9168 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.027731  9168 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:37.028554  9210 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.071s	user 0.011s	sys 0.001s
I20260709 13:17:36.835278  9011 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:36.791960  9006 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:36.835315  9326 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:36.795986  9385 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.223s
I20260709 13:17:37.029337  9220 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:37.029511  9396 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:17:37.033578  9151 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:37.040500  9151 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:37.041023  9389 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
W20260709 13:17:37.041889  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.042791  9121 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:36.758546  8949 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:36.779976  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 142428us
W20260709 13:17:36.813118  6514 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.000s
W20260709 13:17:36.868165  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 148998us
W20260709 13:17:37.045936  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 255705us
I20260709 13:17:37.047616  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:36.985284  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 235863us
I20260709 13:17:36.986116  9392 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.023013  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 533) took 1187 ms. Trace:
I20260709 13:17:37.024704  9353 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:36.851588  7233 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:37.048779  9392 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:37.019642  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:37.037566  7235 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:17:37.049364  9333 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:37.045375  6663 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:37.048728  6522 rpcz_store.cc:276] 0709 13:17:35.835306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.835459 (+   153us) service_pool.cc:224] Handling call
0709 13:17:35.835810 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:35.846474 (+ 10664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:35.847405 (+   931us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:35.847950 (+   545us) log.cc:844] Serialized 52 byte log entry
0709 13:17:36.601979 (+754029us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:36.602018 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.602085 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:37.022812 (+420727us) raft_consensus.cc:1685] finished
0709 13:17:37.022832 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:37.022957 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42077,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":161,"thread_start_us":717733,"threads_started":1,"wal-append.queue_time_us":750847}
I20260709 13:17:37.051101  9392 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:36.793675  9264 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:37.046326  9151 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:37.030807  9291 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:37.052812  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 511) took 2275 ms. Trace:
I20260709 13:17:37.053462  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:36.813704  9375 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:36.985220  9018 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:36.901628  9304 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.954257  9379 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.078024  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.079304  6522 raft_consensus.cc:2410] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:37.053277  6667 rpcz_store.cc:276] 0709 13:17:34.769345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.769438 (+    93us) service_pool.cc:224] Handling call
0709 13:17:34.790860 (+ 21422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.793817 (+  2957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.793848 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.794303 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.969153 (+1174850us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:35.969206 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.969293 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:37.027792 (+1058499us) raft_consensus.cc:1685] finished
0709 13:17:37.027814 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:37.044974 (+ 17160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2553,"raft.queue_time_us":91719,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":54182,"spinlock_wait_cycles":3968,"thread_start_us":1173716,"threads_started":1,"wal-append.queue_time_us":1631806}
I20260709 13:17:37.056387  6487 leader_election.cc:400] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:37.083714  6487 leader_election.cc:403] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:17:37.085191  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:37.087431  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:37.089829  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:36.994354  9398 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
W20260709 13:17:37.018669  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:37.096055  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.058089  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2267) took 2208 ms. Trace:
I20260709 13:17:37.058215  6663 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:37.097021  6519 rpcz_store.cc:276] 0709 13:17:34.845713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.881163 (+ 35450us) service_pool.cc:224] Handling call
0709 13:17:34.881478 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:34.891873 (+ 10395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.891915 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:36.332740 (+1440825us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b2c000014c8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:17:36.333277 (+   537us) log.cc:844] Serialized 52 byte log entry
0709 13:17:36.684279 (+351002us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:37.045480 (+361201us) log.cc:844] Serialized 14 byte log entry
0709 13:17:37.045808 (+   328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:37.045908 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:37.045962 (+    54us) raft_consensus.cc:1685] finished
0709 13:17:37.054317 (+  8355us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:37.054449 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9966,"raft.queue_time_us":116983,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":31015,"spinlock_wait_cycles":8746880,"thread_start_us":617879,"threads_started":2,"wal-append.queue_time_us":406931}
I20260709 13:17:36.902596  9387 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.052834  6514 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:37.058291  9385 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:37.063189  9384 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:37.063597  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44027.7us
I20260709 13:17:37.150116  7230 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:37.023484  9381 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:37.022478  9204 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:17:37.158022  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:36.954005  9362 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:37.045251  9372 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:37.223728  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.030813  9002 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:37.046954  9172 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:37.224735  6592 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
W20260709 13:17:37.069497  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 171260us
I20260709 13:17:37.072078  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:36.812511  9397 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:37.226754  6591 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:37.071681  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.093871  6590 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:37.228435  6597 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:37.094817  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:37.056452  9382 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:36.923890  9378 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.109107  6593 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:37.134506  6596 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:17:37.134843  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 87293.5us
W20260709 13:17:37.135349  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:37.151770  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 112416us
I20260709 13:17:37.152813  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.240775  6521 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:37.155164  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.241935  6588 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:37.242987  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.243711  6588 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:37.244645  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.155737  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:37.164472  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.245944  6589 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:37.162223  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 524) took 1928 ms. Trace:
I20260709 13:17:37.246649  6663 rpcz_store.cc:276] 0709 13:17:35.233769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.709606 (+475837us) service_pool.cc:224] Handling call
0709 13:17:35.709871 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:37.045046 (+1335175us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b58008d0bd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fdd0aa 00007f04b905f447 00007f04b9077fe7 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4
0709 13:17:37.097162 (+ 52116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:37.161914 (+ 64752us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54001e0ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:37.162146 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092898304}
I20260709 13:17:37.185917  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.045817  9006 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:37.229060  6594 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:37.133356  9018 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:37.108570  9396 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4.
I20260709 13:17:37.237061  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.245280  6588 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:37.053225  9326 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:37.285856  6590 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:37.286266  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.287916  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.289198  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.289885  6588 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:37.151175  8949 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.068977  9304 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.266530  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:37.270112  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.275969  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.277245  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.279268  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.283507  6594 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
W20260709 13:17:37.284535  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 58171.2us
I20260709 13:17:37.292068  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.292276  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.301405  6592 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:37.301692  6597 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:37.302639  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.303361  6597 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:37.304328  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:37.304945  6597 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:37.293128  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.307230  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.307968  6590 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:37.309087  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.293324  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:37.293349  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 203606us
I20260709 13:17:37.312830  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:37.334758  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.322415  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.336452  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:37.338040  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:37.338692  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.264463  9385 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:37.313109  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.166112  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2270) took 1169 ms. Trace:
I20260709 13:17:37.310328  6589 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:37.311868  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:37.341432  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 190792us
I20260709 13:17:37.342053  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:37.343822  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:37.345492  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:37.284284  9397 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:37.260185  9362 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:37.260186  9384 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:37.349275  6594 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:37.356786  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:37.150921  9387 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:37.123714  9379 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.164541  8911 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.156s
I20260709 13:17:37.360342  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.357820  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:37.319430  6591 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:37.372884  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.360939  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:37.320498  9288 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.363265  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:37.296644  9320 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:17:37.370064  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:37.382575  6663 raft_consensus.cc:2410] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:37.352883  9011 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:37.373894  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.320382  9151 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:17:37.374970  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.392838  6596 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:37.331187  9265 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.376478  7230 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:37.320210  9343 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.331586  9264 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.379768  7234 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:37.316859  9291 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:37.352576  9289 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:37.320214  9078 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.384353  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:37.385346  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:37.381388  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:37.405418  7237 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:37.389134  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.314527  8949 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.374173  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:37.372323  9398 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:37.369853  9262 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.284673  9378 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.349656  6514 rpcz_store.cc:276] 0709 13:17:35.987235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.988071 (+   836us) service_pool.cc:224] Handling call
0709 13:17:37.156452 (+1168381us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:37.404526  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:37.408174  6667 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:37.377048  9318 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:17:37.410081  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68083.7us
W20260709 13:17:37.410516  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 97825.1us
I20260709 13:17:37.412307  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.622334  6591 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:37.362178  9297 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 13:17:37.413419  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.386999  7229 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:37.407256  6516 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:37.413075  9011 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:37.411408  9289 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.376940  9155 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:17:37.347990  9402 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:17:37.379258  9329 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:17:37.403324  9291 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.616025  6597 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:37.669957  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:37.649111  9384 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:37.379149  9305 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:17:37.382992  9171 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 13:17:37.672325  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.383373  9288 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:37.700567  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 78898.6us
I20260709 13:17:37.671066  9316 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:37.650032  9318 consensus_queue.cc:1243] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:37.391435  9406 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.298s
I20260709 13:17:37.383345  9006 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.409372  9265 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:37.701642  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79853.3us
I20260709 13:17:37.754596  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:37.648486  9408 net_util.cc:396] Time spent resolve address for 127.5.72.132: real 0.225s	user 0.000s	sys 0.000s
I20260709 13:17:37.687095  9289 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:37.702804  9411 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:17:37.703709  9303 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.954s	user 0.010s	sys 0.000s
I20260709 13:17:37.423969  9078 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.619297  9018 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:17:37.418148  9264 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:37.712250  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.618006  9262 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.749605  9379 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:37.752774  9378 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:37.695426  9291 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:37.791685  9265 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:37.696658  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 898) took 1475 ms. Trace:
I20260709 13:17:37.409744  9372 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.815184  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.816471  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.413949  9343 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.763792  6591 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:37.757061  6596 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:37.702191  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:37.768360  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.768275  9415 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:37.763420  9407 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.344s	user 0.001s	sys 0.000s
I20260709 13:17:37.757658  9376 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:17:37.776834  9398 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:37.777551  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 395533us
I20260709 13:17:37.778057  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:37.779181  7236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.005s	sys 0.001s
W20260709 13:17:37.868772  7236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.008s	sys 0.008s
I20260709 13:17:37.782318  6515 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:37.705787  9316 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:37.788558  6594 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:37.808527  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:37.817965  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:37.870110  6595 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:37.821039  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:37.828125  6588 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 4: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:37.831319  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.871559  6593 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:37.831789  6591 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:37.831893  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.807906  7229 rpcz_store.cc:276] 0709 13:17:36.219134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.236523 (+ 17389us) service_pool.cc:224] Handling call
0709 13:17:37.266093 (+1029570us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:37.324023 (+ 57930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:37.648523 (+324500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:37.694189 (+ 45666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4971392}
I20260709 13:17:37.828576  9387 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:37.833150  9326 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:37.764645  9371 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:37.793092  9210 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:37.835120  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:37.781327  9362 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:37.866248  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.968984  6592 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:37.866056  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:37.869586  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 936) took 1186 ms. Trace:
W20260709 13:17:37.845501  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.817415  9320 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.829397  9402 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.836892  9304 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:37.888652  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 81212.5us
I20260709 13:17:37.899664  7235 raft_consensus.cc:2376] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5. Re-sending same reply.
I20260709 13:17:37.845158  9379 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.695923  9397 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.970098  7236 rpcz_store.cc:276] 0709 13:17:36.683676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.789854 (+106178us) service_pool.cc:224] Handling call
0709 13:17:36.790118 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:36.790308 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:36.956928 (+166620us) log.cc:844] Serialized 14 byte log entry
0709 13:17:37.869206 (+912278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:37.869261 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:37.869311 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:37.869387 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:37.869524 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":205404,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":437,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":12212,"thread_start_us":1073532,"threads_started":2,"wal-append.queue_time_us":856572}
I20260709 13:17:37.865868  9381 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:37.873682  9155 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.873993  9220 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:37.985697  7200 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.899s (SENT) [suppressed 33 similar messages]
I20260709 13:17:37.991669  9379 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.819144  9011 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:38.061574  9410 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:17:38.062285  9379 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:38.062880  9121 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.020s	user 0.002s	sys 0.015s
I20260709 13:17:38.063007  7234 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
W20260709 13:17:38.063196  9425 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 13:17:37.990919  9320 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.072467  7232 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:37.891167  9210 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:37.992349  9078 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:37.968544  9318 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.993201  9288 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:38.066546  9402 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:38.136921  9418 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:17:38.104210  9320 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:38.141357  9168 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.113s	user 0.010s	sys 0.000s
W20260709 13:17:38.067551  9380 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:17:38.211378  8949 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:38.104897  9373 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.143235  9343 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:38.104677  9155 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.982821  9316 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.137297  9398 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.282632  9402 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:38.266464  9373 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:37.868259  6516 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:38.319864  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.319795  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.321707  6519 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:37.767630  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:17:38.323979  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.325564  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 557773us
I20260709 13:17:38.327216  6514 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:38.328389  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2280) took 1047 ms. Trace:
I20260709 13:17:38.329038  6514 rpcz_store.cc:276] 0709 13:17:37.281225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.686902 (+405677us) service_pool.cc:224] Handling call
0709 13:17:38.328230 (+641328us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:37.930578  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.885006  6594 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:37.884889  6588 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:38.335121  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:37.876647  6591 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:38.337288  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:38.338152  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2277) took 1057 ms. Trace:
I20260709 13:17:38.338663  6519 rpcz_store.cc:276] 0709 13:17:37.280366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.849462 (+569096us) service_pool.cc:224] Handling call
0709 13:17:38.338003 (+488541us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:38.327477  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 439843us
W20260709 13:17:38.344285  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.336149  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1967) took 1064 ms. Trace:
I20260709 13:17:38.349682  6588 rpcz_store.cc:276] 0709 13:17:37.271878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.876112 (+604234us) service_pool.cc:224] Handling call
0709 13:17:37.884667 (+  8555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.335244 (+450577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.335881 (+   637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.352449  6588 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:17:38.353452  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 660) took 1036 ms. Trace:
I20260709 13:17:38.353889  6588 rpcz_store.cc:276] 0709 13:17:37.317193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.351272 (+1034079us) service_pool.cc:224] Handling call
0709 13:17:38.351998 (+   726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.352946 (+   948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.353318 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.338569  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1972) took 1065 ms. Trace:
I20260709 13:17:38.356002  6591 rpcz_store.cc:276] 0709 13:17:37.273228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.876178 (+602950us) service_pool.cc:224] Handling call
0709 13:17:37.876471 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.337860 (+461389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.338335 (+   475us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:38.001222  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.372539  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.373396  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.406873  6516 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:38.412288  6591 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:38.419404  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 661) took 1099 ms. Trace:
I20260709 13:17:38.419996  6591 rpcz_store.cc:276] 0709 13:17:37.317497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.357339 (+1039842us) service_pool.cc:224] Handling call
0709 13:17:38.358129 (+   790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.412877 (+ 54748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.416380 (+  3503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.417930  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2281) took 1136 ms. Trace:
W20260709 13:17:38.418591  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72027.9us
I20260709 13:17:38.421579  6516 rpcz_store.cc:276] 0709 13:17:37.281497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.777724 (+496227us) service_pool.cc:224] Handling call
0709 13:17:38.414213 (+636489us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.378482  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 928) took 1773 ms. Trace:
I20260709 13:17:38.428306  6592 raft_consensus.cc:1217] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Deduplicated request from leader. Original: 4.6->[5.7-5.7]   Dedup: 5.7->[]
W20260709 13:17:38.429374  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 95294.9us
I20260709 13:17:38.430226  9280 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:38.246027  9318 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:38.432789  9370 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.841s	user 0.021s	sys 0.024s
I20260709 13:17:38.438858  6592 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:17:38.443747  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.445322  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1977) took 1095 ms. Trace:
I20260709 13:17:38.447986  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 667) took 1125 ms. Trace:
W20260709 13:17:38.451826  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.454209  9121 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.455081  9121 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.455607  9316 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:38.455801  9381 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.456775  9381 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:38.455637  9414 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.458001  9121 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.457876  9414 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:38.445886  6590 rpcz_store.cc:276] 0709 13:17:37.317210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.873160 (+555950us) service_pool.cc:224] Handling call
0709 13:17:37.895738 (+ 22578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.339998 (+444260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:38.340041 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:38.340767 (+   726us) log.cc:844] Serialized 26 byte log entry
0709 13:17:38.408332 (+ 67565us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:38.408389 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.408525 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:38.411327 (+  2802us) raft_consensus.cc:1685] finished
0709 13:17:38.411350 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.412273 (+   923us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32623,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":195,"thread_start_us":66999,"threads_started":1,"wal-append.queue_time_us":67556}
I20260709 13:17:38.450045  6592 rpcz_store.cc:276] 0709 13:17:37.319038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.425738 (+1106700us) service_pool.cc:224] Handling call
0709 13:17:38.427997 (+  2259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.444623 (+ 16626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.444785 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.461259  9280 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:38.461506  9297 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.432804  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:38.464320  6588 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:17:38.464576  6595 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.594s	user 0.007s	sys 0.010s
I20260709 13:17:38.465301  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 663) took 1147 ms. Trace:
I20260709 13:17:38.466995  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:38.467667  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.469198  6588 rpcz_store.cc:276] 0709 13:17:37.318023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.415018 (+1096995us) service_pool.cc:224] Handling call
0709 13:17:38.449501 (+ 34483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.464985 (+ 15484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.465234 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.462203  8949 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.464258  7229 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:38.473501  9397 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:38.474352  9397 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:38.267010  9210 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.476130  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 543) took 1338 ms. Trace:
I20260709 13:17:38.476943  9210 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:38.476761  6522 rpcz_store.cc:276] 0709 13:17:37.137488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.141642 (+  4154us) service_pool.cc:224] Handling call
0709 13:17:37.831579 (+689937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:37.844476 (+ 12897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:37.844520 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:37.845067 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:17:38.471243 (+626176us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:38.475460 (+  4217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.475575 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:38.475652 (+    77us) raft_consensus.cc:1685] finished
0709 13:17:38.475667 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.475896 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12478,"raft.queue_time_us":13353,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":23518,"thread_start_us":92673,"threads_started":1,"wal-append.queue_time_us":590460}
I20260709 13:17:38.451210  9288 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:17:38.436107  7231 rpcz_store.cc:276] 0709 13:17:36.604531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.708305 (+103774us) service_pool.cc:224] Handling call
0709 13:17:38.377847 (+1669542us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540051a254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:38.378207 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611703680}
I20260709 13:17:38.481487  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:38.482662  7231 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:38.483644  6595 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3.
I20260709 13:17:38.484241  6591 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:38.483935  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:38.484417  6595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2240) took 1390 ms (client timeout 1807 ms). Trace:
I20260709 13:17:38.484747  9414 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.484901  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 666) took 1166 ms. Trace:
I20260709 13:17:38.485430  6591 rpcz_store.cc:276] 0709 13:17:37.318788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.436414 (+1117626us) service_pool.cc:224] Handling call
0709 13:17:38.483910 (+ 47496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.484655 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.484849 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:38.485128  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.484886  6595 rpcz_store.cc:269] 0709 13:17:37.094084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.094344 (+   260us) service_pool.cc:224] Handling call
0709 13:17:38.484325 (+1389981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.485455  6522 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:38.497296  6592 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:38.497728  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.498090  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 669) took 1178 ms. Trace:
I20260709 13:17:38.497370  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:38.498479  6592 rpcz_store.cc:276] 0709 13:17:37.319494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.460964 (+1141470us) service_pool.cc:224] Handling call
0709 13:17:38.497068 (+ 36104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.497856 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.498035 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:38.500559  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 74377.3us
I20260709 13:17:38.504161  7237 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:38.476038  9397 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.508500  6588 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:17:38.513142  6522 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:38.516191  9155 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.546958  9318 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.517066  9422 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:38.547817  9422 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.548110  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1968) took 1275 ms. Trace:
I20260709 13:17:38.548391  9422 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.548616  6593 rpcz_store.cc:276] 0709 13:17:37.272135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.876764 (+604629us) service_pool.cc:224] Handling call
0709 13:17:37.967207 (+ 90443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.328450 (+361243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:38.328534 (+    84us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:38.329471 (+   937us) log.cc:844] Serialized 26 byte log entry
0709 13:17:38.442540 (+113069us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:38.442635 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.448732 (+  6097us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:38.547717 (+ 98985us) raft_consensus.cc:1685] finished
0709 13:17:38.547743 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.547925 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5871,"thread_start_us":112564,"threads_started":1,"wal-append.queue_time_us":162022}
W20260709 13:17:38.518181  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46326us
I20260709 13:17:38.524711  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:38.551221  7231 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:38.551357  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:38.552404  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:38.552953  9288 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:38.553686  9288 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:38.530956  6514 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:38.531934  8949 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:38.554539  9291 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.764s	user 0.000s	sys 0.015s
I20260709 13:17:38.555130  9291 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.555258  9288 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.556802  6668 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:38.462325  9297 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.554574  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2276) took 1274 ms. Trace:
I20260709 13:17:38.556707  8949 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:38.543322  6563 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 1.625s (SENT)
I20260709 13:17:38.563709  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.567257  9155 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.567142  6514 rpcz_store.cc:276] 0709 13:17:37.280059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.330558 (+1050499us) service_pool.cc:224] Handling call
0709 13:17:38.554482 (+223924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:38.570684  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.571790  9381 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.559221  9297 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.581398  6517 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:38.581658  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 662) took 1263 ms. Trace:
I20260709 13:17:38.583249  6589 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:17:38.590919  9155 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.592617  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.593176  9318 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:38.594328  6594 rpcz_store.cc:276] 0709 13:17:37.317756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.357605 (+ 39849us) service_pool.cc:224] Handling call
0709 13:17:38.581345 (+1223740us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54000933d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:38.581505 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956508544}
I20260709 13:17:38.607878  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:38.614734  9168 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:38.615000  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:38.615693  9318 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.618867  9280 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:38.619182  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.620635  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.621876  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:38.608108  6597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.002s	sys 0.017s
I20260709 13:17:38.615185  9168 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.619334  9280 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:38.632639  6597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.002s	sys 0.018s
I20260709 13:17:38.614882  9078 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.618438  9318 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:38.647523  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1965) took 1376 ms. Trace:
I20260709 13:17:38.647764  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:38.649369  7231 raft_consensus.cc:2410] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:38.648087  6597 rpcz_store.cc:276] 0709 13:17:37.271326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.692261 (+420935us) service_pool.cc:224] Handling call
0709 13:17:37.692696 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:37.693128 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:37.693167 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:37.764300 (+ 71133us) log.cc:844] Serialized 26 byte log entry
0709 13:17:38.646970 (+882670us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:38.647054 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.647186 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:38.647247 (+    61us) raft_consensus.cc:1685] finished
0709 13:17:38.647264 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.647447 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3174,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2080,"thread_start_us":868570,"threads_started":1,"wal-append.queue_time_us":757303}
I20260709 13:17:38.650962  9078 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:38.629477  9168 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.658761  9399 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.659675  9399 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.666832  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:38.667563  9291 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:38.669965  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:38.673035  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:38.673769  6594 raft_consensus.cc:2376] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3. Re-sending same reply.
W20260709 13:17:38.674968  9289 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.888s	user 0.006s	sys 0.011s
I20260709 13:17:38.675457  9289 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.676983  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:38.677616  9289 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:38.677668  6593 raft_consensus.cc:2376] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5. Re-sending same reply.
I20260709 13:17:38.679046  7228 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:38.681365  6514 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:38.683851  9155 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.628922  9280 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:38.687412  7233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 931) took 2058 ms (client timeout 2093 ms). Trace:
W20260709 13:17:38.687938  7233 rpcz_store.cc:269] 0709 13:17:36.628659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.684876 (+ 56217us) service_pool.cc:224] Handling call
0709 13:17:38.686944 (+2002068us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b54000ba754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:38.687211 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203703680}
I20260709 13:17:38.692461  8912 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:38.692962  8912 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:38.696041  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 547) took 1443 ms. Trace:
I20260709 13:17:38.696424  6521 rpcz_store.cc:276] 0709 13:17:37.247093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.247320 (+   227us) service_pool.cc:224] Handling call
0709 13:17:37.247601 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:37.338194 (+ 90593us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:37.874045 (+535851us) log.cc:844] Serialized 14 byte log entry
0709 13:17:38.331575 (+457530us) log.cc:844] Serialized 14 byte log entry
0709 13:17:38.332056 (+   481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:38.332157 (+   101us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:38.332291 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.690289 (+357998us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b54003cc154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:38.690347 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.690547 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":474709,"prepare.queue_time_us":808785,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":137,"raft.queue_time_us":118114,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":99530,"spinlock_wait_cycles":207667456,"thread_start_us":495121,"threads_started":1}
I20260709 13:17:38.697108  9422 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.698315  9274 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:38.698395  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:38.711550  6517 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:38.715149  6521 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.723690  9399 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.727778  9422 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:38.731274  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.736742  9220 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:38.737390  9220 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.681388  9261 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:38.747061  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.752408  9220 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.752887  9376 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:38.758205  9376 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.755792  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.764353  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:38.765064  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:38.770423  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.771396  6594 raft_consensus.cc:2376] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6. Re-sending same reply.
I20260709 13:17:38.705175  9078 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:38.780136  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:38.781522  6597 raft_consensus.cc:2376] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 3. Re-sending same reply.
I20260709 13:17:38.694231  8912 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.745457  9280 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.789222  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:38.800225  7232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 904) took 2532 ms (client timeout 1878 ms). Trace:
W20260709 13:17:38.800719  7232 rpcz_store.cc:269] 0709 13:17:36.238947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.239094 (+   147us) service_pool.cc:224] Handling call
0709 13:17:38.771265 (+2532171us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b540025c454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:38.771483 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4183132544}
I20260709 13:17:38.801764  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 897) took 2583 ms. Trace:
I20260709 13:17:38.802219  7228 rpcz_store.cc:276] 0709 13:17:36.218704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.236508 (+ 17804us) service_pool.cc:224] Handling call
0709 13:17:37.154703 (+918195us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:37.261307 (+106604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.679629 (+1418322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.801444 (+121815us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400575dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:38.801680 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3257254912}
W20260709 13:17:38.803197  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 141360us
W20260709 13:17:38.803740  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.804795  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:38.805099  6668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 527) took 1461 ms (client timeout 1879 ms). Trace:
W20260709 13:17:38.805543  6668 rpcz_store.cc:269] 0709 13:17:37.344060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.344340 (+   280us) service_pool.cc:224] Handling call
0709 13:17:38.804851 (+1460511us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000cce54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:38.804992 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2719442176}
W20260709 13:17:38.806268  6515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 545) took 1648 ms (client timeout 1529 ms). Trace:
W20260709 13:17:38.806707  6515 rpcz_store.cc:269] 0709 13:17:37.157631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.157807 (+   176us) service_pool.cc:224] Handling call
0709 13:17:38.805891 (+1648084us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b540014dcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:38.806054 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434284928}
I20260709 13:17:38.807862  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.808933  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.809934  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:38.811645  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2244) took 1707 ms. Trace:
I20260709 13:17:38.812053  6596 rpcz_store.cc:276] 0709 13:17:37.103870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.393625 (+289755us) service_pool.cc:224] Handling call
0709 13:17:37.417408 (+ 23783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:37.829994 (+412586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.811172 (+981178us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54002f71d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:38.811442 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780328320}
I20260709 13:17:38.814905  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.816222  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:38.817241  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.817917  6517 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:38.819911  6670 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:17:38.818571  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.845268  9220 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:38.846872  9305 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.828034  9403 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:38.853859  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.863524  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.828248  9210 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:38.879837  9078 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.846129  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:38.850554  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.997394  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.996748  6665 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:38.845575  9376 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:38.844399  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:38.823230  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:38.820253  6670 pending_rounds.cc:85] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 7
I20260709 13:17:38.875921  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.824885  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:38.838873  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:38.827629  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:38.866581  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.847205  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:38.828660  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:38.850143  6671 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:38.998143  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:39.004416  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:39.004626  6597 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:39.005086  6515 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:39.005589  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:39.005666  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:39.006071  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:39.006361  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.006471  7228 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
W20260709 13:17:38.957731  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.005674  9403 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:39.009585  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.010840  8949 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.933723  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2249) took 1828 ms. Trace:
W20260709 13:17:38.891698  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.011890  8949 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.013147  6589 rpcz_store.cc:276] 0709 13:17:37.105256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.619978 (+514722us) service_pool.cc:224] Handling call
0709 13:17:38.582595 (+962617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.612726 (+ 30131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.933475 (+320749us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b54000933d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:38.933644 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2212477184}
I20260709 13:17:38.935113  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:39.014460  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:38.973695  6517 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:38.961750  6521 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:17:38.931249  9442 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.351s	user 0.000s	sys 0.003s
I20260709 13:17:39.009442  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.010979  9399 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:38.935663  9262 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:38.993880  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.984257  8912 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:38.945379  9375 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:39.036530  6518 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:39.036792  9375 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:39.039606  6669 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:39.040920  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.041924  6669 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:38.994524  9168 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:38.957731  6416 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 3, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 3 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.006126  9316 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:17:38.919092  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:39.045181  6416 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 4 to 5, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.887409  9393 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:39.001510  6670 pending_rounds.cc:107] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:17:39.002102  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:39.031469  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 130257us
I20260709 13:17:39.032598  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:38.886349  9373 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:39.016783  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:39.046900  7236 raft_consensus.cc:2376] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4. Re-sending same reply.
I20260709 13:17:39.059072  6670 raft_consensus.cc:2889] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:39.018168  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:39.061875  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.066431  6517 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 1.
I20260709 13:17:39.063160  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:39.060551  9291 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.068070  6520 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:39.047243  9078 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.047720  6522 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:39.048233  6663 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:39.050877  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:39.035454  9403 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:38.943410  9006 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.939479  9264 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:39.054471  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 45193.1us
I20260709 13:17:39.027262  9399 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:39.052192  9280 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.091307  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:39.072722  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.076753  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:39.054821  9393 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:39.089114  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.092499  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:39.090062  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:39.095790  7230 raft_consensus.cc:2376] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6. Re-sending same reply.
I20260709 13:17:39.075445  9261 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:39.095526  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 148327us
I20260709 13:17:39.101320  9403 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.105744  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.101987  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:39.092623  9399 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.157400  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:39.075090  9291 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:39.107439  9261 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.158145  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:39.157840  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48499.9us
I20260709 13:17:39.169263  7232 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:39.169898  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:39.175670  7230 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.183992  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:39.179878  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 89240.6us
I20260709 13:17:39.179416  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:39.191360  6597 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:39.195583  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:39.174656  9261 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.200515  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.096427  9305 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:39.199605  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 151728us
I20260709 13:17:39.202268  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:39.209416  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:39.214964  6670 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:17:39.213047  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:39.197921  9264 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:17:39.224776  9264 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:39.225106  9264 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:39.226864  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:39.227615  6588 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:39.227910  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.173600  9403 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.230818  6594 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:39.232193  9403 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.179121  9289 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:17:39.237812  9210 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:39.238309  9210 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.238757  9210 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:39.240718  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46543.8us
I20260709 13:17:39.215114  9411 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:39.242653  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:39.242863  9411 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.243312  9415 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:39.199810  9339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.140s	user 0.011s	sys 0.013s
I20260709 13:17:39.244736  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.245065  9411 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.245816  9406 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:17:39.245965  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 64526.2us
I20260709 13:17:39.246238  9406 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:39.246730  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.247267  9406 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.248340  9406 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:39.248689  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 69374us
I20260709 13:17:39.249457  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.250304  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:39.250878  8949 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.251495  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.252878  9305 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.253427  9305 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:39.253995  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:39.231462  6416 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 3, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 3 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.257517  6416 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 3, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 3 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.258313  8912 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:39.258802  8912 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.259171  8912 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.259342  6416 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:39.262650  9373 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:39.263075  9373 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.263399  9373 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:39.266080  9453 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.284s	user 0.001s	sys 0.001s
I20260709 13:17:39.222204  6671 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:39.251559  8949 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:39.221690  6593 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
W20260709 13:17:39.245132  9339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.361s	user 0.011s	sys 0.013s
I20260709 13:17:39.279978  7230 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:39.274645  6597 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:39.269788  6519 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:39.207548  9329 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.226040  9264 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:39.254377  9305 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:39.263399  9168 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.231070  9291 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:39.252903  9078 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.274590  8949 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.317555  9464 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:39.317895  9376 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:39.318360  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.322779  9396 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:39.355496  9396 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:39.355916  9396 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:39.280150  9316 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:17:39.349730  8949 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:39.360457  9451 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.377s	user 0.001s	sys 0.001s
W20260709 13:17:39.362802  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.323822  9411 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:39.363508  9373 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.363620  9375 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.365190  9375 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:39.326946  9393 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.313539  9403 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:17:39.366336  9393 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:39.366451  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:39.341092  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:39.367353  6513 raft_consensus.cc:606] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:39.343214  9264 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:39.343351  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.343803  9002 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:17:39.373363  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.307315  9155 raft_consensus.cc:2764] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:17:39.348099  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63576.3us
I20260709 13:17:39.283325  9415 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:39.318364  9329 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.354066  9333 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:17:39.350469  9376 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:39.323356  9362 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:39.347178  7232 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:39.313009  9280 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:39.385988  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.365810  9373 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:39.410012  6596 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:39.432312  9399 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.432828  9355 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:39.432978  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:17:39.433617  6513 raft_consensus.cc:606] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:39.433503  9355 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.435266  9002 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:39.435652  9002 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:39.436496  9002 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:39.436733  9392 consensus_queue.cc:1175] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64b95375974b48d18c596a23c9cac75c" responder_term: 9 status { last_received { term: 9 index: 9 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:39.437799  9002 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:39.439744  9210 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.441305  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:39.441948  9210 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:39.434504  9399 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:39.442762  9411 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.443302  9411 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.443688  8949 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.444818  6664 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:39.445294  9411 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.445391  8912 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:39.445842  9411 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:39.447394  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:39.447242  9376 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.448140  6522 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:39.448573  9376 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:39.433956  6513 raft_consensus.cc:614] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:39.451817  9329 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.446357  9411 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.454133  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:39.457010  9411 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.457666  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:39.458415  9464 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:39.461627  9464 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:39.380894  9415 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:39.382856  9362 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:39.400691  9280 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:39.478134  9462 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:39.482870  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.483620  6667 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:17:39.484774  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.487527  8912 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:39.484699  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.481003  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.476400  9415 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:39.482754  9462 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:39.495389  6667 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:39.477715  9362 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.497661  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:39.526592  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.497164  9462 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.543655  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.544450  6667 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:39.539018  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.549187  9396 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.554793  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:39.556706  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:39.542099  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:39.564082  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56425.6us
W20260709 13:17:39.585939  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 74660.9us
I20260709 13:17:39.594267  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:39.593271  6664 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:39.600893  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 356254us
I20260709 13:17:39.596369  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:39.604916  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 375750us
I20260709 13:17:39.598067  9291 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:39.597960  8949 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.600477  9396 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.606374  9316 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:39.634145  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.606841  9355 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.612033  9291 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.591481  9280 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:39.650183  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:39.650360  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 408442us
W20260709 13:17:39.650821  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.626368  9393 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.666162  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:39.612758  9261 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.644124  9291 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.638675  8912 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:39.697947  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 61643.8us
I20260709 13:17:39.696583  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:39.698974  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.636003  9316 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.712895  7232 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:39.715078  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:39.715498  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:39.715816  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.704252  9289 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:39.720623  9155 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:39.720880  9384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.013s	sys 0.009s
I20260709 13:17:39.709760  9476 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:39.743597  9476 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:39.744164  9476 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.728855  9375 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:17:39.729875  9465 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:17:39.730885  9280 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:39.731523  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64470.8us
I20260709 13:17:39.732087  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:39.732791  7229 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:39.733590  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:39.734993  6669 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:17:39.738660  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:39.743116  9384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.014s	sys 0.009s
I20260709 13:17:39.704100  8912 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.695309  9393 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.755610  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:39.762090  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 64350.1us
I20260709 13:17:39.768357  9415 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.777541  7232 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:39.770519  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:39.774422  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.775822  7228 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.779902  9155 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:39.780504  9006 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.671s	user 0.009s	sys 0.002s
I20260709 13:17:39.781458  6665 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:39.782284  9155 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:39.782976  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:39.784657  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:39.796394  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:39.797139  7232 raft_consensus.cc:2368] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
W20260709 13:17:39.798180  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64716.8us
I20260709 13:17:39.783129  9481 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.783866  8912 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.796227  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:39.769599  9289 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:39.815758  7230 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:39.777240  9375 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:39.831619  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.795791  9002 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:39.842242  9261 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:39.842926  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:39.848541  9415 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.762677  8949 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:17:39.851081  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:39.814417  9006 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:39.856578  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:39.857014  7228 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:39.854645  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:17:39.860387  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.856364  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:39.826064  9155 raft_consensus.cc:2764] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:17:39.861745  7232 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:39.869071  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 84509.4us
I20260709 13:17:39.849865  9002 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:39.927561  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 73097.5us
W20260709 13:17:39.927865  9478 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:17:39.904042  8949 consensus_queue.cc:1243] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:17:39.952775  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 98605.9us
I20260709 13:17:39.953202  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.954057  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.866132  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:39.833379  9385 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:39.813602  9373 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:17:39.965329  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:39.967141  9482 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:39.857649  9210 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:39.966604  9385 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:39.968405  9482 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.968071  9462 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.968796  9482 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:39.969794  9482 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:39.956878  9470 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:39.957643  9375 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:39.958884  9289 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:39.966645  8912 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:40.000692  8912 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.001155  8912 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:40.003554  6596 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:39.969851  9385 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:39.962639  9393 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.963083  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:39.855893  9006 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:39.963825  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:39.951416  9399 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:39.967501  9483 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:39.951081  9002 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.971045  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 533) took 1199 ms. Trace:
I20260709 13:17:39.971462  9415 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:40.041389  9415 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.041832  9415 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.972016  7231 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:39.951067  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:40.048267  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.972822  9316 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:39.973109  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:39.973361  9291 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:39.973991  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 545) took 1112 ms. Trace:
W20260709 13:17:39.957039  9467 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:17:39.970314  9462 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:40.053535  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:39.994576  9305 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:39.998512  9482 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:40.007050  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:39.990525  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:39.963611  9476 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.009050  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:39.961091  9376 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
W20260709 13:17:40.010458  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47484.3us
W20260709 13:17:40.036374  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.040203  8949 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:40.040330  9385 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:40.052192  9333 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:39.997917  9362 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.044445  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 93706.9us
I20260709 13:17:40.050280  7237 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:40.015450  6662 rpcz_store.cc:276] 0709 13:17:38.754359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.785807 (+ 31448us) service_pool.cc:224] Handling call
0709 13:17:38.846075 (+ 60268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:39.027800 (+181725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.027829 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:39.071490 (+ 43661us) log.cc:844] Serialized 26 byte log entry
0709 13:17:39.466668 (+395178us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:39.466709 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:39.466774 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:39.950788 (+484014us) raft_consensus.cc:1685] finished
0709 13:17:39.950811 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:39.954004 (+  3193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173499,"raft.queue_time_us":203550,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":23237,"thread_start_us":305932,"threads_started":1,"wal-append.queue_time_us":507056}
I20260709 13:17:40.051223  6665 rpcz_store.cc:276] 0709 13:17:38.861856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.049589 (+187733us) service_pool.cc:224] Handling call
0709 13:17:39.973776 (+924187us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54001e0354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:39.973894 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653101440}
I20260709 13:17:40.010169  9006 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:39.983539  7232 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
W20260709 13:17:40.055058  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 196489us
I20260709 13:17:40.013254  9399 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.043905  9483 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:40.060484  6588 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:40.062597  7236 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:17:40.062630  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 72164.7us
I20260709 13:17:40.067818  6592 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:17:40.040740  6593 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:40.069255  6596 raft_consensus.cc:2410] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:40.066468  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:40.086427  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:40.104223  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.069686  9384 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:40.104455  6592 pending_rounds.cc:85] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 5
I20260709 13:17:40.119190  6592 pending_rounds.cc:107] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:17:40.120066  6592 raft_consensus.cc:2889] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:40.124050  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 57615.5us
I20260709 13:17:40.075753  9333 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:40.094689  9399 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:40.125558  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:40.136608  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:40.077075  8949 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:40.140448  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:40.143999  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:40.139977  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:40.148056  6596 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:40.148221  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:40.149123  7234 raft_consensus.cc:2410] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:40.149940  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.151051  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:40.151459  9316 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:40.151674  6664 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:40.152014  9316 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:40.152969  9316 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.148090  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.135660  9376 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:40.155524  6670 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:40.126448  9384 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.161408  9384 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:40.161810  9472 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:40.162276  9472 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.162658  9472 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:40.098392  9483 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:40.144759  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:40.163370  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:40.175040  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.125419  9411 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:40.110230  9468 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:17:40.141284  9333 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:40.185075  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.184165  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.185781  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:40.187482  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 222250us
I20260709 13:17:40.188469  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.192349  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:40.145751  9261 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:40.195228  7236 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:40.191977  6662 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:40.197057  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.194005  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:40.176178  9481 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:40.202531  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:40.175511  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:40.207273  9392 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:40.231015  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:40.231602  6589 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:40.231292  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:40.232170  6592 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
W20260709 13:17:40.237383  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:40.264179  9464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.000s	sys 0.016s
I20260709 13:17:40.189651  9393 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:17:40.200415  9280 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:17:40.227181  9373 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:40.320278  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 128501us
W20260709 13:17:40.321245  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 142209us
W20260709 13:17:40.323104  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 98724.4us
I20260709 13:17:40.266635  9291 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:40.322463  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 116827us
I20260709 13:17:40.325759  6664 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:40.322701  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:40.326125  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 63631.2us
I20260709 13:17:40.225309  9411 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.305791  9464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.000s	sys 0.016s
I20260709 13:17:40.229147  9375 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:17:40.358273  6636 leader_election.cc:400] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:40.355757  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.360455  6636 leader_election.cc:403] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:40.371495  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.371335  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:40.343705  9376 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:40.378230  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.379511  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:40.319906  8912 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:40.359570  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:40.380403  6663 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:40.394058  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:40.400064  6670 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:40.404711  9204 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:40.407671  9485 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:40.408144  9485 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:40.409060  9485 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:40.409325  9002 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:40.378775  9291 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:40.416021  6596 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:40.360432  9392 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:40.357335  9396 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:17:40.384833  9462 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:40.436595  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:40.442147  6589 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:40.410056  9485 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:40.441661  7233 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:40.453110  9392 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.454866  9493 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:17:40.343044  8949 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.455734  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 131482us
I20260709 13:17:40.462554  6588 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
W20260709 13:17:40.463485  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 87526.5us
I20260709 13:17:40.456434  9289 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:40.419917  9265 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.243s
I20260709 13:17:40.469561  9396 consensus_queue.cc:1243] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:40.463801  9462 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:40.522828  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:40.543656  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 124523us
W20260709 13:17:40.549448  9510 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 13:17:40.561259  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 167708us
W20260709 13:17:40.568166  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:40.568480  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 102414us
I20260709 13:17:40.578538  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2083) took 1117 ms. Trace:
I20260709 13:17:40.548588  9375 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.545792  9521 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:40.584475  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 242204us
I20260709 13:17:40.601011  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:40.602545  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 138326us
W20260709 13:17:40.610605  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45724.3us
I20260709 13:17:40.579048  6667 rpcz_store.cc:276] 0709 13:17:39.460704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.598743 (+138039us) service_pool.cc:224] Handling call
0709 13:17:39.602004 (+  3261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:40.196866 (+594862us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b180044be40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b9079bfb 00007f04b908464b
0709 13:17:40.196971 (+   105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.196993 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:40.223034 (+ 26041us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.352878 (+129844us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:40.352926 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.353008 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:40.567865 (+214857us) raft_consensus.cc:1685] finished
0709 13:17:40.567883 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.578446 (+ 10563us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1967,"raft.queue_time_us":89211,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6594,"spinlock_wait_cycles":8665984,"thread_start_us":122218,"threads_started":2,"wal-append.queue_time_us":246486}
I20260709 13:17:40.519799  9376 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.665424  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76922.2us
W20260709 13:17:40.700922  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 109019us
I20260709 13:17:40.666930  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:40.706712  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:40.709704  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.614144  9411 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:40.618769  9524 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:40.709156  9483 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 13:17:40.595221  9375 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:40.740010  6663 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:40.613765  9521 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.751091  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44487.3us
I20260709 13:17:40.751582  9495 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:40.752218  9384 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.000s
W20260709 13:17:40.753710  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 153874us
I20260709 13:17:40.709352  8912 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:40.728763  9002 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:40.726797  9524 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:40.785311  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:40.774173  9472 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.006s
W20260709 13:17:40.806246  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.769868  9533 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:40.812510  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:40.812080  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 145250us
I20260709 13:17:40.755159  9483 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:40.820430  9466 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.169s	user 0.001s	sys 0.005s
I20260709 13:17:40.812204  9533 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.780400  9384 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:40.756191  9462 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.838165  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 189160us
I20260709 13:17:40.836583  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:40.762779  9505 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 13:17:40.843650  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 69249.4us
I20260709 13:17:40.846648  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:40.813375  9316 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:40.851142  9316 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:40.851622  9316 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:40.854027  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.855334  9372 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:40.855759  9372 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:40.856202  9372 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:40.857033  9520 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:17:40.819291  9002 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:17:40.858753  9528 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:40.860316  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:40.861183  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.811198  9472 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:40.816514  9155 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:17:40.837456  9533 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:40.893801  9504 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:17:40.871181  9002 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:40.894311  9002 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:40.895217  9002 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:40.875379  9526 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:40.897385  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:40.879870  9415 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.838s	user 0.010s	sys 0.002s
I20260709 13:17:40.886353  9289 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:40.880673  9483 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:40.881310  9384 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:40.884074  9521 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:40.884956  9503 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:17:40.885354  9524 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.886909  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58911us
I20260709 13:17:40.887662  9530 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:40.887765  9375 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:40.888625  9482 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.823s	user 0.010s	sys 0.000s
I20260709 13:17:40.890811  9534 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:40.890235  9362 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:40.862936  9506 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:40.896531  9002 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:40.921772  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.912914  9505 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:40.913672  9333 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.000s
I20260709 13:17:40.906212  9291 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:40.906425  9399 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.734s	user 0.012s	sys 0.000s
W20260709 13:17:40.942767  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 127158us
I20260709 13:17:40.919641  9468 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:40.865084  9519 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:17:40.909536  9415 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:40.946945  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:40.946504  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81126us
W20260709 13:17:40.949451  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 102878us
I20260709 13:17:40.925174  9506 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:40.955554  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:40.955876  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 254072us
I20260709 13:17:40.958527  7232 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:40.955885  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:40.957067  7230 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:40.960412  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:40.962546  6518 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:17:40.959556  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:40.943491  9333 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:40.969770  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.968065  7235 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:40.956889  9362 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:40.959975  9506 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:40.976540  9541 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.977159  9541 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:40.977737  9535 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:40.979425  9535 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:40.979842  9535 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:40.980260  9535 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:40.983326  9498 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.604s	user 0.003s	sys 0.000s
W20260709 13:17:40.984287  9485 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.000s
I20260709 13:17:40.988606  9538 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.972297  9534 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.991726  9472 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:40.990679  6514 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:40.980186  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:40.944854  9399 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:40.944578  9291 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:41.008123  6561 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.544s (SENT)
I20260709 13:17:41.017656  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:41.018030  9495 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:41.018905  9375 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:41.020850  6561 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.551s (SENT)
I20260709 13:17:41.021512  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:41.022951  9375 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:40.961702  9468 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.016407  9376 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:41.025142  9376 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:41.025602  9376 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:40.989192  9538 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.029042  9538 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:40.991137  9534 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.029939  9399 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.026268  9468 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:41.030769  7229 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 4: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:41.031659  9534 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.032992  9333 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:40.986543  9533 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.034991  9291 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.036288  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:41.036653  9376 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.038807  9376 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.040238  9533 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.048900  9316 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:41.038872  9291 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:41.041339  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:41.002472  9372 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:40.969097  9542 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:41.038179  6561 leader_election.cc:336] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.576s (SENT)
I20260709 13:17:41.003211  7234 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:40.972679  9482 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:41.080086  9507 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.673s	user 0.007s	sys 0.015s
I20260709 13:17:41.075256  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:41.072043  6666 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:17:41.000795  9385 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.925s	user 0.008s	sys 0.004s
I20260709 13:17:41.094592  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:41.097702  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 138230us
W20260709 13:17:41.098676  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 57469.1us
I20260709 13:17:41.099376  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:41.104647  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:41.106592  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.109009  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:41.110368  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:41.110910  6520 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:41.099169  9396 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.009s
I20260709 13:17:41.116619  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:41.088068  9291 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:41.099895  9481 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:41.092732  9542 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.999982  9362 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:41.107615  9538 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:41.121630  9521 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:41.122051  9372 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:41.123133  9497 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.725s	user 0.003s	sys 0.000s
I20260709 13:17:41.125133  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1889) took 1024 ms. Trace:
W20260709 13:17:41.112406  9006 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.010s	user 0.008s	sys 0.005s
I20260709 13:17:41.107082  9002 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:17:41.119728  9396 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.002s	sys 0.009s
W20260709 13:17:41.132666  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 94523.5us
W20260709 13:17:41.210824  6561 leader_election.cc:336] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.617s (SENT)
W20260709 13:17:41.212113  6561 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.625s (SENT)
I20260709 13:17:41.212632  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:41.138602  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 137052us
I20260709 13:17:41.167094  9522 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:41.130342  9542 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:41.121649  9541 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.128342  9505 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:41.133898  9376 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:41.175923  9264 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:41.134495  9519 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:41.183153  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 929) took 1437 ms. Trace:
I20260709 13:17:41.185206  9495 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:41.185612  9485 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:41.232703  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 137561us
I20260709 13:17:41.233214  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.126293  9476 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.212033  9538 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:41.190050  6515 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:41.215116  9521 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:41.237079  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.192353  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 970) took 1025 ms. Trace:
I20260709 13:17:41.237736  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:41.195268  6562 leader_election.cc:336] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.608s (SENT)
I20260709 13:17:41.214197  9204 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:41.195406  9316 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:41.167966  9549 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:17:41.197992  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 89057.6us
W20260709 13:17:41.150512  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.238451  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:41.237269  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:41.241935  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.127632  9291 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:41.247761  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:41.132791  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:41.227038  7228 rpcz_store.cc:276] 0709 13:17:39.746097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.921943 (+175846us) service_pool.cc:224] Handling call
0709 13:17:39.947351 (+ 25408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:39.983564 (+ 36213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.983594 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:39.998841 (+ 15247us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.226854 (+228013us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:40.226895 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.226971 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.122315 (+895344us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:41.182759 (+ 60444us) raft_consensus.cc:1685] finished
0709 13:17:41.182788 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.183056 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21247,"raft.queue_time_us":32003,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":2914,"spinlock_wait_cycles":20601216,"thread_start_us":227485,"threads_started":1,"wal-append.queue_time_us":352599}
W20260709 13:17:41.187436  9471 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.210s	user 0.003s	sys 0.001s
I20260709 13:17:41.208317  6514 rpcz_store.cc:276] 0709 13:17:40.101126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.108406 (+  7280us) service_pool.cc:224] Handling call
0709 13:17:41.124852 (+1016446us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b5400111054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:41.125053 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715251968}
I20260709 13:17:41.189493  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1993) took 1145 ms. Trace:
I20260709 13:17:41.261664  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:41.261988  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:41.262264  6514 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:41.262586  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:41.261155  6595 rpcz_store.cc:276] 0709 13:17:40.043582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.064013 (+ 20431us) service_pool.cc:224] Handling call
0709 13:17:40.075373 (+ 11360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:40.135491 (+ 60118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.135521 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:40.171915 (+ 36394us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.312749 (+140834us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:40.312911 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.313580 (+   669us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.049693 (+736113us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:41.189126 (+139433us) raft_consensus.cc:1685] finished
0709 13:17:41.189156 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.189417 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11106,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":23343,"spinlock_wait_cycles":13528704,"thread_start_us":116928,"threads_started":1,"wal-append.queue_time_us":197887}
I20260709 13:17:41.265460  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:41.191376  9362 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:41.243027  9475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.278s	user 0.010s	sys 0.012s
I20260709 13:17:41.240581  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.237819  7229 rpcz_store.cc:276] 0709 13:17:40.166584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.167185 (+   601us) service_pool.cc:224] Handling call
0709 13:17:41.192137 (+1024952us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b540027d8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:41.192277 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892841728}
I20260709 13:17:41.240943  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.286101  6593 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:41.286482  6590 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:41.243016  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:41.243734  6594 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:41.223387  9528 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:41.226692  9329 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.226672  9002 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:41.300997  9329 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.301599  9002 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:41.302991  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 107747us
I20260709 13:17:41.303367  9329 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.304040  9329 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:41.229885  9264 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:41.305708  6513 raft_consensus.cc:2410] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:41.305536  9264 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.226544  9376 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:41.234184  6588 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:41.306761  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:41.306924  9532 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:17:41.307564  6513 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:41.307603  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.308480  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:41.235154  9495 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.249495  9305 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:17:41.309496  6595 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:41.309710  9305 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:41.310207  9305 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:41.310691  9495 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.311309  9305 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:41.312676  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.312784  9305 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:41.313298  6589 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:41.247526  9204 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.213074  9006 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.254942  9538 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.240790  9396 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:41.237440  9485 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.247327  9519 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:41.259336  6591 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:41.259222  9505 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.259727  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:41.258224  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:41.277319  6515 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:41.238343  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.331480  6520 raft_consensus.cc:2410] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:41.211127  9506 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:41.290776  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 158066us
I20260709 13:17:41.292806  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.293121  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.334965  6593 raft_consensus.cc:2376] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5. Re-sending same reply.
I20260709 13:17:41.294046  9528 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.337368  7230 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:41.285207  9316 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:41.297170  9555 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.287313  9535 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.342593  9316 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:41.344352  9535 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:41.299264  9494 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.980s	user 0.000s	sys 0.009s
I20260709 13:17:41.344033  9316 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.308822  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.345857  6588 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:41.251859  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:41.288929  9475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.324s	user 0.010s	sys 0.013s
I20260709 13:17:41.347805  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:41.342643  9555 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.349323  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:41.349957  9555 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.359287  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:41.324838  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 94121us
I20260709 13:17:41.364095  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:41.368134  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.369149  7228 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:41.330025  6589 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:41.332139  6516 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:41.379021  7237 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5.
I20260709 13:17:41.379200  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:41.382942  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50452.6us
I20260709 13:17:41.383174  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:41.306942  9376 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:41.384435  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:41.385546  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.386335  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.388722  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.389413  7237 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:41.390583  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.391744  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.395965  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.399870  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.400723  7228 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:41.401607  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.405606  6519 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:41.408094  6588 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:41.411968  7230 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:41.413026  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.410501  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:41.427439  9499 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.043s	user 0.005s	sys 0.000s
I20260709 13:17:41.404822  9481 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:41.447321  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:41.451041  9551 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:41.451454  9551 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.451800  9551 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:41.453243  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.455180  9415 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.438890  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.442721  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:41.334604  9396 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.420778  9515 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.433164  9519 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.443709  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:41.455252  6669 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:41.455247  6664 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:41.455971  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 969) took 1294 ms. Trace:
I20260709 13:17:41.456244  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.457921  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:41.438598  7228 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:41.486393  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:41.486675  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.442279  6596 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:17:41.466143  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.407258  9204 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.479234  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.479470  6588 raft_consensus.cc:2368] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3.
I20260709 13:17:41.481534  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:41.489871  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.481853  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.483121  6513 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:41.463721  6516 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:41.487314  6663 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:41.469025  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:41.485190  7234 rpcz_store.cc:276] 0709 13:17:40.161581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.175291 (+ 13710us) service_pool.cc:224] Handling call
0709 13:17:40.191926 (+ 16635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.100309 (+908383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.455749 (+355440us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b540051f754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:41.455918 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865226624}
I20260709 13:17:41.487860  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:41.491602  6667 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:17:41.490032  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46392.9us
I20260709 13:17:41.493028  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.493705  6596 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:41.493780  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:41.494586  6665 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:41.493100  6522 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:17:41.497556  9568 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:17:41.499918  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 1984) took 1492 ms. Trace:
I20260709 13:17:41.500280  6597 rpcz_store.cc:276] 0709 13:17:40.007734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.008240 (+   506us) service_pool.cc:224] Handling call
0709 13:17:40.013957 (+  5717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:40.014306 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.014338 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:40.040170 (+ 25832us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.196902 (+156732us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:40.268115 (+ 71213us) log.cc:844] Serialized 14 byte log entry
0709 13:17:40.308202 (+ 40087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.308276 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:40.892728 (+584452us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800423700. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:17:41.366913 (+474185us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:41.499717 (+132804us) raft_consensus.cc:1685] finished
0709 13:17:41.499738 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.499862 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1103,"prepare.queue_time_us":66212,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":41687,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":231,"spinlock_wait_cycles":17234944,"thread_start_us":156372,"threads_started":1,"wal-append.queue_time_us":271971}
I20260709 13:17:41.504817  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.505625  7236 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:41.508477  6596 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:41.509111  6669 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:41.483621  9482 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:41.513798  6488 leader_election.cc:400] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3.
I20260709 13:17:41.514103  6488 leader_election.cc:403] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:41.514999  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.513497  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:41.519843  6664 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 2.
I20260709 13:17:41.518303  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:41.506399  9532 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:41.521239  6596 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:41.521389  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:41.525225  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.525933  6588 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:41.525766  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:41.529139  6669 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:41.527983  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.527144  9495 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:17:41.531100  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:41.531546  9523 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:41.531786  6588 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 3.
I20260709 13:17:41.532050  9523 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:41.532609  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:41.630117  6666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 548) took 1481 ms (client timeout 1578 ms). Trace:
I20260709 13:17:41.492704  9515 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.630465  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:41.657464  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.658150  6588 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:41.658399  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.636839  7236 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:17:41.607998  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 77555.2us
W20260709 13:17:41.630658  6666 rpcz_store.cc:269] 0709 13:17:40.149181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.176342 (+ 27161us) service_pool.cc:224] Handling call
0709 13:17:41.629945 (+1453603us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54000e65d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:41.630059 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580252032}
I20260709 13:17:41.614954  6596 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:41.631731  9485 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.663982  9495 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:17:41.641139  9526 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.669100  9526 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.671041  9536 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.671574  9536 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:41.671998  9551 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:41.673700  9536 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.676618  9551 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:41.517758  9522 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:41.678292  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.520339  9392 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.159s
I20260709 13:17:41.524700  9532 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.533952  8949 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:41.680845  6666 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:17:41.682085  8949 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.693270  8949 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:41.518159  9316 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:41.681617  9532 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:41.684510  7237 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:41.697537  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:41.698194  7237 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5.
I20260709 13:17:41.684949  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.685042  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.683969  6596 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:17:41.678414  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 235736us
I20260709 13:17:41.688629  9524 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:41.681367  7236 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:41.694242  9316 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:41.517390  9411 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 2. Current term is 3.
W20260709 13:17:41.700610  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.700555  6588 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:17:41.727792  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.631405  9393 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:41.709504  6595 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:41.713239  6667 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:41.713517  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:41.730489  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.716498  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.720616  6591 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:17:41.733107  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.634457  9006 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:41.737279  6561 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.769s (SENT)
I20260709 13:17:41.737893  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:41.748165  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.747499  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:41.753760  7234 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:41.756014  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.698592  9506 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.763306  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.764122  7237 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:41.634435  9542 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:41.765918  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:41.771654  7236 raft_consensus.cc:2368] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6.
I20260709 13:17:41.772995  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:41.774057  6663 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:41.730931  9393 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:41.775935  6665 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:41.777184  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:41.781188  7237 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:41.719197  9291 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:41.795754  9526 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:41.804797  9515 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:41.816324  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:41.807165  9567 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:17:41.809909  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:41.810822  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:41.843638  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 115234us
I20260709 13:17:41.844179  6594 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:41.811508  7233 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:41.812187  9542 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:41.818989  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:41.821377  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:41.621258  9523 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:41.780645  9006 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.157s
W20260709 13:17:41.991340  6514 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.729s	user 0.005s	sys 0.005s
W20260709 13:17:42.768313  9524 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.053s	user 0.007s	sys 0.004s
W20260709 13:17:42.016350  9516 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.029s	sys 0.062s
W20260709 13:17:42.769066  9516 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.289s	user 0.044s	sys 0.064s
I20260709 13:17:42.769439  6513 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:17:42.769832  6513 pending_rounds.cc:85] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 6
I20260709 13:17:42.770067  6513 pending_rounds.cc:107] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:17:42.770303  6513 raft_consensus.cc:2889] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:42.309854  9505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.011s	sys 0.002s
I20260709 13:17:42.772073  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 698) took 1404 ms. Trace:
W20260709 13:17:42.753224  9376 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.355s	user 0.007s	sys 0.005s
I20260709 13:17:42.768308  6514 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:41.909768  9556 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.865s
W20260709 13:17:42.774668  6664 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.005s	sys 0.011s
I20260709 13:17:41.895620  9291 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:42.814266  6664 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.005s	sys 0.011s
I20260709 13:17:42.422197  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 701) took 1014 ms. Trace:
I20260709 13:17:42.777499  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:41.937075  9515 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:41.952411  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:42.221974  9581 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:17:42.090103  9579 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:41.896270  9532 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:41.892427  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.773456  9517 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:42.780354  6596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.004s	sys 0.012s
W20260709 13:17:42.826574  6596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.004s	sys 0.012s
I20260709 13:17:41.911799  9316 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:42.780704  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 987339us
I20260709 13:17:42.231855  9530 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:17:42.461906  7234 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.702s	user 0.006s	sys 0.003s
I20260709 13:17:42.367835  6517 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:42.781332  9542 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.793761  9396 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:42.783864  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.306283  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.899875  8949 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:42.789310  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:41.960079  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:42.843187  6662 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:42.243975  6635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.005s	sys 0.029s
W20260709 13:17:42.263243  6669 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.002s
I20260709 13:17:42.113435  9546 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:42.791677  6667 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.000s	sys 0.024s
I20260709 13:17:42.845077  6669 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:17:42.845534  6667 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.000s	sys 0.025s
I20260709 13:17:42.845760  7234 raft_consensus.cc:2410] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:42.845870  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2112) took 1333 ms. Trace:
W20260709 13:17:42.792029  6487 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.002s	sys 0.023s
I20260709 13:17:42.846745  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 843) took 1163 ms. Trace:
W20260709 13:17:42.847080  6487 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.002s	sys 0.023s
I20260709 13:17:42.846474  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:41.846582  9578 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:17:42.793617  6595 rpcz_store.cc:276] 0709 13:17:41.367760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.368185 (+   425us) service_pool.cc:224] Handling call
0709 13:17:42.771903 (+1403718us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b540023cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:42.772014 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657955072}
I20260709 13:17:42.870378  7237 raft_consensus.cc:2410] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:41.964833  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:42.811046  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 562) took 1046 ms. Trace:
W20260709 13:17:42.195310  9570 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.564s	user 0.004s	sys 0.000s
I20260709 13:17:42.814764  6591 rpcz_store.cc:276] 0709 13:17:41.407738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.407836 (+    98us) service_pool.cc:224] Handling call
0709 13:17:42.421930 (+1014094us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54002fdad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:42.422088 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705932544}
W20260709 13:17:42.811679  9505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.013s	sys 0.014s
I20260709 13:17:42.818578  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 563) took 1039 ms. Trace:
W20260709 13:17:42.810752  9573 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.002s	sys 0.019s
W20260709 13:17:42.821071  6666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.024s	sys 0.010s
W20260709 13:17:42.902127  6666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.024s	sys 0.011s
I20260709 13:17:42.781692  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:42.823794  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 871368us
I20260709 13:17:41.904235  9411 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.908s
I20260709 13:17:42.145908  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:42.844431  6635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.011s	sys 0.050s
I20260709 13:17:42.847487  7234 rpcz_store.cc:276] 0709 13:17:41.683614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.718585 (+ 34971us) service_pool.cc:224] Handling call
0709 13:17:42.846542 (+1127957us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:42.847777  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 1.01186e+06us
I20260709 13:17:42.828837  9516 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:42.847131  6669 rpcz_store.cc:276] 0709 13:17:41.512686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.517496 (+  4810us) service_pool.cc:224] Handling call
0709 13:17:42.845761 (+1328265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:42.872011  6663 rpcz_store.cc:276] 0709 13:17:41.764417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.778805 (+ 14388us) service_pool.cc:224] Handling call
0709 13:17:42.810976 (+1032171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:42.819681  6514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 564) took 1555 ms (client timeout 1842 ms). Trace:
I20260709 13:17:42.894640  6588 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:42.897225  6665 rpcz_store.cc:276] 0709 13:17:41.779545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.818049 (+ 38504us) service_pool.cc:224] Handling call
0709 13:17:42.818415 (+1000366us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:42.825254  9210 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.503s	user 0.015s	sys 0.035s
I20260709 13:17:42.903177  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 557) took 1192 ms. Trace:
I20260709 13:17:42.814471  9376 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:42.903855  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:42.849968  9481 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.005s	sys 0.028s
I20260709 13:17:42.915805  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:42.921684  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:42.929009  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2003) took 1659 ms. Trace:
I20260709 13:17:42.931813  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:42.933051  6669 raft_consensus.cc:2410] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:42.935873  6668 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:42.936584  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 566) took 1033 ms. Trace:
I20260709 13:17:42.936972  6668 rpcz_store.cc:276] 0709 13:17:41.903072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.091464 (+188392us) service_pool.cc:224] Handling call
0709 13:17:42.936513 (+845049us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:42.910086  9573 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.274s	user 0.003s	sys 0.019s
I20260709 13:17:42.929659  6590 rpcz_store.cc:276] 0709 13:17:41.269540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.316641 (+ 47101us) service_pool.cc:224] Handling call
0709 13:17:41.367666 (+ 51025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:41.388786 (+ 21120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.388827 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:41.399011 (+ 10184us) log.cc:844] Serialized 26 byte log entry
0709 13:17:41.760373 (+361362us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:41.760410 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.760477 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:42.928705 (+1168228us) raft_consensus.cc:1685] finished
0709 13:17:42.928739 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.928929 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20708,"raft.queue_time_us":21282,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5071,"thread_start_us":280294,"threads_started":1,"wal-append.queue_time_us":731540}
W20260709 13:17:42.934244  9481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.005s	sys 0.029s
I20260709 13:17:42.951083  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:42.951840  7228 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:42.926191  6666 rpcz_store.cc:276] 0709 13:17:41.711202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.711331 (+   129us) service_pool.cc:224] Handling call
0709 13:17:41.711623 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.711811 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:42.033598 (+321787us) log.cc:844] Serialized 14 byte log entry
0709 13:17:42.902510 (+868912us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:42.902559 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:42.902900 (+   341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.902989 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.903117 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":135558,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":13700,"thread_start_us":1150502,"threads_started":2}
W20260709 13:17:42.934737  9210 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.627s	user 0.015s	sys 0.036s
I20260709 13:17:42.958914  6671 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:42.871662  9542 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:42.922466  7231 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:42.958024  6669 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:42.977936  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 567) took 1074 ms. Trace:
I20260709 13:17:42.978418  6669 rpcz_store.cc:276] 0709 13:17:41.903392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.942042 (+1038650us) service_pool.cc:224] Handling call
0709 13:17:42.957786 (+ 15744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.977652 (+ 19866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.977855 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:42.977494  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 201991us
I20260709 13:17:42.966884  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:42.979928  6666 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:42.948550  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:42.960105  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 187042us
I20260709 13:17:42.957504  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:42.943279  9516 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:42.946066  9411 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:42.980755  7234 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:42.980944  6665 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:42.963934  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:42.982648  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.035009  6668 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:42.983657  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.985908  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:42.987447  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 558) took 1258 ms. Trace:
I20260709 13:17:43.046770  6667 rpcz_store.cc:276] 0709 13:17:41.729591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.731046 (+  1455us) service_pool.cc:224] Handling call
0709 13:17:41.731316 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.731498 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:42.846285 (+1114787us) log.cc:844] Serialized 14 byte log entry
0709 13:17:42.987020 (+140735us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:42.987088 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:42.987147 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.987235 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.987379 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19838,"prepare.queue_time_us":116075,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"thread_start_us":1217958,"threads_started":2}
I20260709 13:17:42.996563  6588 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:43.005064  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:42.997305  6561 connection.cc:256] RPC call timeout handler was delayed by -1.2190489059998981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
I20260709 13:17:42.988703  9516 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:42.989356  7229 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5.
I20260709 13:17:42.948477  9006 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:42.924849  6514 rpcz_store.cc:269] 0709 13:17:41.260970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.261418 (+   448us) service_pool.cc:224] Handling call
0709 13:17:42.816726 (+1555308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:42.967336  9291 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.050103  6591 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:42.981686  9531 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:17:43.052161  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.087000  6667 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.053121  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.091076  7232 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
W20260709 13:17:43.091289  9594 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:17:43.051743  6561 connection.cc:256] RPC call timeout handler was delayed by -1.1823825650001254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.336s for a response
W20260709 13:17:43.092769  9591 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:17:43.093065  6561 connection.cc:256] RPC call timeout handler was delayed by -1.1632489060000353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2311s for a response
W20260709 13:17:43.093530  6561 connection.cc:256] RPC call timeout handler was delayed by -1.1193993489998775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 13:17:43.094046  6561 leader_election.cc:336] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.886s (SENT)
I20260709 13:17:43.094388  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:43.071161  6669 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:43.011248  9589 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.051450  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.100730  6666 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:43.074697  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.101311  7234 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:43.076763  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.078224  9605 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:43.078634  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.080732  9542 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:43.061966  9411 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.083580  9604 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:43.084342  9599 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:43.109622  9593 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:17:43.092818  9611 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:43.094288  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.117257  7228 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:43.117406  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.118245  6663 raft_consensus.cc:2410] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:43.118796  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.119366  7236 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
W20260709 13:17:43.005848  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.103175  6667 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
W20260709 13:17:43.104419  6561 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.961s (SENT)
I20260709 13:17:43.120574  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:43.105261  9584 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:17:43.104904  9393 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.049714  9596 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.108161  6662 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:43.109616  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:43.115262  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 165878us
I20260709 13:17:43.116307  7229 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:43.121093  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.121675  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:42.848903  9530 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:42.896595  9546 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.111539  9599 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.161054  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.167235  6669 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:43.162901  7234 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:43.162932  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 564) took 1319 ms. Trace:
I20260709 13:17:43.168529  6662 rpcz_store.cc:276] 0709 13:17:41.844009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.959789 (+115780us) service_pool.cc:224] Handling call
0709 13:17:43.162875 (+1203086us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:43.168941  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.164494  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:43.160698  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.171451  6667 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:43.166342  7232 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:43.072350  9505 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:17:43.184553  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:43.194410  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 214024us
W20260709 13:17:43.175124  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 189281us
I20260709 13:17:43.134831  9542 raft_consensus.cc:2764] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:17:43.207515  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 170063us
W20260709 13:17:43.207314  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.205080  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1901) took 1534 ms. Trace:
W20260709 13:17:43.157760  7198 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.849s (SENT) [suppressed 44 similar messages]
I20260709 13:17:43.094739  9006 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.209403  6513 rpcz_store.cc:276] 0709 13:17:41.674761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.675265 (+   504us) service_pool.cc:224] Handling call
0709 13:17:41.675645 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.770745 (+1095100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.204779 (+434034us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001e9454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:43.205009 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2514618752}
I20260709 13:17:43.209540  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.128459  9515 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:43.212122  9515 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.101212  9291 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.215269  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.216781  7237 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:43.217319  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 97786.8us
I20260709 13:17:43.218954  9612 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.220709  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:43.222941  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.227507  6662 raft_consensus.cc:2368] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:43.230086  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.113181  9589 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.214824  9393 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.201238  8949 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:43.222604  9598 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:17:43.233669  6667 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:43.140044  9604 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.243942  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.291633  6662 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:43.223618  9606 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.162490  9611 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:43.259652  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.263553  9524 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:43.268602  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 104206us
W20260709 13:17:43.228811  9580 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:17:43.278827  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2110) took 1789 ms. Trace:
I20260709 13:17:43.279417  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.280248  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 562) took 2019 ms. Trace:
I20260709 13:17:43.282117  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 841) took 1658 ms. Trace:
I20260709 13:17:43.276099  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.298889  9608 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:43.299557  9622 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:43.313915  7232 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:17:43.314266  7232 pending_rounds.cc:85] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 4
I20260709 13:17:43.242108  9532 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:43.305048  7233 rpcz_store.cc:276] 0709 13:17:41.623381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.623574 (+   193us) service_pool.cc:224] Handling call
0709 13:17:41.641593 (+ 18019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.792455 (+1150862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.281836 (+489381us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54004e0554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:43.282043 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390163840}
I20260709 13:17:43.268078  9519 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:43.241104  9513 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:43.309692  7235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 858) took 1257 ms (client timeout 1571 ms). Trace:
I20260709 13:17:43.303186  6517 rpcz_store.cc:276] 0709 13:17:41.260402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.263185 (+  2783us) service_pool.cc:224] Handling call
0709 13:17:41.263465 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.834444 (+1570979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.279971 (+445527us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540014e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:43.280175 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2539168640}
I20260709 13:17:43.314478  7232 pending_rounds.cc:107] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:17:43.303463  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:43.301591  6671 rpcz_store.cc:276] 0709 13:17:41.490861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.503698 (+ 12837us) service_pool.cc:224] Handling call
0709 13:17:41.503990 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.959457 (+1455467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.278488 (+319031us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540020fdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:43.278736 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3345400704}
I20260709 13:17:43.222678  9607 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
W20260709 13:17:43.388636  7235 rpcz_store.cc:269] 0709 13:17:42.052028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.052402 (+   374us) service_pool.cc:224] Handling call
0709 13:17:43.279230 (+1226828us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:43.309613 (+ 30383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12579584}
I20260709 13:17:43.239455  9609 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:43.355674  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.173837  9596 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.278596  9376 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:43.369493  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 265077us
W20260709 13:17:43.368891  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 171403us
W20260709 13:17:43.341748  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.302129  7231 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:43.231772  9612 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.389804  7232 raft_consensus.cc:2889] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:43.389734  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 86362us
W20260709 13:17:43.401217  6561 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.974s (SENT)
I20260709 13:17:43.412370  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:43.407997  6663 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:43.414016  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.414944  6663 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:43.415881  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.416791  6663 raft_consensus.cc:2410] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:43.417644  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.392879  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.419095  6662 raft_consensus.cc:2368] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:43.419983  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.420778  6662 raft_consensus.cc:2368] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:43.421579  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:43.357224  9584 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:43.423861  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2106) took 1953 ms. Trace:
I20260709 13:17:43.424273  6670 rpcz_store.cc:276] 0709 13:17:41.470488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.481978 (+ 11490us) service_pool.cc:224] Handling call
0709 13:17:41.496898 (+ 14920us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:41.523022 (+ 26124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.523078 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:41.523743 (+   665us) log.cc:844] Serialized 52 byte log entry
0709 13:17:41.964421 (+440678us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:42.771914 (+807493us) log.cc:844] Serialized 14 byte log entry
0709 13:17:42.772100 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.772182 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.423626 (+651444us) raft_consensus.cc:1685] finished
0709 13:17:43.423650 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.423804 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21758,"prepare.queue_time_us":813955,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":80598,"raft.run_cpu_time_us":595,"raft.run_wall_time_us":83415,"thread_start_us":328631,"threads_started":1,"wal-append.queue_time_us":1316086}
W20260709 13:17:43.360467  9593 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:17:43.409807  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:43.418543  6663 raft_consensus.cc:2410] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:43.305985  9604 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:43.307991  9606 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.429804  9596 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:43.431128  9411 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:43.276072  9210 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:17:43.478951  9210 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:43.479321  9210 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:43.480461  9597 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.708s	user 0.004s	sys 0.000s
I20260709 13:17:43.480155  9210 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.481367  9210 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:43.432114  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.483274  9006 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:43.483484  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.484036  6662 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:43.484818  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 560) took 1729 ms. Trace:
I20260709 13:17:43.485412  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 863) took 1253 ms. Trace:
I20260709 13:17:43.485855  7237 rpcz_store.cc:276] 0709 13:17:42.232459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.903550 (+671091us) service_pool.cc:224] Handling call
0709 13:17:43.485163 (+581613us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400677354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:43.485338 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666720640}
I20260709 13:17:43.172886  9516 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.433156  6487 leader_election.cc:400] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6.
I20260709 13:17:43.487118  9516 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:43.437305  6665 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:43.488025  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 565) took 1594 ms. Trace:
I20260709 13:17:43.488210  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.488468  6665 rpcz_store.cc:276] 0709 13:17:41.894014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.936173 (+1042159us) service_pool.cc:224] Handling call
0709 13:17:43.487954 (+551781us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:43.488826  6669 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:43.489697  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.485172  6664 rpcz_store.cc:276] 0709 13:17:41.755489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.755639 (+   150us) service_pool.cc:224] Handling call
0709 13:17:41.767990 (+ 12351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:41.793013 (+ 25023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.793048 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:41.892682 (+ 99634us) log.cc:844] Serialized 52 byte log entry
0709 13:17:42.818685 (+926003us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:42.818767 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.818898 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.479818 (+660920us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:43.484554 (+  4736us) raft_consensus.cc:1685] finished
0709 13:17:43.484573 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.484762 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":761,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":51352,"spinlock_wait_cycles":40921856,"thread_start_us":821306,"threads_started":1,"wal-append.queue_time_us":810935}
I20260709 13:17:43.491850  9396 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:43.492378  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.442972  9604 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:43.448628  9611 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:43.357085  9519 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:43.458992  9291 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:43.394369  9619 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:43.471875  9612 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:43.472980  9599 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:43.397799  9579 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.507009  9608 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:43.523573  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 328277us
I20260709 13:17:43.325094  9524 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.489605  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.527846  6665 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:43.529031  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.508777  9609 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.442667  9617 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:17:43.493106  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.290900  9542 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
W20260709 13:17:43.493525  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 268241us
I20260709 13:17:43.455951  9530 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:43.504976  6561 connection.cc:256] RPC call timeout handler was delayed by -1.027622127000086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3053s for a response
I20260709 13:17:43.389339  9513 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:43.506672  7200 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:43.511672  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 707) took 1836 ms. Trace:
I20260709 13:17:43.513211  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.516139  6668 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:43.552963  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.517097  6662 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3.
I20260709 13:17:43.554308  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.555003  6662 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:43.478230  9624 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:17:43.509938  9523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.012s	sys 0.018s
W20260709 13:17:43.526372  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 317177us
I20260709 13:17:43.559232  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:43.559803  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:17:43.487085  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 54095.9us
I20260709 13:17:43.529850  6665 raft_consensus.cc:2368] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:43.433859  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:43.530848  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 373099us
W20260709 13:17:43.511229  9538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.880s	user 0.027s	sys 0.013s
I20260709 13:17:43.532017  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:43.534982  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:43.539078  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:43.567117  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.567538  6517 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:43.567865  6662 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:43.540280  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.555253  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.555651  6596 rpcz_store.cc:276] 0709 13:17:41.674895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.714796 (+ 39901us) service_pool.cc:224] Handling call
0709 13:17:41.715132 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:41.715586 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:41.756467 (+ 40881us) log.cc:844] Serialized 14 byte log entry
0709 13:17:43.003376 (+1246909us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.009093 (+  5717us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.009271 (+   178us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:43.202896 (+193625us) log.cc:844] Serialized 14 byte log entry
0709 13:17:43.506899 (+304003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.510793 (+  3894us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.511398 (+   605us) raft_consensus.cc:1685] finished
0709 13:17:43.511422 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.511596 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47419,"prepare.queue_time_us":335117,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":44656,"raft.queue_time_us":297936,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":53190,"thread_start_us":1634852,"threads_started":4,"wal-append.queue_time_us":1170796}
I20260709 13:17:43.518731  9615 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:43.564257  6671 raft_consensus.cc:2376] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8. Re-sending same reply.
I20260709 13:17:43.574895  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.575361  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.575743  6669 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:43.575575  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:43.526281  9604 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:43.525611  9589 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:43.576057  6668 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:17:43.572865  6487 leader_election.cc:336] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.571s (SENT)
W20260709 13:17:43.576866  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 70305.6us
I20260709 13:17:43.581851  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:43.577752  6561 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.249s (SENT)
I20260709 13:17:43.491801  9628 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:17:43.579344  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:43.584924  6487 leader_election.cc:336] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.582s (SENT)
W20260709 13:17:43.574802  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44164.6us
I20260709 13:17:43.588197  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.588404  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.589033  6662 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:17:43.588876  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:43.532346  9606 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:43.590219  6521 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:17:43.602900  6561 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.311s (SENT)
I20260709 13:17:43.594388  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.603443  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:43.604199  6665 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:43.591351  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.602283  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:43.509873  9601 consensus_queue.cc:1175] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64b95375974b48d18c596a23c9cac75c" responder_term: 5 status { last_received { term: 3 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: [4.7-4.7]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:17:43.611100  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51947.1us
W20260709 13:17:43.612664  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.610482  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.615048  6662 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:43.593927  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.594043  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:43.617005  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.594652  9546 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:43.550545  9411 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:43.589231  9530 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:43.595959  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:43.617764  6664 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:43.574225  9615 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:43.597517  9604 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:43.559329  9523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.012s	sys 0.018s
I20260709 13:17:43.601461  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:43.606527  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:43.574354  9538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.936s	user 0.028s	sys 0.013s
I20260709 13:17:43.585091  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:43.609817  6487 leader_election.cc:336] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.670s (SENT)
I20260709 13:17:43.596338  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:43.608902  9629 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:17:43.618367  9393 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:17:43.555784  9609 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.625630  9634 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:43.578562  9505 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:43.580869  9608 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.623773  6514 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:43.649509  6520 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:43.625466  6561 leader_election.cc:336] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.491s (SENT)
W20260709 13:17:43.650274  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54044.7us
I20260709 13:17:43.531307  9579 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.580021  9519 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.635865  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.627315  6669 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:17:43.629657  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:43.628101  6487 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.723s (SENT)
I20260709 13:17:43.650424  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:43.590049  9619 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:43.650547  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44094.4us
I20260709 13:17:43.606540  9627 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.589826  9396 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:17:43.631129  9524 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:43.683701  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:43.633173  9622 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:43.684875  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56789.7us
I20260709 13:17:43.635229  9634 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.685438  6668 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:43.686080  6664 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:43.695231  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:43.624025  9530 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.699502  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.720012  9530 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:43.720299  6669 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:43.637390  9606 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:43.690433  6662 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:43.720811  9606 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.631238  9626 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:43.721310  9606 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:43.682322  6671 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:17:43.666296  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.652966  9628 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:43.726919  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.728282  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44663us
I20260709 13:17:43.649535  9589 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:43.655615  9617 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:43.682040  9599 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:43.735486  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 110043us
I20260709 13:17:43.681385  9639 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:43.737819  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 151013us
W20260709 13:17:43.739607  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.601217  9624 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.718568  9618 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:43.766747  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.736620  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 128400us
I20260709 13:17:43.773042  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.653553  9393 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:43.734302  9546 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:43.785274  6591 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:43.788130  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.715252  9515 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:43.789299  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.835506  6668 raft_consensus.cc:2376] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3. Re-sending same reply.
W20260709 13:17:43.791102  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 61977.3us
I20260709 13:17:43.734752  9617 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.718779  9622 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:43.729538  9634 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:43.822733  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 55630.5us
I20260709 13:17:43.651513  9411 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.698060  9538 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:17:43.861240  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:43.842778  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:43.843433  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.843518  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:43.844399  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 112500us
I20260709 13:17:43.844779  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.846026  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:43.776338  9618 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:43.847658  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 67241.2us
I20260709 13:17:43.836971  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:43.850510  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 131510us
I20260709 13:17:43.852264  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.871520  6591 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.730135  9620 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 13:17:43.655952  9608 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.862365  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:43.785451  9641 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:43.862440  6594 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:43.866317  6593 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:43.866348  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:43.872830  6590 raft_consensus.cc:2368] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6.
I20260709 13:17:43.887627  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:43.888196  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.889665  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.891053  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:43.834116  6596 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:43.771307  9589 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.844324  9546 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:43.895524  9546 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.848929  9628 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:43.849277  9617 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:43.639101  9513 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.768154  9599 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.715349  9627 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.727473  9619 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.848656  9505 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:43.899860  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:43.900383  6588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2017) took 2091 ms (client timeout 2111 ms). Trace:
I20260709 13:17:43.904721  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:43.820914  9393 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.862097  9639 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:43.852530  9515 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:43.739312  9615 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:43.951555  7233 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:43.986243  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.960395  6593 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:43.877596  9618 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:44.070060  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:43.965112  6591 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:43.926378  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:44.072316  6588 rpcz_store.cc:269] 0709 13:17:41.809101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.809582 (+   481us) service_pool.cc:224] Handling call
0709 13:17:43.900202 (+2090620us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002f62d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:43.900322 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416679168}
I20260709 13:17:44.074268  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.065517  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2117) took 1029 ms. Trace:
I20260709 13:17:44.256062  6597 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:43.898054  9626 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:44.092242  6668 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:44.089195  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:44.176894  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.089286  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:43.970649  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.259290  6514 raft_consensus.cc:2410] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:17:44.259672  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.260371  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.260766  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.256412  6666 rpcz_store.cc:276] 0709 13:17:43.036476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.064710 (+1028234us) service_pool.cc:224] Handling call
0709 13:17:44.065028 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.065204 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.065228 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:44.065266 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:44.065295 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.065349 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.065449 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.257880  6521 raft_consensus.cc:2368] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:44.264389  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:44.265766  6596 raft_consensus.cc:2410] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:17:44.266803  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:44.267675  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.267879  6514 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:44.272820  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:44.277508  6561 leader_election.cc:336] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.079s (SENT)
I20260709 13:17:44.278415  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.280455  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.282312  6590 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:44.265094  6666 raft_consensus.cc:2376] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 3. Re-sending same reply.
W20260709 13:17:44.285146  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 397567us
I20260709 13:17:44.286096  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:44.290100  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:44.290489  6514 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 1. Current term is 3. Ops: []
I20260709 13:17:44.290665  6597 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:44.289924  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.283702  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:43.909089  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:43.973867  9538 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:43.969645  9637 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:17:43.957653  9655 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:44.090497  9530 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.294061  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:44.331493  6597 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:43.931610  9210 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:44.296026  6594 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:44.296739  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:44.348624  6588 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:43.958137  9596 raft_consensus.cc:2764] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:17:43.970263  9636 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:17:43.941403  9651 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:44.297653  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:43.886150  9629 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.397s
I20260709 13:17:43.906654  9641 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:44.359331  6596 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:43.977981  9658 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.306246  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:44.307163  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:44.312884  7232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.064s	sys 0.056s
I20260709 13:17:44.282675  9006 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:44.308897  6637 leader_election.cc:400] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6.
I20260709 13:17:44.373734  6637 leader_election.cc:403] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:44.319156  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:44.319370  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.089754  9579 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:44.321026  6668 raft_consensus.cc:1217] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.7->[8.8-8.8]   Dedup: 8.8->[]
I20260709 13:17:44.323055  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:43.940635  9606 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.293695  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:44.363269  9645 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.482s	user 0.000s	sys 0.001s
I20260709 13:17:43.900590  9628 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:44.337452  6517 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:43.970399  9653 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.383s
I20260709 13:17:44.341792  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:44.323146  9615 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:44.355459  6518 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:44.342316  9210 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:44.363809  7232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.064s	sys 0.056s
I20260709 13:17:44.362605  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:44.404978  6588 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:44.363922  6591 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:44.364423  6514 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:44.366953  9658 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.410534  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:44.371260  6596 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:44.373040  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.337620  9655 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:44.376587  6521 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:17:44.388108  9650 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.482s	user 0.000s	sys 0.001s
I20260709 13:17:44.378687  6666 raft_consensus.cc:2376] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6. Re-sending same reply.
I20260709 13:17:44.422273  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.300438  9538 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:44.360589  9624 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:43.941169  9622 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:44.382642  6487 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.799s (SENT)
I20260709 13:17:44.431664  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:44.385066  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.358119  9542 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:44.389670  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:44.403416  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.379096  9596 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:44.386428  9642 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.465s	user 0.000s	sys 0.001s
I20260709 13:17:44.377352  6668 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:17:44.417012  6594 raft_consensus.cc:2410] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:44.433194  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1759) took 1344 ms. Trace:
I20260709 13:17:44.433483  6668 rpcz_store.cc:276] 0709 13:17:43.088492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.268842 (+1180350us) service_pool.cc:224] Handling call
0709 13:17:44.299364 (+ 30522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.432961 (+133597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.433146 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.382548  9579 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6.
I20260709 13:17:44.421053  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.430685  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:44.404584  9411 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.477216  6514 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:44.417435  9657 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:44.484664  6521 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:44.486761  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:44.486922  6590 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:44.490890  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:44.487609  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 105028us
I20260709 13:17:44.410820  9599 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:44.492355  6487 leader_election.cc:336] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.799s (SENT)
I20260709 13:17:44.492864  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.496163  6591 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
W20260709 13:17:44.496819  6487 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.859s (SENT)
I20260709 13:17:44.497377  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.498908  6588 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:44.502164  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:44.503285  6591 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:44.504474  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71773.8us
I20260709 13:17:44.502532  6588 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:44.426964  9608 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.382462  9505 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.343007  9639 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.439313  9596 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.485558  9624 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:44.524811  7233 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
W20260709 13:17:44.522256  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 228264us
I20260709 13:17:44.523361  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2009) took 3119 ms. Trace:
W20260709 13:17:44.523579  6487 leader_election.cc:336] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.876s (SENT)
I20260709 13:17:44.530527  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.529842  6589 rpcz_store.cc:276] 0709 13:17:41.385296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.638935 (+253639us) service_pool.cc:224] Handling call
0709 13:17:41.641267 (+  2332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:41.641711 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.641751 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:41.657102 (+ 15351us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.487229 (+1830127us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b1800469e40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:17:43.702959 (+215730us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:43.703009 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.703095 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.489813 (+786718us) raft_consensus.cc:1685] finished
0709 13:17:44.489829 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.505187 (+ 15358us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15627,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5774,"spinlock_wait_cycles":23055232,"thread_start_us":215538,"threads_started":1,"wal-append.queue_time_us":2084276}
I20260709 13:17:44.521970  6518 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:44.532099  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.489246  9657 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:44.533650  6514 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:44.536419  6588 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:44.502689  9601 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:44.538028  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 985) took 2742 ms. Trace:
I20260709 13:17:44.543113  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:44.540118  7230 rpcz_store.cc:276] 0709 13:17:41.795911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.770281 (+974370us) service_pool.cc:224] Handling call
0709 13:17:44.537379 (+1767098us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:44.537573 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.537743 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.537765 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:44.537806 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:44.537833 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.537891 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.537985 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25299840}
I20260709 13:17:44.547709  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 942) took 1427 ms. Trace:
I20260709 13:17:44.381669  9626 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.555325  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.505518  9673 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:44.557405  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.563325  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.553133  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:44.530989  9596 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:44.560699  6589 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:44.551362  7236 rpcz_store.cc:276] 0709 13:17:43.119933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.159756 (+ 39823us) service_pool.cc:224] Handling call
0709 13:17:43.160188 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.526162 (+365974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.526196 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.532056 (+  5860us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.722708 (+190652us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:43.722748 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.722860 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.547500 (+824640us) raft_consensus.cc:1685] finished
0709 13:17:44.547520 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.547643 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33884,"raft.queue_time_us":135448,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":15170,"thread_start_us":479197,"threads_started":2,"wal-append.queue_time_us":258797}
I20260709 13:17:44.568787  6521 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:44.570598  6588 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:17:44.572166  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 263409us
I20260709 13:17:44.572937  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.575820  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.576083  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:44.576773  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 53223.6us
W20260709 13:17:44.578013  6487 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.887s (SENT)
W20260709 13:17:44.578817  6562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.009s	sys 0.000s
I20260709 13:17:44.578636  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:44.580533  6562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.009s	sys 0.000s
I20260709 13:17:44.579432  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:44.581019  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 705334us
I20260709 13:17:44.581230  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.581353  6513 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:44.582290  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:44.583269  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:44.584725  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.556602  9515 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.555441  9393 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.590111  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:44.595790  6487 leader_election.cc:336] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.928s (SENT)
I20260709 13:17:44.560204  9542 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.596227  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.572649  9653 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.599858  7230 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:44.603124  9627 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.579972  9599 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:44.580391  9663 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.621608  9612 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:44.622787  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.623143  6487 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.248s (SENT)
I20260709 13:17:44.623813  9677 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:44.624356  9677 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.552332  9624 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.589943  9579 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.603224  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:44.584959  9629 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:44.599124  9669 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.607812  6664 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:17:44.587281  9411 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.607332  9627 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:44.609530  9670 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:44.610169  9585 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.843s	user 0.005s	sys 0.000s
I20260709 13:17:44.590210  9530 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:44.610387  9583 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.841s	user 0.003s	sys 0.009s
I20260709 13:17:44.613731  9542 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:44.659521  9542 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:44.659920  9542 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:44.645010  9657 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.575081  9655 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.628105  9612 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:44.614899  9647 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.443s	user 0.000s	sys 0.001s
I20260709 13:17:44.615164  9606 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:44.615180  9644 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:17:44.617702  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.619230  9658 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:44.672281  6696 heartbeater.cc:646] Failed to heartbeat to 127.5.72.190:46511 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.72.190:46511 timed out after 15.000s (SENT)
I20260709 13:17:44.683488  6513 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:44.697942  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.607899  9653 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.698534  6513 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:44.699338  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.700555  9653 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.588024  9006 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:44.572767  9619 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.701661  6668 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 6. Ops: []
W20260709 13:17:44.632347  9634 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.770s	user 0.009s	sys 0.000s
I20260709 13:17:44.632110  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.709450  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.633359  9668 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.546861  9538 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.556054  9210 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.717582  6519 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:44.664799  9657 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:44.489581  9628 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.644380  6664 pending_rounds.cc:85] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
I20260709 13:17:44.720530  6664 pending_rounds.cc:107] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:17:44.720990  6664 raft_consensus.cc:2889] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:44.645373  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.656456  9677 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:44.657431  9678 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:17:44.614629  9665 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:17:44.523150  9641 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:44.664420  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.730187  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 572) took 1570 ms. Trace:
W20260709 13:17:44.730386  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 91225.2us
I20260709 13:17:44.730702  6664 rpcz_store.cc:276] 0709 13:17:43.157512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.607137 (+1449625us) service_pool.cc:224] Handling call
0709 13:17:44.607467 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.727423 (+119956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.727626 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.677847  6591 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:44.674746  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.731204  6637 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.634317  9646 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.735044  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1795) took 1296 ms. Trace:
I20260709 13:17:44.735095  9646 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.735560  6671 rpcz_store.cc:276] 0709 13:17:43.438707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.574665 (+135958us) service_pool.cc:224] Handling call
0709 13:17:44.734854 (+1160189us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54000cbf54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:44.734987 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244855680}
I20260709 13:17:44.736860  6671 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 6. Ops: []
I20260709 13:17:44.737399  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1770) took 1564 ms. Trace:
I20260709 13:17:44.713290  9634 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.737679  6671 rpcz_store.cc:276] 0709 13:17:43.173172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.736422 (+1563250us) service_pool.cc:224] Handling call
0709 13:17:44.736695 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.737168 (+   473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.737335 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.663012  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1899) took 3327 ms. Trace:
I20260709 13:17:44.739089  6664 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:44.739423  9634 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.739281  6522 rpcz_store.cc:276] 0709 13:17:41.335938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.506230 (+170292us) service_pool.cc:224] Handling call
0709 13:17:41.518560 (+ 12330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:41.518911 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.518941 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:41.527826 (+  8885us) log.cc:844] Serialized 26 byte log entry
0709 13:17:41.937429 (+409603us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:41.937486 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.937582 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.405317 (+2467735us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:44.662690 (+257373us) raft_consensus.cc:1685] finished
0709 13:17:44.662713 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.662929 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":540,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":6487,"spinlock_wait_cycles":103104384,"thread_start_us":314435,"threads_started":1,"wal-append.queue_time_us":1143279}
I20260709 13:17:44.741783  9618 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:44.688241  9643 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.767s	user 0.006s	sys 0.003s
I20260709 13:17:44.652653  9627 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.644870  9676 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:44.683207  9612 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.702293  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1767) took 1611 ms. Trace:
W20260709 13:17:44.666054  9671 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:17:44.710260  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1789) took 1408 ms. Trace:
W20260709 13:17:44.662544  9617 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.741s	user 0.006s	sys 0.001s
I20260709 13:17:44.758148  6670 rpcz_store.cc:276] 0709 13:17:43.301919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.426424 (+124505us) service_pool.cc:224] Handling call
0709 13:17:44.709298 (+1282874us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:44.710186 (+   888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21585792}
I20260709 13:17:44.714509  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:44.603300  9664 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.722064  9210 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:44.722918  9538 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:44.731992  6663 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:44.732679  6637 leader_election.cc:400] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 4 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5.
I20260709 13:17:44.767130  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.733702  9677 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:44.742935  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 990) took 1147 ms. Trace:
I20260709 13:17:44.774991  9579 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:44.746773  9646 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.775163  7232 rpcz_store.cc:276] 0709 13:17:43.595293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.615922 (+ 20629us) service_pool.cc:224] Handling call
0709 13:17:43.616229 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.628842 (+ 12613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.628880 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.701200 (+ 72320us) log.cc:844] Serialized 26 byte log entry
0709 13:17:44.404554 (+703354us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:44.404593 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.404657 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.742677 (+338020us) raft_consensus.cc:1685] finished
0709 13:17:44.742704 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.742876 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45064,"raft.queue_time_us":19895,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8140,"spinlock_wait_cycles":84224,"thread_start_us":638292,"threads_started":1,"wal-append.queue_time_us":657038}
I20260709 13:17:44.739702  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1769) took 1573 ms. Trace:
I20260709 13:17:44.777192  6664 rpcz_store.cc:276] 0709 13:17:43.166773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.731457 (+1564684us) service_pool.cc:224] Handling call
0709 13:17:44.738902 (+  7445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.739462 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.739629 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.743615  9006 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:44.750244  6668 rpcz_store.cc:276] 0709 13:17:43.090563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.701204 (+1610641us) service_pool.cc:224] Handling call
0709 13:17:44.701499 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.702042 (+   543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.702245 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.749887  9627 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:44.755772  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 948) took 1634 ms. Trace:
I20260709 13:17:44.780642  7228 rpcz_store.cc:276] 0709 13:17:43.121792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.231437 (+109645us) service_pool.cc:224] Handling call
0709 13:17:43.276514 (+ 45077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.295471 (+ 18957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.295508 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.409941 (+114433us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.606156 (+196215us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:43.606216 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.606297 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.687624 (+1081327us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:44.755465 (+ 67841us) raft_consensus.cc:1685] finished
0709 13:17:44.755488 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.755712 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133793,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":37846,"spinlock_wait_cycles":19824512,"thread_start_us":158846,"threads_started":1,"wal-append.queue_time_us":404758}
I20260709 13:17:44.757350  9612 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:44.760128  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.760157  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45735.4us
I20260709 13:17:44.768239  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1779) took 1469 ms. Trace:
I20260709 13:17:44.768623  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.786355  6663 rpcz_store.cc:276] 0709 13:17:43.299269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.427996 (+128727us) service_pool.cc:224] Handling call
0709 13:17:44.674563 (+1246567us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:17:44.768161 (+ 93598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197631232}
I20260709 13:17:44.788192  6519 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:44.789557  9646 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.790138  9646 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.766577  9676 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.774763  6670 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:17:44.793493  6670 pending_rounds.cc:85] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 5
I20260709 13:17:44.793790  6670 pending_rounds.cc:107] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:17:44.793558  9676 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.783995  6671 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:44.766026  9634 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:44.794669  9634 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.795166  9634 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:44.796125  9676 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.796895  9646 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.794065  6670 raft_consensus.cc:2889] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:44.783846  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 568) took 1661 ms. Trace:
I20260709 13:17:44.798765  6666 rpcz_store.cc:276] 0709 13:17:43.122396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.499603 (+1377207us) service_pool.cc:224] Handling call
0709 13:17:44.499946 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.501470 (+  1524us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:44.545856 (+ 44386us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.783512 (+237656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:44.783561 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:44.783603 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.783661 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.783793 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78493,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":4769,"raft.queue_time_us":1952,"thread_start_us":258450,"threads_started":2}
I20260709 13:17:44.805764  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:44.811017  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 228773us
I20260709 13:17:44.813728  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.814594  6522 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:44.818647  9376 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.823218  9376 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:44.823473  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:44.825407  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:44.826835  9006 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:44.827369  9006 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:44.827831  9006 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:44.831022  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1777) took 1535 ms. Trace:
I20260709 13:17:44.831197  9579 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:44.831461  6670 rpcz_store.cc:276] 0709 13:17:43.295182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.758927 (+1463745us) service_pool.cc:224] Handling call
0709 13:17:44.768256 (+  9329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.830739 (+ 62483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.830954 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.831920  9579 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:44.833112  9579 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:44.836771  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.797994  9677 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:44.841552  9607 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.303s	user 0.023s	sys 0.024s
I20260709 13:17:44.826061  9542 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.765803  9617 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.843804  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:44.857493  7228 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:44.858878  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 238073us
I20260709 13:17:44.842547  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:44.863442  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.853734  9607 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.315s	user 0.024s	sys 0.024s
I20260709 13:17:44.867604  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:44.868214  7235 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:44.843001  9658 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:44.866946  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.001s	sys 0.013s
I20260709 13:17:44.843091  9305 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.843449  9329 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.869599  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:44.872723  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.002s	sys 0.014s
I20260709 13:17:44.836073  9596 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:44.879503  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53104.1us
I20260709 13:17:44.880685  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:44.881181  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:44.877303  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 599772us
W20260709 13:17:44.922863  6667 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1757) took 1835 ms (client timeout 1961 ms). Trace:
I20260709 13:17:44.924168  7229 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:44.925451  6561 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.766s (SENT)
W20260709 13:17:44.925205  6667 rpcz_store.cc:269] 0709 13:17:43.087906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.209300 (+121394us) service_pool.cc:224] Handling call
0709 13:17:44.922686 (+1713386us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b540011e9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:44.922805 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28068736}
W20260709 13:17:44.882026  6636 connection.cc:256] RPC call timeout handler was delayed by -1.200930040999765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:17:44.927309  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46690.5us
W20260709 13:17:44.911231  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 358134us
I20260709 13:17:44.928241  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:44.929395  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.929760  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.925994  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:44.938732  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.939591  6666 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:44.939780  6667 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:44.940798  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.941540  6514 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:44.902722  9596 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.949285  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.889042  9264 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:44.950513  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:44.883530  9611 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:44.904706  9618 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.893815  9624 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.957094  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:44.957383  6668 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:44.970239  6514 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:44.970492  7228 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:44.957667  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.964344  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.905943  9617 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:44.958707  6670 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:44.957397  7229 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:44.873441  9658 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:44.966871  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.922786  9376 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:44.982215  6667 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:44.985572  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.969151  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:44.986161  6666 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:44.986784  6667 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:44.971769  6522 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:44.924326  9655 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:44.978650  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.958091  9677 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:44.979530  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.979713  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.940006  9622 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:44.977213  6514 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:44.979022  6664 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:44.879030  9305 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:44.968006  9620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.002s	sys 0.012s
I20260709 13:17:44.991957  7235 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:17:44.982216  7200 connection.cc:256] RPC call timeout handler was delayed by -1.790769224000087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 13:17:44.983310  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:44.879730  9329 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.990391  9638 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:44.990877  6668 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:44.995646  9316 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:44.995366  9376 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.984583  9624 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.997165  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:44.999195  9006 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.002074  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 992) took 1156 ms. Trace:
I20260709 13:17:45.005025  9609 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:45.037730  9655 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:45.006174  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:45.008411  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 570) took 1885 ms. Trace:
I20260709 13:17:45.065416  6665 rpcz_store.cc:276] 0709 13:17:43.123086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.007567 (+1884481us) service_pool.cc:224] Handling call
0709 13:17:45.007916 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.008096 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.008125 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.008168 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:45.008202 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.008253 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.008347 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.010860  9519 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:45.075062  9305 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:45.012843  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 710) took 1452 ms. Trace:
I20260709 13:17:45.016060  6666 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:45.016542  9291 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:45.079036  6595 rpcz_store.cc:276] 0709 13:17:43.560657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.562740 (+  2083us) service_pool.cc:224] Handling call
0709 13:17:43.563158 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.593771 (+ 30613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.593805 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.594237 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.735983 (+141746us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:43.784918 (+ 48935us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.012377 (+1227459us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b18003b63e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287
0709 13:17:45.012487 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.012558 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.012602 (+    44us) raft_consensus.cc:1685] finished
0709 13:17:45.012679 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.012789 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26062,"prepare.queue_time_us":51211,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":210349,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":194,"spinlock_wait_cycles":16971136,"thread_start_us":207352,"threads_started":2,"wal-append.queue_time_us":319620}
I20260709 13:17:45.018705  9589 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:44.926282  9546 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:17:44.959070  9611 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:44.926581  9604 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:17:44.982741  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.085039  9611 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:44.954416  9615 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:17:45.040683  6667 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:45.409675  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:45.053987  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.410811  6592 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:44.980878  9688 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:45.055473  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.412153  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.412568  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:45.041887  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:45.058310  7233 rpcz_store.cc:276] 0709 13:17:43.845956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.846072 (+   116us) service_pool.cc:224] Handling call
0709 13:17:45.001884 (+1155812us) spinlock_profiling.cc:243] Waited 74.6 ms on lock 0x7b54007a0154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:45.002007 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171614080}
I20260709 13:17:45.054582  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:45.412755  6521 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:45.413335  6664 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
W20260709 13:17:45.413576  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 359048us
I20260709 13:17:45.414021  6667 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:17:45.059958  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 66649.6us
W20260709 13:17:45.037890  9620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.002s	sys 0.013s
I20260709 13:17:45.005812  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 568) took 3709 ms. Trace:
I20260709 13:17:45.417202  6520 rpcz_store.cc:276] 0709 13:17:41.296039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.372337 (+ 76298us) service_pool.cc:224] Handling call
0709 13:17:41.372642 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:43.653566 (+2280924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.005594 (+1352028us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b540019afd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:45.005756 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5230955776}
I20260709 13:17:45.419464  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:45.419576  9607 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:45.420231  6520 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:45.027040  9677 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.067459  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:45.438396  6513 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:45.041745  9505 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:17:45.072813  9623 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:17:45.083367  9546 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:45.444902  9546 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:45.445807  9546 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.446769  6516 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:45.449250  9677 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:45.093914  9589 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:45.134799  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.096076  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:45.101567  7229 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:45.054589  9316 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.463322  9623 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:45.508101  9316 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:45.508550  9623 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.140738  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:45.480557  6670 raft_consensus.cc:2376] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6. Re-sending same reply.
I20260709 13:17:45.480013  6522 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:45.512981  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.514104  6522 raft_consensus.cc:2410] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:45.146566  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2285) took 1050 ms. Trace:
I20260709 13:17:44.974550  9668 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:17:45.516156  9623 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:45.516464  9668 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:45.516963  9668 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:45.154902  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:45.518134  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.519567  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:45.156103  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:45.520133  6522 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:45.520813  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 364743us
I20260709 13:17:45.463340  9676 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.156718  9609 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:45.159273  9613 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
W20260709 13:17:45.172333  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.094110  9596 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:17:45.317406  9655 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.320587  9329 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:45.325640  9627 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.523s	user 0.006s	sys 0.001s
I20260709 13:17:45.058298  9638 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:45.060985  9624 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.331089  9658 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:45.377223  9210 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.611s	user 0.006s	sys 0.002s
I20260709 13:17:45.377864  9006 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:45.306384  9634 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:45.092187  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:45.025218  9635 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:45.411288  9517 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.420251  9607 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.460171  6561 leader_election.cc:400] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:45.567183  6561 leader_election.cc:403] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:45.459364  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:45.568135  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 108831us
W20260709 13:17:45.462450  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.571171  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.462510  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47766.2us
I20260709 13:17:45.128578  6593 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:45.476464  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:45.572461  7233 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:45.479362  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1771) took 2271 ms. Trace:
I20260709 13:17:45.573448  6671 rpcz_store.cc:276] 0709 13:17:43.220057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.738393 (+1518336us) service_pool.cc:224] Handling call
0709 13:17:44.783809 (+ 45416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.798044 (+ 14235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.479153 (+681109us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b54000e60d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:45.479302 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103755136}
I20260709 13:17:45.470476  6520 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:17:45.527498  6513 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:17:45.543494  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.576581  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.522392  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:45.544421  6591 rpcz_store.cc:276] 0709 13:17:44.095930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.559745 (+463815us) service_pool.cc:224] Handling call
0709 13:17:44.561281 (+  1536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.731552 (+170271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.146265 (+414713us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54003bc4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:45.146497 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248403328}
I20260709 13:17:45.520191  6588 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:45.578821  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.546729  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1903) took 1984 ms. Trace:
I20260709 13:17:45.458566  9546 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:45.556361  6669 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:45.517899  9668 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.562732  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:45.585223  6595 raft_consensus.cc:2376] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 6. Re-sending same reply.
I20260709 13:17:45.563632  7231 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
W20260709 13:17:45.555285  9622 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.004s
I20260709 13:17:45.579358  6515 rpcz_store.cc:276] 0709 13:17:43.562247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.574197 (+ 11950us) service_pool.cc:224] Handling call
0709 13:17:43.607235 (+ 33038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:43.885699 (+278464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.885742 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:43.885796 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:43.885837 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.546396 (+1660559us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400044054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:45.546461 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.546655 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639861120}
W20260709 13:17:45.581707  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 121648us
I20260709 13:17:45.550024  9589 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:45.584414  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1760) took 2495 ms. Trace:
I20260709 13:17:45.572283  9517 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.591627  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:45.578198  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:45.556845  9627 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:45.594672  6669 rpcz_store.cc:276] 0709 13:17:43.088740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.555451 (+2466711us) service_pool.cc:224] Handling call
0709 13:17:45.556041 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.584062 (+ 28021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.584342 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.590423  9329 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6.
I20260709 13:17:45.599514  6591 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:45.603716  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:45.568787  9607 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:45.611799  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:45.612593  6593 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:45.577378  9658 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:45.619808  9652 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:45.522797  9676 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.610286  6588 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:45.602535  9530 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.625912  6593 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:45.617300  9658 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:45.616165  9609 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:45.605537  9690 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:45.627328  9658 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:45.628983  9530 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:45.671500  6637 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.021s	sys 0.123s
W20260709 13:17:45.672762  6637 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.021s	sys 0.123s
W20260709 13:17:45.673127  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 744879us
I20260709 13:17:45.673920  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:45.644029  9316 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:45.719473  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:45.724108  6520 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:45.730581  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.731777  6515 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:45.739989  6517 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:45.706920  9638 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:45.721244  9613 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:45.744385  9702 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:17:45.727401  9624 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:45.723407  9006 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:17:45.732755  8949 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:17:45.768182  6520 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:45.752818  9613 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.753443  9579 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:45.774881  9706 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:17:45.776733  9676 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:45.756506  9634 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:45.753508  9532 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.025s	sys 0.021s
W20260709 13:17:45.782127  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 154665us
W20260709 13:17:45.815899  7198 leader_election.cc:336] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.849s (SENT)
I20260709 13:17:45.816469  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:45.787581  9589 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:45.788352  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.818168  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.799317  9688 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:45.799885  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 277543us
W20260709 13:17:45.800482  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.804656  9613 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:45.781271  9627 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:45.799358  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 195712us
I20260709 13:17:45.829636  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:45.793924  9677 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.823830  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:45.817631  9532 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.026s	sys 0.021s
I20260709 13:17:45.827281  6589 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:17:45.831494  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.832850  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:45.835501  6593 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:45.836889  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:45.838738  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:45.840893  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:45.821425  9676 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:45.853211  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:45.813812  9579 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:45.848078  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:45.858253  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.858955  6520 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:45.859895  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.861153  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.862076  6520 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:45.864732  6593 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:45.865809  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:45.864763  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:45.867257  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:45.869231  6667 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:45.844031  9627 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:45.878152  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:45.880926  6514 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:45.883019  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:45.883188  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:45.883755  7233 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:45.885296  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:45.886337  7234 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:45.866281  9638 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:45.855811  9375 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:45.891352  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:45.901886  6665 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:45.845115  9316 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:45.893389  6670 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:45.906867  6514 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:45.912397  6665 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:45.913700  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:45.914141  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:45.914930  6670 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:45.915114  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:45.915628  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:45.916189  6670 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:45.873667  9609 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:45.866495  9715 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:45.918264  6663 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:45.890703  9329 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:45.938937  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53151.9us
I20260709 13:17:45.943703  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:45.944319  6665 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:45.908964  9652 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:45.944342  6664 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:45.949200  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:45.958834  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.943111  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:45.934403  9375 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:45.976378  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:45.978243  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:45.978991  6664 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:45.946592  9329 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.946038  9677 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:45.997906  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 114779us
I20260709 13:17:45.950942  8949 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:17:46.044237  9579 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:45.977483  9530 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:46.002955  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:46.052021  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 108973us
W20260709 13:17:46.053337  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.054175  9725 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:46.055090  9725 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:46.056195  9725 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:46.057835  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:46.059029  9713 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 13:17:46.060720  9719 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:17:45.984591  9329 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:45.963748  9006 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:17:46.020048  6487 leader_election.cc:336] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.654s (SENT)
W20260709 13:17:46.067849  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47856.1us
W20260709 13:17:46.018922  9685 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 13:17:46.026764  9712 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.191s	user 0.001s	sys 0.001s
I20260709 13:17:46.028122  6665 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:17:45.999576  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:46.072120  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 72666.3us
I20260709 13:17:46.072861  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:46.074796  9605 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:46.075402  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.075472  9609 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:46.076363  9609 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:46.078083  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:46.079821  9524 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:46.080626  9622 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:46.081005  9622 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:46.081342  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2278) took 2196 ms (client timeout 1941 ms). Trace:
W20260709 13:17:46.081719  6596 rpcz_store.cc:269] 0709 13:17:43.884926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.266546 (+381620us) service_pool.cc:224] Handling call
0709 13:17:46.081187 (+1814641us) spinlock_profiling.cc:243] Waited 91.3 ms on lock 0x7b540019d2d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:46.081275 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210042240}
I20260709 13:17:46.081773  9622 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.083396  9622 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:46.083431  6516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2297) took 1760 ms (client timeout 1757 ms). Trace:
W20260709 13:17:46.084192  6592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 693) took 4835 ms (client timeout 2127 ms). Trace:
I20260709 13:17:46.084707  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 996) took 1495 ms. Trace:
I20260709 13:17:46.085054  7231 rpcz_store.cc:276] 0709 13:17:44.589028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.589845 (+   817us) service_pool.cc:224] Handling call
0709 13:17:46.084567 (+1494722us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b540051aed4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:46.084657 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134447488}
I20260709 13:17:46.085881  9316 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:46.084542  6592 rpcz_store.cc:269] 0709 13:17:41.248967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.251600 (+  2633us) service_pool.cc:224] Handling call
0709 13:17:46.084015 (+4832415us) spinlock_profiling.cc:243] Waited 4.05 s on lock 0x7b54003a1954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:46.084107 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9324173056}
I20260709 13:17:46.080102  6597 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:17:46.083983  6516 rpcz_store.cc:269] 0709 13:17:44.322665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.322772 (+   107us) service_pool.cc:224] Handling call
0709 13:17:46.083281 (+1760509us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003ccb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:46.083369 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441870592}
I20260709 13:17:46.097199  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:46.100744  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.102370  6670 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:46.105199  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.106392  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:46.107125  6596 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:46.086390  9316 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:46.114449  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.114928  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:46.119578  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 177300us
I20260709 13:17:46.127283  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.132311  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:46.132969  6593 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:46.101554  9677 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.104382  9614 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:46.105386  9690 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.105073  9646 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:46.102969  9627 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:46.141436  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:46.147747  6589 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:46.143121  6593 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:46.155874  6596 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:46.156104  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.164345  6593 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:46.110975  9524 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:17:46.129962  8949 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:46.126832  9676 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.159855  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.168027  6597 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:46.132980  9542 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:46.166417  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.170154  6596 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:46.141206  9614 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:46.156912  6589 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:17:46.171833  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 67657.2us
I20260709 13:17:46.075587  9605 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:46.146471  9677 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.181806  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:46.174255  9658 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.530s	user 0.014s	sys 0.000s
I20260709 13:17:46.160288  9690 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:46.182966  6596 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:46.172910  9614 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:46.161331  9704 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:17:46.218266  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:46.190966  9607 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.567s	user 0.014s	sys 0.004s
I20260709 13:17:46.219368  6596 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:46.226965  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.172013  9676 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.235028  9676 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:46.244551  9677 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.251729  9316 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:46.252303  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.230731  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:46.252897  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79822.8us
I20260709 13:17:46.254228  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:46.261883  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.263345  6593 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:46.192466  9658 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.223276  9646 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:46.268525  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:46.227761  9607 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:46.271517  6519 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:46.272625  9607 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:46.273619  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:46.274211  6671 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:46.274904  9746 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:46.276309  9746 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:46.276746  9746 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:46.277516  9746 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:46.278695  9622 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:46.279539  9622 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.279942  9622 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.281122  6671 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:46.284313  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:46.191942  9627 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.303128  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:46.303284  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:46.306093  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.306253  6522 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:46.312680  6513 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:17:46.309979  9546 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.718s	user 0.013s	sys 0.000s
I20260709 13:17:46.278501  9613 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.324596  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:46.326367  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:46.328449  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:46.315757  9329 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.347446  6589 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:46.347652  6515 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:46.349064  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:46.323434  9646 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:46.355599  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.358914  6589 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:46.360129  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:46.360898  7237 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:46.356101  9660 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:46.365957  9660 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:46.366241  9747 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:17:46.391157  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.399531  6666 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:46.366818  9660 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.416934  9628 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:46.410315  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:46.355010  9658 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.371515  6596 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:46.375950  9329 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.354277  9690 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:46.379763  6517 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2299) took 2020 ms (client timeout 1941 ms). Trace:
W20260709 13:17:46.438871  6517 rpcz_store.cc:269] 0709 13:17:44.359749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.389365 (+ 29616us) service_pool.cc:224] Handling call
0709 13:17:46.379533 (+1990168us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002157d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:46.379674 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670231808}
I20260709 13:17:46.381321  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:46.439723  6589 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:46.440150  6515 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:46.381765  9613 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.382144  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:46.385114  7232 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:46.384706  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.386847  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.388772  9631 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:46.388396  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:46.442871  7229 raft_consensus.cc:2410] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:46.442936  7230 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:46.411350  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:46.445104  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.446441  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:46.399895  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2275) took 2525 ms (client timeout 1757 ms). Trace:
I20260709 13:17:46.415037  6522 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:46.419368  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:46.421552  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:46.437935  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 343039us
I20260709 13:17:46.453439  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.452809  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.465266  7237 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:46.450258  6590 rpcz_store.cc:269] 0709 13:17:43.873735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.890763 (+ 17028us) service_pool.cc:224] Handling call
0709 13:17:46.399203 (+2508440us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54002f67d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:46.399475 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896691584}
I20260709 13:17:46.468082  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.469010  7232 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:46.469282  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:46.440522  9750 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:17:46.473210  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.453630  7200 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:46.477607  7232 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:46.474836  7228 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:46.420214  9546 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:46.478662  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:46.479473  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.481006  7235 raft_consensus.cc:2368] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
I20260709 13:17:46.483640  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.484215  7237 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:46.484288  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:46.484860  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.485631  6665 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:46.485855  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.486460  6667 raft_consensus.cc:2368] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4.
I20260709 13:17:46.487329  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.487828  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:46.491446  7233 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:46.491845  6515 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:17:46.492372  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.492859  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.493858  6666 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:46.419560  9614 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:46.640379  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.495267  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:46.548293  7230 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:46.512789  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2036) took 1359 ms. Trace:
I20260709 13:17:46.687415  6591 rpcz_store.cc:276] 0709 13:17:45.152937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.604441 (+451504us) service_pool.cc:224] Handling call
0709 13:17:45.604739 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.604916 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:46.375409 (+770493us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800429a60. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf
0709 13:17:46.375593 (+   184us) log.cc:844] Serialized 14 byte log entry
0709 13:17:46.512317 (+136724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:46.512386 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:46.512442 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.512534 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.512711 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35644,"prepare.queue_time_us":192245,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":33693,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12294,"spinlock_wait_cycles":2751360,"thread_start_us":287561,"threads_started":2}
I20260709 13:17:46.646036  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.690121  6670 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:46.647540  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.690963  6671 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:46.573616  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.515946  6590 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:46.671854  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.462103  9542 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:46.495899  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.585637  6520 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:46.673684  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:46.464627  9631 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:46.479465  9704 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:46.750949  6517 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:46.751456  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 263701us
I20260709 13:17:46.752447  6520 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:17:46.581121  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:46.585412  9754 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:17:46.755575  6520 pending_rounds.cc:85] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 6
I20260709 13:17:46.755991  6520 pending_rounds.cc:107] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:17:46.756342  6520 raft_consensus.cc:2889] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:46.747684  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.750274  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:46.750480  9631 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.757802  6666 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:46.758108  9631 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:46.683583  6589 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:46.759371  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.760890  6589 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:46.762092  6486 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:46.600538  6487 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.972s (SENT)
W20260709 13:17:46.767196  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 166708us
W20260709 13:17:46.767982  6487 leader_election.cc:336] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 3.112s (SENT)
I20260709 13:17:46.768482  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:46.586269  6513 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:17:46.752467  9316 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:46.580730  9634 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.689059  6667 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:46.779251  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:46.568778  9524 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.628545  9656 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.780128  6590 raft_consensus.cc:2410] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:17:46.549137  9546 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:46.548278  9677 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:46.628011  9607 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.752481  9704 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:46.805395  9704 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:46.483453  9530 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:46.780964  6662 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:46.806149  9530 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:46.549510  9714 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:46.548481  9622 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.807503  9622 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:46.782896  9634 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.784734  9524 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.766376  9329 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:46.811069  9329 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.813102  9329 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:46.814316  9720 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.837s	user 0.000s	sys 0.003s
I20260709 13:17:46.791596  9656 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.792024  9607 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.783525  9746 raft_consensus.cc:2764] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:17:46.794420  9546 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:46.796324  9756 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:17:46.798779  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.786162  9613 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:17:46.791700  9517 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:46.820858  9755 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:17:46.825632  9524 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.828667  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2039) took 1674 ms. Trace:
I20260709 13:17:46.889780  6588 rpcz_store.cc:276] 0709 13:17:45.153801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.622233 (+468432us) service_pool.cc:224] Handling call
0709 13:17:45.636158 (+ 13925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.645112 (+  8954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.645146 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.645185 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:45.645212 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.828404 (+1183192us) spinlock_profiling.cc:243] Waited 8.75 ms on lock 0x7b540019dcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:46.828471 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.828600 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20133120}
W20260709 13:17:46.831651  9723 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.818s	user 0.002s	sys 0.000s
I20260709 13:17:46.832237  6664 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:46.891680  6664 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:46.833222  9614 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:46.833161  6667 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:17:46.845288  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 290841us
I20260709 13:17:46.862422  6595 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:46.896306  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:46.864374  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:46.897150  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.841724  9513 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:46.897337  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:46.901216  6589 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:46.900163  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:46.906987  6664 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.908466  6664 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:46.909510  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 162134us
I20260709 13:17:46.910776  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:46.923434  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 170888us
I20260709 13:17:46.924422  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.928274  6589 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:46.929725  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:46.930639  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:46.931354  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 169345us
I20260709 13:17:46.933652  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:46.935031  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:46.936007  6662 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:46.939540  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:46.913074  9634 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.946794  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:46.954519  7231 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:46.947258  9316 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:46.947721  9677 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:46.965099  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 197136us
I20260709 13:17:46.967938  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:46.968274  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:46.968572  7231 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:46.944159  9291 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:17:46.971711  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:46.945839  9735 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:17:46.977249  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:46.980338  6664 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:46.980846  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:46.981454  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 589) took 1068 ms. Trace:
I20260709 13:17:46.982122  6664 rpcz_store.cc:276] 0709 13:17:45.913347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.979751 (+ 66404us) service_pool.cc:224] Handling call
0709 13:17:45.990540 (+ 10789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.832140 (+841600us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54000698a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:17:46.981018 (+148878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.981365 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403513088}
I20260709 13:17:46.917950  9530 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:46.966233  9638 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.000s	sys 0.020s
I20260709 13:17:46.997051  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:46.943662  9546 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:46.966691  9316 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:47.000047  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:47.002169  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:47.003664  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:47.013046  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62797.7us
I20260709 13:17:47.014065  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:47.014588  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:46.986528  9596 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:47.017930  6664 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:46.983520  9676 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:17:46.979399  9542 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:17:47.053788  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73020.9us
I20260709 13:17:47.054932  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:46.998698  9767 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:47.054136  6513 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:47.017426  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:47.059029  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:47.008553  9746 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:47.000638  9638 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.000s	sys 0.020s
I20260709 13:17:47.016942  9396 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:46.985612  9766 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 13:17:47.066133  9628 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.000s
I20260709 13:17:47.002839  9614 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:17:47.013466  9623 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:47.016177  9316 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:47.055714  9607 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:46.998764  9677 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:17:47.004783  9770 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:47.010443  9745 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:17:47.077605  9396 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:47.068699  9758 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:17:47.063670  9656 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:47.079527  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 65010.8us
I20260709 13:17:47.135417  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:47.058262  9767 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.048580  9622 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:17:47.109831  6520 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:47.147321  6518 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:47.147809  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:47.067481  9638 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:47.122429  9677 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:47.119059  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.089468  9291 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:47.140494  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 85644.3us
I20260709 13:17:47.089468  9742 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:17:47.136246  9676 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:47.165275  7228 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:17:47.107594  9735 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:17:47.151408  9779 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:47.170209  7228 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:47.147871  9505 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.144021  9767 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:47.056617  9596 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:47.171123  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.160686  7237 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:47.162050  9291 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:47.173097  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.168226  9782 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:47.173051  9596 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:47.191089  9781 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:17:47.193082  9773 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:17:47.195001  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2133) took 1330 ms. Trace:
I20260709 13:17:47.194799  9638 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:47.196345  9762 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 13:17:47.195362  6669 rpcz_store.cc:276] 0709 13:17:45.864347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.874813 (+ 10466us) service_pool.cc:224] Handling call
0709 13:17:45.875245 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:45.875746 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.875802 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:46.809644 (+933842us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b2c0004d658. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:17:46.810183 (+   539us) log.cc:844] Serialized 52 byte log entry
0709 13:17:46.938499 (+128316us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:46.938534 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.938629 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:47.194683 (+256054us) raft_consensus.cc:1685] finished
0709 13:17:47.194707 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:47.194944 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8536,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":20680,"spinlock_wait_cycles":17888512,"thread_start_us":127955,"threads_started":1,"wal-append.queue_time_us":190324}
I20260709 13:17:47.198230  9376 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:47.172782  9656 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:47.169935  9780 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:47.224414  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:47.223429  6487 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.127s (SENT)
I20260709 13:17:47.225732  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:47.173624  9505 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:47.228775  7236 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:47.191098  9771 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 13:17:47.222551  9579 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:17:47.223023  9688 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:47.176357  9676 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:47.234445  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:47.235472  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 87724.9us
I20260709 13:17:47.182327  9782 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:47.240880  7236 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:47.243278  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:47.227376  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:47.230082  9622 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:47.248170  9676 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.265805  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:47.229137  9656 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:47.240669  9787 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:47.224874  9396 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:47.273397  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:47.230199  9786 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:17:47.234452  9780 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:47.276122  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:47.277679  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:47.248700  9784 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:47.279173  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:47.279531  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:47.228237  9291 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.281281  6596 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:47.299189  7230 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:17:47.253575  9783 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.280584  6589 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:47.269697  9542 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:47.290861  6487 leader_election.cc:400] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
I20260709 13:17:47.280616  9785 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:47.288208  9623 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:47.243732  9688 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.329977  9622 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:47.353076  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 109835us
I20260709 13:17:47.360054  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:47.341723  6487 leader_election.cc:403] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:47.354746  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75296us
I20260709 13:17:47.278885  9786 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:17:47.360800  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 70099.2us
I20260709 13:17:47.361061  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:47.364012  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:47.340474  9530 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.306728  9779 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:47.352278  9619 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.340545  9790 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:47.355933  9785 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:47.289160  9316 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.348969  9783 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:47.473474  6521 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2304) took 2923 ms (client timeout 1948 ms). Trace:
I20260709 13:17:47.357788  9542 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:47.516394  6521 rpcz_store.cc:269] 0709 13:17:44.550965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.575694 (+ 24729us) service_pool.cc:224] Handling call
0709 13:17:47.471011 (+2895317us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b540007dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:47.471124 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108862208}
I20260709 13:17:47.279316  9787 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.364305  9523 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
I20260709 13:17:47.553267  6592 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:47.363267  9622 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:47.373234  9775 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:47.370707  9579 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:47.409621  9517 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 13:17:47.517096  9785 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.562912  9793 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:47.367141  9656 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.569555  9006 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:47.566841  9611 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.570423  9787 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:47.570448  9752 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.985s	user 0.008s	sys 0.008s
I20260709 13:17:47.373382  9782 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.473912  9767 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.571720  9688 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:47.571823  9792 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:17:47.598702  9656 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:47.499577  9779 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.574721  9783 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:47.592491  9658 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:47.603413  9658 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:47.399765  9619 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:47.580550  6516 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:47.588765  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:47.572073  9668 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.364537  9700 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.203s
I20260709 13:17:47.410789  9795 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.188s
I20260709 13:17:47.583179  9775 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:47.572969  9622 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:47.616017  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 230321us
I20260709 13:17:47.592806  9517 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:47.636574  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.460896  9780 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.654213  9316 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:47.655045  9619 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:47.655761  9767 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:47.636184  9782 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.590950  9793 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:47.658116  9775 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.659515  9793 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:47.593259  9006 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.642339  9658 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:47.664326  9006 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:47.665194  9513 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.759s	user 0.013s	sys 0.000s
I20260709 13:17:47.666119  6594 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:47.668036  9596 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.670575  9783 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:47.666517  9717 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.671037  9783 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:47.671267  9717 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.671886  9783 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:47.673748  9677 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.511s	user 0.005s	sys 0.001s
I20260709 13:17:47.674187  9677 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:47.679944  9316 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:47.681321  9767 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:47.681691  9767 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:47.682266  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1928) took 1822 ms. Trace:
I20260709 13:17:47.682488  9767 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.682688  6519 rpcz_store.cc:276] 0709 13:17:45.859483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.861315 (+  1832us) service_pool.cc:224] Handling call
0709 13:17:45.885836 (+ 24521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:46.271859 (+386023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.682030 (+1410171us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b540014d7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:47.682214 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886449408}
I20260709 13:17:47.683859  9596 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:47.684868  9717 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:47.640229  9517 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.689245  9517 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:47.640179  9625 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:47.685103  9291 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.702790  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.703003  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.708820  7230 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:47.707054  7234 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:17:47.711079  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:47.729409  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 354722us
W20260709 13:17:47.737042  6518 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2303) took 3225 ms (client timeout 2034 ms). Trace:
W20260709 13:17:47.737591  6518 rpcz_store.cc:269] 0709 13:17:44.511583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.542769 (+ 31186us) service_pool.cc:224] Handling call
0709 13:17:47.736796 (+3194027us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b54001e9954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:47.736957 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5820379904}
W20260709 13:17:47.747326  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 161908us
I20260709 13:17:47.748020  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:47.749606  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 161310us
W20260709 13:17:47.752583  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 141072us
I20260709 13:17:47.756767  6561 leader_election.cc:400] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4.
I20260709 13:17:47.759179  9625 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.762562  9794 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:47.772603  9717 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.751780  9656 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:17:47.747628  9795 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:17:47.759788  9638 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:47.761006  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:47.761332  9517 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.761767  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:47.783686  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.787148  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.797401  7229 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:17:47.807367  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 167909us
I20260709 13:17:47.787745  7230 raft_consensus.cc:2410] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:47.788380  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:47.802968  6514 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:47.809049  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.783841  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:47.815403  7228 raft_consensus.cc:2410] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:47.817941  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.819206  7230 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:47.819082  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:47.744689  9596 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:17:47.820361  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:47.821182  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:47.823813  6671 pending_rounds.cc:85] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 4
I20260709 13:17:47.831984  6671 pending_rounds.cc:107] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:17:47.832810  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:47.823659  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:47.824326  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:47.834903  6665 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:47.822827  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:47.842068  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:47.842350  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:47.842960  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:47.843008  6589 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:47.842507  6671 raft_consensus.cc:2889] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:47.828626  6664 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:47.829283  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:47.846884  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:47.847131  6669 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:17:47.830027  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:47.848762  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:47.847164  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.851141  6591 raft_consensus.cc:2376] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8. Re-sending same reply.
I20260709 13:17:47.852207  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.818923  9531 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:47.854300  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.854861  9530 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:47.855077  6595 raft_consensus.cc:2376] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5. Re-sending same reply.
I20260709 13:17:47.855963  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.856714  9622 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.795565  9517 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:47.819410  9611 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.859534  9611 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.817751  9782 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:47.776269  9787 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:17:47.861402  6589 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:47.862126  6663 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:47.802811  9800 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:47.887060  6670 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:47.877151  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:47.863186  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.817922  9660 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.860376  9531 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:47.875730  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55227.2us
I20260709 13:17:47.890239  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.891355  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.892879  6664 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:17:47.893252  6664 pending_rounds.cc:85] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 7
I20260709 13:17:47.893478  6664 pending_rounds.cc:107] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:17:47.893703  6664 raft_consensus.cc:2889] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:47.875866  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:47.876421  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:47.816156  9634 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.817276  9579 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:47.899108  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:47.854357  9808 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:47.832731  9513 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:47.900130  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:47.901651  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:47.904467  6665 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:47.815801  9805 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:17:47.840256  9658 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:47.848453  9375 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:47.795475  9802 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:17:47.994012  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:47.877104  9662 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:47.849418  9668 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:47.983059  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:47.983316  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:47.985532  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 85462.2us
I20260709 13:17:47.890866  9800 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:48.020123  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.973408  9523 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.029239  6588 raft_consensus.cc:2376] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6. Re-sending same reply.
I20260709 13:17:47.979555  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:48.017107  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:48.023097  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:48.031172  7237 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
W20260709 13:17:48.033012  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 191278us
I20260709 13:17:47.988592  9779 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:47.972573  9634 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.989181  9622 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.040851  6665 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:48.044749  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.045462  6667 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:48.045920  6664 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:48.046541  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.047135  6665 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:48.020226  9782 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:47.972714  9531 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:47.916608  9660 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.023305  9629 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.983795  9732 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:17:47.991447  9804 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:17:48.048082  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 585) took 1565 ms. Trace:
W20260709 13:17:48.037015  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 119588us
I20260709 13:17:48.097082  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.118366  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1941) took 1104 ms. Trace:
I20260709 13:17:48.156543  6514 rpcz_store.cc:276] 0709 13:17:47.013657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.013837 (+   180us) service_pool.cc:224] Handling call
0709 13:17:48.118202 (+1104365us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54003f6bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:48.118309 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812691328}
I20260709 13:17:48.139052  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.049021  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.159366  6663 raft_consensus.cc:2410] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:17:48.159972  6592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 711) took 2059 ms (client timeout 1524 ms). Trace:
I20260709 13:17:48.029832  9800 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.057482  6515 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:48.063663  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:48.051997  9810 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:17:48.161784  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.192821  6664 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:48.113653  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:48.033465  9811 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.111690  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 970) took 1166 ms. Trace:
I20260709 13:17:48.161231  6671 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:48.039458  9631 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:48.039675  9812 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:48.139874  9662 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.044241  9638 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:48.146052  9411 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.169790  9517 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:48.170220  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.171438  6595 raft_consensus.cc:2376] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5. Re-sending same reply.
I20260709 13:17:48.205395  6663 raft_consensus.cc:2410] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:48.205523  6591 raft_consensus.cc:2376] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6. Re-sending same reply.
I20260709 13:17:48.174229  9629 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.212913  6514 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:48.175483  9782 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:48.187394  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.102572  9726 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:17:48.097885  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:48.213943  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:48.214569  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 116810us
W20260709 13:17:48.049783  9807 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:17:48.189451  6665 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
W20260709 13:17:48.160463  6592 rpcz_store.cc:269] 0709 13:17:46.100247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.100404 (+   157us) service_pool.cc:224] Handling call
0709 13:17:48.159790 (+2059386us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400303754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:48.159906 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3338547072}
I20260709 13:17:48.049341  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:48.127561  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:17:48.051185  9803 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:17:48.217293  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 89817.8us
I20260709 13:17:48.217911  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:48.194041  7229 rpcz_store.cc:276] 0709 13:17:46.945775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.946300 (+   525us) service_pool.cc:224] Handling call
0709 13:17:46.946717 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:47.807516 (+860799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.111418 (+303902us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b540028cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:48.111621 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1955435392}
I20260709 13:17:48.218207  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:48.214030  9668 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:48.048106  9579 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:48.203109  9814 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 13:17:48.203801  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 183639us
I20260709 13:17:48.204161  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:48.202778  9812 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.173862  9660 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:48.207058  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:48.203379  9815 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:17:48.213140  6517 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:48.102427  9658 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 13:17:48.217136  9638 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:48.216006  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:48.215948  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:48.217293  6667 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:48.219084  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:48.243520  6514 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:48.032994  9523 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.209033  9411 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:48.239217  9785 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.009s
I20260709 13:17:48.213850  9622 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.237742  7237 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:48.198267  9811 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.239794  6517 pending_rounds.cc:85] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 6
I20260709 13:17:48.258335  6517 pending_rounds.cc:107] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:48.258389  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:48.243705  6515 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:48.259176  6588 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:48.244251  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:48.259788  9811 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.260054  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:48.260589  6588 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:48.261003  9786 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:48.261694  9786 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:48.262451  9786 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:48.263659  9786 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:48.263958  9786 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:48.264662  9786 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.265755  9786 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:48.245441  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:48.267485  6592 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:48.267935  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.246236  9802 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:17:48.260645  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:48.271519  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:48.272500  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:48.205189  9662 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.221169  9806 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:48.219084  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 972) took 1267 ms. Trace:
I20260709 13:17:48.222712  9613 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:48.219039  9668 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.250597  9766 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:48.251367  9411 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.258716  6517 raft_consensus.cc:2889] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:48.220707  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:48.278299  6592 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:48.244499  9638 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:48.247285  9660 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:48.295691  9824 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.302294  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:48.287155  6588 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:17:48.286574  7231 rpcz_store.cc:276] 0709 13:17:46.952066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.048586 (+ 96520us) service_pool.cc:224] Handling call
0709 13:17:47.056645 (+  8059us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:47.271683 (+215038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:47.271731 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:47.282978 (+ 11247us) log.cc:844] Serialized 26 byte log entry
0709 13:17:47.748892 (+465914us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:47.748943 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.749039 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:48.218870 (+469831us) raft_consensus.cc:1685] finished
0709 13:17:48.218894 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.219029 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5571,"raft.queue_time_us":91994,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7139,"thread_start_us":622645,"threads_started":2,"wal-append.queue_time_us":507597}
I20260709 13:17:48.324988  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:48.326120  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:48.326462  9714 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:48.311568  9662 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.327135  9714 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.329792  9714 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.273021  9785 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.313915  9523 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:48.304865  9766 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:48.274236  9744 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.316161  9316 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.332144  7228 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:48.334791  6588 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:48.317029  9812 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.317821  9613 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:48.318652  9729 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 2.160s	user 0.004s	sys 0.003s
I20260709 13:17:48.319054  9638 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:48.319942  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 99265.2us
I20260709 13:17:48.363183  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:48.286550  9806 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.298010  9596 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:48.339599  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:48.388541  7237 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:48.331645  9589 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:48.389438  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:48.390097  7237 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:48.297195  9531 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.306507  9802 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:48.341571  6592 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:48.381063  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:48.392238  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:48.360035  9291 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:48.466225  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 194277us
I20260709 13:17:48.466715  6668 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:17:48.339726  9801 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:48.469846  7237 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:48.339499  9530 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:17:48.390986  9812 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:48.387890  9806 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:48.462983  9729 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:17:48.396013  9677 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:48.394340  9802 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:48.339661  9316 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.331902  9622 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:48.475071  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 134376us
I20260709 13:17:48.473596  9801 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.482990  9782 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:48.531648  9782 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:48.532258  9782 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.533102  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 731) took 1255 ms. Trace:
I20260709 13:17:48.533466  6593 rpcz_store.cc:276] 0709 13:17:47.277770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.278763 (+   993us) service_pool.cc:224] Handling call
0709 13:17:47.284286 (+  5523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:47.373470 (+ 89184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:47.373516 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:47.373575 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:47.373619 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.532859 (+1159240us) spinlock_profiling.cc:243] Waited 88.9 ms on lock 0x7b5400210f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:48.532903 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.533043 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204508928}
W20260709 13:17:48.537230  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2051) took 1647 ms (client timeout 1729 ms). Trace:
W20260709 13:17:48.537616  6597 rpcz_store.cc:269] 0709 13:17:46.889955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.891128 (+  1173us) service_pool.cc:224] Handling call
0709 13:17:48.537049 (+1645921us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54002fdad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:48.537176 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782347648}
W20260709 13:17:48.539181  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 147010us
I20260709 13:17:48.336697  9824 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.542936  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:48.543128  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:48.553550  6519 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:48.552093  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:48.555833  6518 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:48.480615  9790 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:48.539921  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:48.340109  9744 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.472990  9596 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:17:48.566418  9613 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.472064  9291 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.523770  9668 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.525424  9802 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:48.554903  9629 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:48.528081  9677 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:48.578830  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:48.552277  9658 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:48.540971  9801 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:48.575811  9375 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.001s
I20260709 13:17:48.477582  9766 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.590552  9316 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:48.591132  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:48.592620  9658 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.575837  9824 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:48.581928  9812 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:48.609333  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.597788  9375 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:48.628674  9517 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:48.606890  9589 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:48.624142  9812 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:48.631333  9677 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:48.632772  9622 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:48.634224  9744 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:48.635672  9794 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:48.602039  9530 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:49.360052  7199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.017s	sys 0.021s
W20260709 13:17:49.360453  6563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.022s	sys 0.013s
W20260709 13:17:49.296754  6562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.003s	sys 0.014s
W20260709 13:17:49.374217  6590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.004s	sys 0.014s
W20260709 13:17:49.374596  6597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.009s	sys 0.006s
I20260709 13:17:48.907548  6519 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:49.502276  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:49.376915  9806 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:49.379492  9794 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:49.381568  9846 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.623558  9505 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:49.504747  6562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.004s	sys 0.015s
I20260709 13:17:48.660804  9658 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.391086  6595 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:17:49.391736  9749 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:49.392372  9785 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:49.393630  9631 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:17:49.394313  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.396621  9291 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.397455  9530 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.609452  9766 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:49.400128  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:48.897461  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 345213us
I20260709 13:17:49.400564  9812 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.682890  9782 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:49.402953  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:48.837244  9801 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:48.891978  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:49.406720  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1028) took 1228 ms. Trace:
I20260709 13:17:48.679811  9589 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:48.610834  9316 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.801s
W20260709 13:17:48.831159  9800 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.639s	user 0.013s	sys 0.000s
W20260709 13:17:48.859321  9838 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
I20260709 13:17:48.638032  9517 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:48.825060  9660 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.002s
W20260709 13:17:49.371883  9546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.001s	sys 0.001s
W20260709 13:17:49.499959  6563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.023s	sys 0.013s
W20260709 13:17:49.500265  6590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.005s	sys 0.015s
W20260709 13:17:49.500638  6597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.010s	sys 0.006s
I20260709 13:17:49.427321  9744 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.905596  6518 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:17:49.503841  7199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.018s	sys 0.022s
I20260709 13:17:49.523279  9800 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.526118  6515 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:49.526712  9806 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:49.508594  9749 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.511561  9631 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:49.518862  6519 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:49.530825  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 738) took 1654 ms. Trace:
I20260709 13:17:49.530690  9744 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:49.533536  9546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.001s	sys 0.001s
W20260709 13:17:49.547680  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 145290us
I20260709 13:17:49.548007  9782 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:49.548978  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:49.548570  9517 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.550212  9517 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:49.550783  9846 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.551885  9505 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.552573  9505 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.553278  9846 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.554039  9846 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:49.558766  9714 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:49.559197  9530 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:49.559566  9714 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:49.560278  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2301) took 1713 ms (client timeout 1646 ms). Trace:
W20260709 13:17:49.560719  6589 rpcz_store.cc:269] 0709 13:17:47.846547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.846667 (+   120us) service_pool.cc:224] Handling call
0709 13:17:49.559825 (+1713158us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b54003bd8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:49.560074 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31765376}
I20260709 13:17:49.561990  9714 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.559206  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 740) took 1007 ms. Trace:
I20260709 13:17:49.537585  9749 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:49.581732  6670 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:49.567009  9375 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:49.583228  9850 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:49.583961  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:49.586081  9638 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.190s	user 0.012s	sys 0.005s
I20260709 13:17:49.579257  6597 rpcz_store.cc:276] 0709 13:17:48.551477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.551626 (+   149us) service_pool.cc:224] Handling call
0709 13:17:48.555121 (+  3495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:48.557745 (+  2624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.557770 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:48.569610 (+ 11840us) log.cc:844] Serialized 52 byte log entry
0709 13:17:49.539159 (+969549us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:49.558734 (+ 19575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.558850 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.558928 (+    78us) raft_consensus.cc:1685] finished
0709 13:17:49.558942 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.559131 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2264,"raft.queue_time_us":14811,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":6619,"thread_start_us":958651,"threads_started":1,"wal-append.queue_time_us":975193}
I20260709 13:17:49.591882  9800 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.593026  9744 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.537454  6588 rpcz_store.cc:276] 0709 13:17:47.876321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.336629 (+460308us) service_pool.cc:224] Handling call
0709 13:17:48.341946 (+  5317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.342399 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.342440 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:48.342934 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:17:48.585288 (+242354us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:48.585314 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.585362 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.530225 (+944863us) raft_consensus.cc:1685] finished
0709 13:17:49.530274 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.530604 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17786,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":26636,"spinlock_wait_cycles":1711360,"thread_start_us":194793,"threads_started":1,"wal-append.queue_time_us":196831}
I20260709 13:17:49.594488  9530 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:49.595125  9785 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.595219  9782 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:49.596189  7232 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:49.621663  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:49.547757  9660 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:49.638897  9785 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.640344  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:49.641399  6665 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:17:49.514102  7230 rpcz_store.cc:276] 0709 13:17:48.178456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.535689 (+357233us) service_pool.cc:224] Handling call
0709 13:17:48.535964 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.571133 (+ 35169us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:48.585743 (+ 14610us) log.cc:844] Serialized 14 byte log entry
0709 13:17:48.980529 (+394786us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:49.401840 (+421311us) log.cc:844] Serialized 26 byte log entry
0709 13:17:49.402050 (+   210us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:49.402102 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.402220 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.406281 (+  4061us) raft_consensus.cc:1685] finished
0709 13:17:49.406304 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.406506 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54114,"prepare.queue_time_us":15635,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":34,"raft.queue_time_us":350153,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":52658,"thread_start_us":373427,"threads_started":1,"wal-append.queue_time_us":747471}
I20260709 13:17:49.674185  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:49.575093  9853 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.766784  6596 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
W20260709 13:17:49.645026  7233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.013s	sys 0.035s
I20260709 13:17:49.665319  6519 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:49.759433  9846 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:49.801172  9846 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:49.801699  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:49.801733  9846 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:49.803406  6637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 1.547s (SENT) [suppressed 45 similar messages]
I20260709 13:17:49.803628  9634 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.643643  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:49.777145  9660 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.805061  6667 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:49.805104  9660 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.644420  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:49.650056  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 597) took 1827 ms. Trace:
I20260709 13:17:49.642468  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:49.806885  9778 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.807193  6666 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:49.641647  9785 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.807541  9778 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.709149  9794 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:49.808264  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 186879us
I20260709 13:17:49.809904  9714 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:17:49.811254  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2280) took 5741 ms (client timeout 2734 ms). Trace:
W20260709 13:17:49.811444  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 139285us
I20260709 13:17:49.811750  9714 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:49.811690  6594 rpcz_store.cc:269] 0709 13:17:44.069447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.486439 (+416992us) service_pool.cc:224] Handling call
0709 13:17:49.810770 (+5324331us) spinlock_profiling.cc:243] Waited 3.18 s on lock 0x7b54002fbcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:49.811010 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7304495360}
I20260709 13:17:49.812279  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:49.808349  9660 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.813073  9854 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:49.813648  9660 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:49.813814  9854 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:49.814092  9660 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:49.810453  9794 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.814662  9660 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:49.818161  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:49.602523  9589 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.821825  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:49.823832  7233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.025s	sys 0.043s
I20260709 13:17:49.813892  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:49.806448  6668 rpcz_store.cc:276] 0709 13:17:47.822233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.822332 (+    99us) service_pool.cc:224] Handling call
0709 13:17:47.864498 (+ 42166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.469085 (+604587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.649520 (+1180435us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54001ed7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:49.649863 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384237952}
W20260709 13:17:49.759578  9802 consensus_meta.cc:287] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.171s	user 0.008s	sys 0.012s
I20260709 13:17:49.641435  9530 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.828217  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1030) took 1555 ms. Trace:
I20260709 13:17:49.651533  9505 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.677826  9668 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.676853  9744 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.701401  9735 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.703766  9767 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:49.650790  9622 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:49.833166  7233 rpcz_store.cc:276] 0709 13:17:48.272850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.979785 (+706935us) service_pool.cc:224] Handling call
0709 13:17:48.981950 (+  2165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.982786 (+   836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.982826 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:48.983737 (+   911us) log.cc:844] Serialized 26 byte log entry
0709 13:17:49.825084 (+841347us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:49.825121 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.825184 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.825413 (+   229us) raft_consensus.cc:1685] finished
0709 13:17:49.825428 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.828042 (+  2614us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":342838,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":40866,"thread_start_us":840785,"threads_started":1,"wal-append.queue_time_us":612031}
I20260709 13:17:49.636497  9638 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:49.856334  9505 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:49.680783  9743 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:49.761857  9631 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:49.868541  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.870110  9631 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.870874  9744 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.858973  9794 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:49.873101  9744 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.872705  9794 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.874769  9824 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:49.646854  9782 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.875959  9854 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:49.877085  9782 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.876807  9749 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.832036  9634 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:49.864410  9622 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.865111  9517 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:49.648612  9850 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.866801  9668 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:49.867874  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54081.5us
I20260709 13:17:49.867790  9743 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.864220  9714 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:49.874816  9735 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.865588  9845 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:49.928454  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 125085us
W20260709 13:17:49.947086  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.947719  9634 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.949913  9794 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.842857  9853 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:49.952273  9530 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:49.953547  9530 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.953702  9853 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:49.954113  9530 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.955354  9668 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.956066  9850 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.848682  9802 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.957088  9846 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.944708  9845 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.958307  9845 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:49.958586  9802 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.959520  9530 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:49.926862  9505 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.980681  9622 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.979154  9749 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.981907  9505 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.978940  9802 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:49.960290  9794 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:49.989421  9622 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:49.989727  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 171736us
I20260709 13:17:49.989801  9802 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.990156  9622 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:49.984071  9517 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:49.990600  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:49.999256  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 977) took 2295 ms. Trace:
I20260709 13:17:49.959918  6671 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:49.950487  9634 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.999614  9867 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:49.992529  9622 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:49.992381  9802 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:49.995671  9743 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:50.047605  6637 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.646s (SENT)
I20260709 13:17:50.000659  7234 rpcz_store.cc:276] 0709 13:17:47.703635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.709731 (+  6096us) service_pool.cc:224] Handling call
0709 13:17:47.710118 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:47.719993 (+  9875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:47.720021 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:47.721699 (+  1678us) log.cc:844] Serialized 52 byte log entry
0709 13:17:47.820033 (+ 98334us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:49.623167 (+1803134us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800496d20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf
0709 13:17:49.676894 (+ 53727us) log.cc:844] Serialized 14 byte log entry
0709 13:17:49.929518 (+252624us) log.cc:844] Serialized 14 byte log entry
0709 13:17:49.929928 (+   410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.930070 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.946762 (+ 16692us) raft_consensus.cc:1685] finished
0709 13:17:49.946792 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.999070 (+ 52278us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":353983,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":94,"raft.queue_time_us":17615,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":4146,"spinlock_wait_cycles":27323136,"thread_start_us":496316,"threads_started":4,"wal-append.queue_time_us":565736,"wal-append.run_cpu_time_us":3409,"wal-append.run_wall_time_us":1364288}
W20260709 13:17:50.048998  6637 leader_election.cc:336] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 1.757s (SENT)
I20260709 13:17:50.049463  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:50.049871  9861 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:17:49.992710  9812 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.050743  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:49.996743  9517 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.058203  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1026) took 1814 ms. Trace:
I20260709 13:17:49.997007  9794 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.063580  9749 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:50.064637  9872 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:50.071924  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 81410us
I20260709 13:17:50.058795  7237 rpcz_store.cc:276] 0709 13:17:48.163643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.409982 (+246339us) service_pool.cc:224] Handling call
0709 13:17:48.410226 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:48.470195 (+ 59969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.885807 (+1415612us) spinlock_profiling.cc:243] Waited 59.3 ms on lock 0x7b54006c8ed4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:49.977870 (+ 92063us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136399744}
W20260709 13:17:50.076680  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68537.1us
I20260709 13:17:49.994297  9845 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.079507  9505 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:50.081236  9790 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:50.081760  9790 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.082422  9735 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.083819  9790 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:50.084506  9846 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.085203  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:50.089489  9867 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:50.090790  9749 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:50.093648  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 895) took 2309 ms. Trace:
I20260709 13:17:50.093829  9749 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.094079  7229 rpcz_store.cc:276] 0709 13:17:47.784274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.218927 (+434653us) service_pool.cc:224] Handling call
0709 13:17:48.219216 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.219558 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.219587 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:48.220174 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:17:49.816671 (+1596497us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b18003d7f20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:17:50.018166 (+201495us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:50.044986 (+ 26820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.050540 (+  5554us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:50.093174 (+ 42634us) raft_consensus.cc:1685] finished
0709 13:17:50.093191 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:50.093444 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1407,"raft.queue_time_us":92,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3514,"spinlock_wait_cycles":4092288,"thread_start_us":150177,"threads_started":1,"wal-append.queue_time_us":1856589}
I20260709 13:17:50.096728  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:50.092054  9660 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.127414  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:50.128497  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:50.128989  9646 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:50.129439  9646 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.129788  9646 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:50.130422  9622 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:50.130786  9872 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.132998  9625 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:50.133481  9625 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:50.134764  9625 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.136219  9625 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:50.128551  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:50.144058  9517 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.147243  6666 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:17:50.148476  9735 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.148800  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.149205  9735 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.150884  9801 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:50.151423  6671 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.126338  9812 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:50.160228  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 111257us
W20260709 13:17:50.169596  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 75646.9us
I20260709 13:17:50.169551  6669 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
W20260709 13:17:50.173619  6637 leader_election.cc:336] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 1.941s (SENT)
I20260709 13:17:50.175077  9611 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.175354  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.175865  6521 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:50.172289  9634 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:50.176641  9875 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:50.176735  9634 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.177474  9634 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:50.176265  6665 raft_consensus.cc:2368] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
W20260709 13:17:50.175125  6520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 1940) took 3161 ms (client timeout 1705 ms). Trace:
W20260709 13:17:50.183756  6520 rpcz_store.cc:269] 0709 13:17:47.013397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.013686 (+   289us) service_pool.cc:224] Handling call
0709 13:17:50.165129 (+3151443us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b5400354b54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:50.174855 (+  9726us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114430720}
I20260709 13:17:50.166101  9854 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.207077  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:50.210477  6487 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.912s (SENT)
I20260709 13:17:50.203572  9517 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:50.215018  6662 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
W20260709 13:17:50.215451  9877 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:17:50.215747  9848 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.216132  9804 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:50.233539  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:50.205438  9611 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:50.223171  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.223469  9660 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.226680  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:50.224319  9517 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:50.229552  9872 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.229724  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.231154  9658 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:50.245612  9872 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.245834  9658 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.246315  9658 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.246167  9872 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.248265  9872 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:50.228662  6637 leader_election.cc:336] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 1.948s (SENT)
I20260709 13:17:50.238905  6666 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:50.238807  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:50.239356  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.239661  9611 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.251138  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.243837  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.243850  9517 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:50.253805  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.255019  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.255122  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:50.255674  6589 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:50.255915  6595 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.256812  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:50.257428  6589 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.258363  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1830) took 2215 ms. Trace:
I20260709 13:17:50.258760  6670 rpcz_store.cc:276] 0709 13:17:48.043244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.043534 (+   290us) service_pool.cc:224] Handling call
0709 13:17:48.049131 (+  5597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:49.620088 (+1570957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.258072 (+637984us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540011e4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:50.258292 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3510045440}
I20260709 13:17:50.259868  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.260483  6670 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:50.260934  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.244778  9848 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.250182  6597 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:50.262024  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.262395  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.262591  6664 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:50.263146  6597 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:17:50.240581  9804 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.250926  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.232335  9812 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:50.254892  9291 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:50.281046  9291 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:17:50.281533  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 73863.6us
I20260709 13:17:50.282052  9291 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.282855  6588 raft_consensus.cc:2410] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:50.283147  9291 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:50.283816  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:50.284946  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.285605  6597 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:17:50.286128  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 593) took 2074 ms. Trace:
I20260709 13:17:50.261245  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:50.266701  6663 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:17:50.224783  9873 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:50.271312  6666 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:50.272179  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.252830  9795 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:50.272298  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.264822  9802 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:50.264573  9790 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.263377  9848 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.286286  6520 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:17:50.372596  9852 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.723s	user 0.005s	sys 0.000s
I20260709 13:17:50.344632  9634 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.495092  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.496050  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 603) took 1587 ms. Trace:
I20260709 13:17:50.496639  6588 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:17:50.397545  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.344936  6589 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:50.503189  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.286532  6595 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
W20260709 13:17:50.373008  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 72447.9us
I20260709 13:17:50.361761  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.372203  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:50.445374  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.493566  6521 rpcz_store.cc:276] 0709 13:17:48.211642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.211777 (+   135us) service_pool.cc:224] Handling call
0709 13:17:50.175248 (+1963471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.176303 (+  1055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.285903 (+109600us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b5400354b50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:50.286040 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4475593088}
I20260709 13:17:50.511226  9873 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.501643  9517 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.445580  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:50.515426  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 300962us
I20260709 13:17:50.509703  7234 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:50.382223  9845 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.513648  9873 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.508754  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.519125  6667 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:17:50.242635  9854 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.496488  6514 rpcz_store.cc:276] 0709 13:17:48.909026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.911142 (+  2116us) service_pool.cc:224] Handling call
0709 13:17:48.911692 (+   550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.912428 (+   736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.912503 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:49.378446 (+465943us) log.cc:844] Serialized 26 byte log entry
0709 13:17:49.536845 (+158399us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:49.536907 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.537230 (+   323us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:50.369617 (+832387us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:50.495656 (+126039us) raft_consensus.cc:1685] finished
0709 13:17:50.495676 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:50.495865 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":442423,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":39131,"spinlock_wait_cycles":67180416,"thread_start_us":148804,"threads_started":1,"wal-append.queue_time_us":258611}
W20260709 13:17:50.522066  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 197042us
W20260709 13:17:50.522505  9884 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
W20260709 13:17:50.523474  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.034s (SENT)
W20260709 13:17:50.524188  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 2.034s (SENT)
I20260709 13:17:50.524573  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:50.525740  6637 leader_election.cc:336] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.180s (SENT)
I20260709 13:17:50.526193  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:50.287051  9662 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:50.526571  6589 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:17:50.526957  9812 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:48.153964  6522 rpcz_store.cc:276] 0709 13:17:46.482258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.482416 (+   158us) service_pool.cc:224] Handling call
0709 13:17:46.576565 (+ 94149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:46.576812 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:46.671761 (+ 94949us) log.cc:844] Serialized 14 byte log entry
0709 13:17:47.693295 (+1021534us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b1800327040. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:17:48.035740 (+342445us) log.cc:844] Serialized 14 byte log entry
0709 13:17:48.035961 (+   221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:48.036007 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:48.036057 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.036136 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.048019 (+ 11883us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":193,"prepare.queue_time_us":344225,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":19518,"raft.queue_time_us":13482,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":18267,"spinlock_wait_cycles":16674176,"thread_start_us":341908,"threads_started":2,"wal-append.queue_time_us":1315215}
I20260709 13:17:50.528190  6516 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:17:50.531263  7228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 899) took 2742 ms (client timeout 1912 ms). Trace:
I20260709 13:17:50.497027  9634 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.306289  9794 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:50.369669  9646 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.533665  7230 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:50.321148  9735 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:50.532625  7228 rpcz_store.cc:269] 0709 13:17:47.785372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.916719 (+131347us) service_pool.cc:224] Handling call
0709 13:17:50.527322 (+2610603us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54005a3554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:50.527527 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718142208}
I20260709 13:17:50.542765  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.543496  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.543437  6589 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:50.545212  6595 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:50.494481  9790 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.530210  9530 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:50.547447  7231 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.548151  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.548758  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.549098  9783 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:50.553768  9646 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:50.550534  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2307) took 2699 ms (client timeout 2034 ms). Trace:
W20260709 13:17:50.554881  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.555768  6486 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.986s (SENT)
I20260709 13:17:50.556308  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:50.556557  6589 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:50.555016  7234 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:50.559073  7236 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:17:50.557986  6596 rpcz_store.cc:269] 0709 13:17:47.851198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.891155 (+ 39957us) service_pool.cc:224] Handling call
0709 13:17:50.550307 (+2659152us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54002fe4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:50.550452 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3468727680}
I20260709 13:17:50.541234  9812 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:50.561256  6588 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:50.538805  9795 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.564292  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.566566  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.564881  7233 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:50.567492  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.568198  7236 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:50.568163  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.569092  7230 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:50.569422  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.570116  6596 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:50.570955  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.560468  9530 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.572257  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.569171  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.574831  6513 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:17:50.572831  7234 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:50.577471  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.544394  9802 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.575733  7235 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:50.584661  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.582968  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:50.584545  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.587990  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:50.589488  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.589422  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.590445  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.590451  7234 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:50.591127  6513 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.591332  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.591943  9794 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:17:50.593098  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72243us
I20260709 13:17:50.593677  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:50.593890  9661 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:50.596498  7236 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:50.593778  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.594524  9726 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:50.594328  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:50.599190  7233 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:50.596302  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:50.595355  6520 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:50.599361  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.597525  6589 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:50.600163  6595 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:50.596107  9661 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:50.600797  9812 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:50.600867  9661 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:50.600895  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.602568  7235 raft_consensus.cc:2410] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:50.605527  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.605515  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:50.605239  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:50.606799  7234 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.608193  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.601300  9812 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:50.610257  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.614255  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.615332  9775 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:50.617064  9778 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.617074  9775 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.618247  9775 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.623019  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.623350  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.623868  6519 raft_consensus.cc:2376] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5. Re-sending same reply.
I20260709 13:17:50.625272  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.625157  6522 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:50.615957  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.626924  6515 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:50.612293  9812 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:50.645327  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.655956  6595 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:50.662526  6589 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:50.667407  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 741) took 2091 ms. Trace:
I20260709 13:17:50.667897  6593 rpcz_store.cc:276] 0709 13:17:48.575502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.590892 (+ 15390us) service_pool.cc:224] Handling call
0709 13:17:48.591199 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.591473 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.591498 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.521138 (+1929640us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b2c0012b1d8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:17:50.521632 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.664367 (+142735us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:50.664401 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.664504 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:50.666973 (+  2469us) raft_consensus.cc:1685] finished
0709 13:17:50.666990 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:50.667241 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36124,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":69,"spinlock_wait_cycles":12039168,"thread_start_us":142350,"threads_started":1,"wal-append.queue_time_us":142829}
I20260709 13:17:50.667932  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.677021  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.677985  6589 raft_consensus.cc:2410] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:50.680739  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.679637  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.626416  9646 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.686899  6520 raft_consensus.cc:2376] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8. Re-sending same reply.
I20260709 13:17:50.691406  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.693297  6517 raft_consensus.cc:2376] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6. Re-sending same reply.
I20260709 13:17:50.695665  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.696254  6522 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:50.696164  9646 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.696776  9652 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:17:50.697170  9856 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.094s	user 0.000s	sys 0.002s
W20260709 13:17:50.702412  9887 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:17:50.661998  9661 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.698796  9848 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.711942  9848 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.713358  9638 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:50.715107  6416 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 328) took 21507 ms (client timeout 15000 ms). Trace:
I20260709 13:17:50.715353  9611 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:50.701036  6597 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.715881  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.709265  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.698068  9775 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.718740  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:50.717679  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.720014  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.715154  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.720883  7233 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:50.721863  7236 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [4.7-5.8]
I20260709 13:17:50.722303  6515 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:17:50.725939  6589 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:50.705971  7234 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:50.723517  9775 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.724778  6595 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:50.722193  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 113502us
I20260709 13:17:50.731767  6597 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:50.731455  9658 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.733194  9661 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.737941  6520 raft_consensus.cc:2376] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6. Re-sending same reply.
W20260709 13:17:50.738389  9885 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:17:50.740931  9291 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:50.742515  9790 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.733669  9530 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.716552  9845 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:50.771154  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.772011  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:50.746183  9505 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:50.751327  6666 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:50.762151  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.687683  9795 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.766342  6670 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:50.771027  6593 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:17:50.772014  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.782001  6594 raft_consensus.cc:2368] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7.
I20260709 13:17:50.774585  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.774827  6513 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:50.774843  6671 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:50.776988  6595 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:50.780624  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.784016  6520 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:50.784600  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.786278  6663 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:50.782699  6665 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:50.783102  6669 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:50.788355  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:50.791833  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 188289us
I20260709 13:17:50.798754  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.817636  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:50.774782  9790 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:50.836153  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.838313  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.840600  6519 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:17:50.849313  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 333955us
I20260709 13:17:50.850700  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.838629  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:50.789785  9878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.006s	sys 0.020s
W20260709 13:17:50.835404  9885 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:17:50.794975  9662 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:50.856632  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:50.854264  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.861009  6665 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:50.858978  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.862176  6515 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:50.862106  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.863085  6519 raft_consensus.cc:2410] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:17:50.856102  9878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.007s	sys 0.020s
I20260709 13:17:50.865455  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.865511  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:50.865757  9646 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:50.866888  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 234721us
I20260709 13:17:50.893116  6595 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:50.868027  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2308) took 3005 ms (client timeout 2180 ms). Trace:
I20260709 13:17:50.873171  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 739) took 2996 ms. Trace:
I20260709 13:17:50.873795  9658 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.874686  9505 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.900408  9505 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:50.900758  9505 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:50.900897  9804 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.850302  9875 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:50.902064  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.880688  9812 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.881283  9873 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:50.863379  9795 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:17:50.907966  9873 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:50.912026  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49695.2us
I20260709 13:17:50.913064  9795 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:50.913602  9812 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.860708  9662 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:50.855279  9848 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:50.913968  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.897518  9836 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:50.894294  6591 rpcz_store.cc:269] 0709 13:17:47.856708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.993726 (+137018us) service_pool.cc:224] Handling call
0709 13:17:50.858714 (+2864988us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54000933d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:50.861656 (+  2942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441912192}
I20260709 13:17:50.898653  6486 leader_election.cc:400] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
I20260709 13:17:50.899318  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:50.878188  9801 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.902526  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.908634  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:50.912803  9873 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.914106  9898 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:50.715663  6416 rpcz_store.cc:269] 0709 13:17:29.207900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.210408 (+  2508us) service_pool.cc:224] Handling call
0709 13:17:29.712851 (+502443us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8feb090 00007f04b9089f3a 00007f04bee401cc 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc 00007f04bd3fe392 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:50.714844 (+21001993us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:39.734762 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:39.735516 (+   754us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:39.735572 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:17:39.735606 (+    34us) tablet.cc:662] Decoding operations
0709 13:17:39.736429 (+   823us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:39.736527 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:39.736572 (+    45us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:17:39.737325 (+   753us) tablet.cc:701] Row locks acquired
0709 13:17:39.737381 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:17:39.737556 (+   175us) write_op.cc:270] Start()
0709 13:17:39.737703 (+   147us) write_op.cc:276] Timestamp: P: 1783603059737492 usec, L: 0
0709 13:17:39.737735 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:17:39.863142 (+125407us) log.cc:844] Serialized 2054 byte log entry
0709 13:17:41.126303 (+1263161us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b
0709 13:17:41.630290 (+503987us) op_driver.cc:464] REPLICATION: finished
0709 13:17:41.799973 (+169683us) spinlock_profiling.cc:243] Waited 9.46 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04bcab7ef0
0709 13:17:43.089370 (+1289397us) write_op.cc:301] APPLY: starting
0709 13:17:43.089492 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:43.089752 (+   260us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:43.089803 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:43.103803 (+ 14000us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:43.104235 (+   432us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:17:43.104307 (+    72us) write_op.cc:312] APPLY: finished
0709 13:17:43.104784 (+   477us) log.cc:844] Serialized 47 byte log entry
0709 13:17:43.347160 (+242376us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:43.356430 (+  9270us) write_op.cc:454] Released schema lock
0709 13:17:43.408420 (+ 51990us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:43.534537 (+126117us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:43.555971 (+ 21434us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":18,"spinlock_wait_cycles":1028091136,"child_traces":[["op",{"apply.queue_time_us":1458723,"apply.run_cpu_time_us":22517,"apply.run_wall_time_us":436260,"num_ops":5,"prepare.queue_time_us":368658,"prepare.run_cpu_time_us":34049,"prepare.run_wall_time_us":1613128,"raft.queue_time_us":174965,"raft.run_cpu_time_us":24044,"raft.run_wall_time_us":1317077,"replication_time_us":1892374,"spinlock_wait_cycles":92331264,"thread_start_us":3189875,"threads_started":5,"wal-append.queue_time_us":1752478,"wal-append.run_cpu_time_us":12944,"wal-append.run_wall_time_us":2203254}]]}
I20260709 13:17:50.915380  9795 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:50.915412  9662 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.913894  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.913621  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.925509  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.918099  6520 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:50.917949  9836 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.921679  9875 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.923242  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:50.894371  6592 rpcz_store.cc:276] 0709 13:17:47.876558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.467494 (+590936us) service_pool.cc:224] Handling call
0709 13:17:48.483795 (+ 16301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.484244 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.484283 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:48.486033 (+  1750us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.522821 (+2036788us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800412900. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:17:50.794862 (+272041us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:50.835302 (+ 40440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.849943 (+ 14641us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:50.872687 (+ 22744us) raft_consensus.cc:1685] finished
0709 13:17:50.872708 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:50.872986 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":741,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":38826,"spinlock_wait_cycles":24953088,"thread_start_us":224852,"threads_started":1,"wal-append.queue_time_us":2258084}
I20260709 13:17:50.923890  9801 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.924902  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:50.919412  6516 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:50.935575  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.919534  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.926258  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:50.932860  6594 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:50.901711  9505 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:50.938457  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.938009  6597 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:50.939877  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:50.938871  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.939931  9873 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.939612  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:50.943879  6596 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:50.944068  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.937291  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.941797  6519 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:50.930418  9799 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:50.942399  6520 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:50.942965  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.925678  9898 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:50.935707  9836 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:50.954213  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.954509  6666 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:50.955621  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.955837  9898 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:50.956799  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.957279  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.957481  9898 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:50.960249  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 963) took 4056 ms. Trace:
I20260709 13:17:50.960628  7232 rpcz_store.cc:276] 0709 13:17:46.903716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.904251 (+   535us) service_pool.cc:224] Handling call
0709 13:17:46.905091 (+   840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:49.642298 (+2737207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.959961 (+1317663us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b5400692dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:50.960196 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6187379584}
I20260709 13:17:50.962510  9662 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:50.963228  6662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 595) took 3141 ms (client timeout 2149 ms). Trace:
W20260709 13:17:50.963620  6662 rpcz_store.cc:269] 0709 13:17:47.821704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.821823 (+   119us) service_pool.cc:224] Handling call
0709 13:17:50.962988 (+3141165us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54003f5f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:50.963142 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5197584256}
W20260709 13:17:50.965634  9883 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.735s	user 0.000s	sys 0.005s
I20260709 13:17:50.967175  9749 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:50.967693  9749 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.967924  9802 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:50.968070  9749 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:50.969146  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:50.972374  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.940115  9735 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:50.967229  6667 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:50.977644  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:50.978470  6589 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:50.975854  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:50.950083  9767 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:50.979353  6668 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:17:50.980840  9505 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.979400  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:50.933386  9812 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:17:50.947820  9799 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:51.001101  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:51.003705  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.011454  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.012180  6589 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
W20260709 13:17:51.012528  9876 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.801s	user 0.004s	sys 0.006s
I20260709 13:17:51.015090  9875 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.015527  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:51.011454  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.024960  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:51.026125  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.026927  6662 raft_consensus.cc:2410] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:51.026376  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.028288  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:51.006106  9799 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:51.036302  9635 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:51.037329  9635 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:51.037642  9635 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:51.038888  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 85166.9us
I20260709 13:17:51.038378  9635 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.041168  9530 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:51.042197  9530 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:51.042521  9530 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:51.042879  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2298) took 3200 ms. Trace:
I20260709 13:17:51.043434  9530 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.044727  9530 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:51.043365  6590 rpcz_store.cc:276] 0709 13:17:47.842308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.843955 (+  1647us) service_pool.cc:224] Handling call
0709 13:17:48.160797 (+316842us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:48.161190 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.161219 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:48.315642 (+154423us) spinlock_profiling.cc:243] Waited 953 us on lock 0x7b2c0012b1d8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:17:48.316136 (+   494us) log.cc:844] Serialized 52 byte log entry
0709 13:17:48.384785 (+ 68649us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:48.576527 (+191742us) log.cc:844] Serialized 14 byte log entry
0709 13:17:49.835416 (+1258889us) log.cc:844] Serialized 14 byte log entry
0709 13:17:50.205092 (+369676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.207764 (+  2672us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:50.213293 (+  5529us) raft_consensus.cc:1685] finished
0709 13:17:50.213309 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.042667 (+829358us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400303750. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:51.042806 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5853,"prepare.queue_time_us":633725,"prepare.run_cpu_time_us":147,"prepare.run_wall_time_us":33954,"raft.queue_time_us":115908,"raft.run_cpu_time_us":579,"raft.run_wall_time_us":57881,"spinlock_wait_cycles":730043520,"thread_start_us":1720345,"threads_started":6,"wal-append.queue_time_us":239152,"wal-append.run_cpu_time_us":2804,"wal-append.run_wall_time_us":1235852}
I20260709 13:17:51.010186  9802 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.046819  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1027) took 2882 ms. Trace:
I20260709 13:17:51.046942  9802 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.047371  7231 rpcz_store.cc:276] 0709 13:17:48.164388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.474398 (+310010us) service_pool.cc:224] Handling call
0709 13:17:50.543029 (+2068631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.562445 (+ 19416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.046598 (+484153us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54005a3550. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:51.046760 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4721218432}
I20260709 13:17:51.049120  9802 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:51.049494  9611 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:51.011272  9899 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:17:51.049983  9611 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:51.027154  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.031718  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:51.032001  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:51.032730  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.033058  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.033227  9867 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:51.054699  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.027483  9795 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:51.055177  9903 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:51.055625  9795 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:51.055706  9903 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.055936  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.056228  9903 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:51.057516  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.058429  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:51.035080  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.035641  9845 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:51.059012  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.065807  6516 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 5: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:51.035394  9714 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:51.041286  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:51.051187  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.029875  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:51.082306  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 47669.1us
I20260709 13:17:51.082849  6522 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:51.084717  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:17:51.086560  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 160353us
I20260709 13:17:51.088147  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:51.089759  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.096565  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.082111  9845 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.060607  9875 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:51.081910  9662 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.098199  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.084789  9635 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.086359  9802 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:51.085889  9848 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:51.112821  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45343.2us
I20260709 13:17:51.086145  9767 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:51.107553  9662 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.115195  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:51.115419  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.051136  9611 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.101440  9804 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:51.084641  9646 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:17:51.101929  9812 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:51.153332  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:51.154033  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 102912us
I20260709 13:17:51.142962  9735 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:51.148685  9662 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.112301  9802 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.108119  9783 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.163595  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.116362  9749 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.158428  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:51.160461  9794 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:51.155459  9804 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.178869  9795 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.182132  6417 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 3, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.186683  9910 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:17:51.240795  9909 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:51.187350  9896 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:17:51.188385  9735 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:51.178987  9783 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.149628  9867 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:17:51.194572  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.195745  9854 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.196370  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.197072  7232 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:51.199220  9901 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:51.199488  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.200404  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:51.169704  9812 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.246251  7229 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:51.246423  6514 raft_consensus.cc:2410] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:51.202234  9801 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:51.203233  9530 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:51.203620  9517 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.205286  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.206449  9646 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:51.207419  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:51.208559  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.171262  9611 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:51.212227  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.256353  7233 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:51.213914  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.214645  9822 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:51.219813  9912 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.168354  9905 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:51.180341  9802 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.258354  9646 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:51.273595  9646 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:51.275108  9646 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:51.283196  6417 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.286247  6417 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 4, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.230403  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:51.232558  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 69033.1us
I20260709 13:17:51.296265  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.297108  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.234817  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:51.235697  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 147220us
I20260709 13:17:51.243847  7231 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:51.244185  9783 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.247957  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:51.249387  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.249938  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:51.239984  9804 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:51.254992  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71668.7us
I20260709 13:17:51.256268  7234 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:51.205973  9845 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.257028  7235 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:51.242343  9896 consensus_queue.cc:1243] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:51.260540  9905 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:51.264829  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.267771  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:51.252669  9899 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:51.288730  9812 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.259536  9822 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:51.108647  9836 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.298975  6417 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.297878  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.320346  9634 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:51.306345  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:51.306375  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 71634.5us
I20260709 13:17:51.356504  7233 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:51.306962  9291 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.307034  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.307446  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.307876  9505 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:51.308111  9867 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:51.308594  9780 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:51.346897  9775 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.374281  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.309860  9910 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:51.310461  9854 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:17:51.310753  9767 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:51.258823  9799 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.314114  9903 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.316299  9804 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.245690  9901 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:51.319207  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.367264  9505 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:51.231654  9908 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:17:51.334690  6517 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:51.336621  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.336733  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.417055  7231 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:51.337502  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.337831  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.339571  9658 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:51.340163  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 242041us
I20260709 13:17:51.419238  6666 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:51.379700  9917 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:51.341272  9878 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.342511  9848 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:17:51.343078  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 227955us
I20260709 13:17:51.343587  9886 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:51.344350  9905 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:51.274348  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 219363us
I20260709 13:17:51.262049  9912 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:17:51.354522  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56703.2us
I20260709 13:17:51.436206  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.440155  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:51.258074  9530 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:51.371315  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64308.4us
I20260709 13:17:51.442840  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.309270  9778 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:51.377650  6663 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:51.345228  9899 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:51.379402  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.359869  9634 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:51.388672  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.319748  9802 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.391628  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.394577  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.395833  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.362224  9291 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:51.398423  6417 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 3 to 5, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.331308  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.400583  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.360577  9913 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:51.388347  9775 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:51.405550  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.418457  7236 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:51.339874  9783 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.428086  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.493825  6669 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:51.431319  9918 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:51.417747  9505 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:51.442065  7231 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:17:51.445966  7237 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:51.425820  9878 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.429103  9749 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.491535  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:51.492022  7229 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:51.492767  6667 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:51.517257  7232 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:51.517586  7230 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:51.496443  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 69872.4us
I20260709 13:17:51.430423  7228 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:51.500699  9795 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.431578  9917 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:51.521777  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 85647.6us
I20260709 13:17:51.522748  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:51.524412  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.524722  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:51.520653  6562 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.492620  9634 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.494258  9778 consensus_queue.cc:1243] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:51.538100  7233 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:51.444373  9912 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:51.538801  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 96055us
I20260709 13:17:51.530800  6417 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 3, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 3 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.497766  9291 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.497931  9913 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:51.546017  6593 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:17:51.546557  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:51.519138  9867 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.403472  9822 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.518739  9517 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:51.519016  9878 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.562408  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:51.527585  9801 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.543504  9925 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:51.549043  6417 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 5 to 6, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 6 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.560961  6595 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:51.535024  9909 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:51.505407  9918 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:51.540511  9749 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:51.536973  9795 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.560523  9845 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.570983  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:51.571218  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 131158us
I20260709 13:17:51.565235  6597 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:51.581420  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.582067  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.582885  6597 raft_consensus.cc:2410] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:51.551904  9867 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.552201  9646 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:51.583961  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.580204  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:51.555119  9658 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:51.584848  6593 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:17:51.548316  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 147797us
I20260709 13:17:51.548892  9913 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.577440  9688 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:51.589039  6417 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.622013  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:51.622665  6517 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:51.623417  6417 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 4, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.619203  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:51.625715  6417 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 3 to 5, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.627599  6417 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.555326  9921 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:17:51.593899  7232 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:17:51.632329  6595 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:51.590076  9867 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.634305  7237 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:51.587457  9661 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:51.569841  9873 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:17:51.638597  6562 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.566167  9916 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:17:51.623785  9836 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.549973  9848 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.620769  9895 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:51.599992  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:51.677042  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 114833us
I20260709 13:17:51.674734  7235 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:51.587497  9801 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.711565  9799 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:51.714293  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 80749.5us
I20260709 13:17:51.714823  7236 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:17:51.673735  9661 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:51.637293  9658 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:51.674723  9922 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:17:51.719465  7236 pending_rounds.cc:85] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 7
W20260709 13:17:51.724431  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 124527us
I20260709 13:17:51.632045  9929 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:51.681682  9775 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:51.685253  9923 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:17:51.593756  9924 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:51.776149  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.565923  9899 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.729784  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.681072  9795 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:51.683003  9783 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:51.731273  7229 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:17:51.778173  7229 pending_rounds.cc:85] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 6
I20260709 13:17:51.778486  7229 pending_rounds.cc:107] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:17:51.778780  7229 raft_consensus.cc:2889] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:51.677770  9804 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.735607  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.791919  6597 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:51.724016  7236 pending_rounds.cc:107] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:17:51.792958  7236 raft_consensus.cc:2889] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:51.712919  9903 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:51.718324  9895 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.723567  9517 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:51.740211  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 101728us
I20260709 13:17:51.586038  9909 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.740819  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:51.718760  9749 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:51.724237  9658 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.651284  9917 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.724431  9661 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:51.745946  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.742105  9634 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.747661  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:51.760784  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:51.641613  9646 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
W20260709 13:17:51.786032  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56309.8us
W20260709 13:17:51.787300  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 262630us
I20260709 13:17:51.808962  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.789321  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:51.761265  9662 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
I20260709 13:17:51.776786  6591 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:51.797227  9767 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:51.593488  9530 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.770869  9929 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:51.804507  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63762us
I20260709 13:17:51.804311  6593 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:51.773224  9802 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:51.737473  9905 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.799062  9895 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:51.820544  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.829587  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:51.829743  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:51.830230  6668 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.834435  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:51.835289  6671 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.823550  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:51.830221  6669 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:51.849334  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:51.853675  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:51.854770  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45878.8us
I20260709 13:17:51.855777  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:51.857579  6664 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:51.859079  6667 pending_rounds.cc:85] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 4
I20260709 13:17:51.859478  6667 pending_rounds.cc:107] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:17:51.859776  6667 raft_consensus.cc:2889] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:51.810890  9795 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:51.847860  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:51.863729  7235 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:51.839041  9688 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:51.864953  9688 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.678880  9918 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.866253  9918 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.812884  9903 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:51.881492  6668 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:51.891920  9854 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:51.892791  9611 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:51.892618  9854 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:51.893172  9611 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.893559  9611 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.894834  6666 raft_consensus.cc:1217] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-7.8]   Dedup: 3.6->[5.7-7.8]
I20260709 13:17:51.895329  6666 pending_rounds.cc:85] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
I20260709 13:17:51.895542  6666 pending_rounds.cc:107] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:17:51.895751  6666 raft_consensus.cc:2889] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:51.892058  9801 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:51.826224  9929 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:51.806838  9505 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.907594  9939 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:51.811081  9886 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:17:51.823875  9783 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:17:51.814956  9662 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:51.878523  9896 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.839366  9812 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:17:51.812896  9931 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:17:51.915736  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:51.905253  9923 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:17:51.825628  9925 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:17:51.901736  9517 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:51.920737  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.915124  9903 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:17:51.919559  9635 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:51.903663  9795 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:51.883467  9932 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:51.935782  9801 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:51.909076  9854 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:51.908020  6663 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 64b95375974b48d18c596a23c9cac75c (127.5.72.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:17:51.941582  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.877951  9926 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:17:51.919153  9896 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.942729  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:51.925858  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 179945us
I20260709 13:17:51.949467  9903 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.953856  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.945354  9934 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:51.957160  7230 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:17:51.939013  9939 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:17:51.943193  9783 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:51.943254  9635 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:51.935544  9794 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
W20260709 13:17:51.875883  9799 consensus_queue.cc:1175] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64b95375974b48d18c596a23c9cac75c" responder_term: 6 status { last_received { term: 3 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [4.7-5.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:51.968910  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:51.970500  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:51.955457  9917 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:51.957527  9790 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.950496  9801 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:51.918990  9749 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:52.037158  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 68279us
I20260709 13:17:51.959110  9896 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.964214  9903 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:51.942193  9517 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.093559  6671 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:52.074538  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:17:52.090083  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 125272us
I20260709 13:17:52.087487  7235 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:52.062949  9915 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:52.159042  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:52.160069  6664 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:52.007652  9530 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:52.161432  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.162858  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:52.174110  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.174253  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.172570  7235 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:52.175482  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:52.073810  9898 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.074836  9634 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.057852  9917 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:52.070452  9291 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.512s	user 0.004s	sys 0.001s
I20260709 13:17:52.179499  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:52.179739  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:52.181653  6517 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:52.159178  9916 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:52.184767  6588 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:52.188174  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.061717  9611 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:52.180945  9940 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
I20260709 13:17:52.080267  9517 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.116894  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:52.207551  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.114356  9878 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.212463  6595 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:52.173688  9505 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.214745  6592 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:52.079092  9903 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.071089  9635 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.073717  9662 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.222148  7235 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:52.075492  9867 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.209374  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.229427  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:52.231055  7231 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:52.238829  6590 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:52.199239  9909 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.232486  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:52.219939  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:52.206192  9916 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:52.202531  9905 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.074033  9795 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.177814  9944 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:52.231787  6514 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:52.240643  7237 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:52.243587  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:52.071300  9801 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.250844  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.254033  6671 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:52.255337  6517 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:52.213358  9291 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.260790  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.262915  6514 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:52.252554  9790 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:52.252717  9916 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:52.252154  9517 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.255299  9635 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.285676  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.289431  6665 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:52.280499  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 91248.7us
I20260709 13:17:52.292207  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.259174  9895 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.307094  9895 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.310182  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.311312  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.311439  9867 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.312090  6514 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:52.312824  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.313536  6514 raft_consensus.cc:2410] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:52.313571  7237 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:52.314584  6514 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:52.257692  9658 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.341884  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.258164  9944 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.398937  9944 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:52.399751  9912 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.844s	user 0.010s	sys 0.000s
I20260709 13:17:52.400184  9912 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.401178  6522 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:52.404228  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 106499us
I20260709 13:17:52.405530  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:52.406839  9957 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:52.407526  9878 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.407219  9956 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:52.408787  6518 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:52.316102  9611 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:52.437726  6487 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.659s (SENT)
I20260709 13:17:52.439621  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.410178  9896 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.340268  9928 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:52.289145  9517 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.445428  9944 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.467006  9928 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.467679  9944 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.468477  9896 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.469647  9944 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.469632  9928 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.481230  9912 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.316422  9790 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:17:52.487603  9790 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:52.488116  9790 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:52.489326  9790 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.492648  9291 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.449085  9918 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:52.499223  9918 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.501994  9918 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:52.450661  9867 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:52.505571  9916 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.450944  9854 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.507907  9854 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.525086  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 999) took 1095 ms. Trace:
I20260709 13:17:52.525699  7236 rpcz_store.cc:276] 0709 13:17:51.429816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.630089 (+200273us) service_pool.cc:224] Handling call
0709 13:17:51.641995 (+ 11906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.799355 (+157360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.524707 (+725352us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b5400469454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:52.524975 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102624640}
I20260709 13:17:52.414572  9929 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.298748  9661 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.529815  9517 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:52.547299  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:52.547883  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.552322  6513 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:52.560165  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 154692us
I20260709 13:17:52.563544  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:52.565649  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.564919  6517 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:52.510567  9854 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.569180  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.558331  9845 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:52.571921  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.572253  9845 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.572703  6518 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:52.574980  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.576282  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.577638  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.579059  9661 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.580292  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.581699  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.585803  6596 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:52.587071  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.587308  9949 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.587785  6596 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:52.587971  9949 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.588914  6595 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:17:52.589861  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.591176  9949 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.593132  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:52.593643  9896 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:52.594790  9968 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:52.573879  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.595866  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:52.595978  9929 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.596290  6518 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:52.598353  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:52.598518  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1985) took 1015 ms. Trace:
I20260709 13:17:52.599314  9968 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:52.600137  9968 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:52.600526  9968 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.598687  9916 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.600935  9968 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:52.601157  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.601994  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:52.601811  9845 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.602828  6588 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:52.605062  9896 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:52.606319  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:52.597071  6596 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:52.607355  9783 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:52.594877  9836 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:52.607631  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:52.608135  9836 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:52.608927  9801 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.609023  9836 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.610301  9836 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:52.596194  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.614912  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:52.615026  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.615636  6666 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:52.599131  6522 rpcz_store.cc:276] 0709 13:17:51.582990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.583098 (+   108us) service_pool.cc:224] Handling call
0709 13:17:51.619649 (+ 36551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.401535 (+781886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.598319 (+196784us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b5400216bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:52.598465 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796880256}
W20260709 13:17:52.654906  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 91469.1us
I20260709 13:17:52.663935  6663 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:52.676177  6588 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:52.591729  6595 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:52.677692  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.671957  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:52.678850  6664 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
W20260709 13:17:52.564587  9913 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.925s	user 0.012s	sys 0.000s
I20260709 13:17:52.663523  9929 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.667033  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:52.684388  6667 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:52.602286  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:17:52.684706  6666 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:52.684976  6665 raft_consensus.cc:606] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.685382  6665 raft_consensus.cc:614] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:52.667765  9714 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:52.668116  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.668329  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.672792  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.558418  9635 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.608945  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:52.682299  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.681599  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.667235  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 1984) took 1084 ms. Trace:
I20260709 13:17:52.665735  9783 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:52.687772  9910 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:52.666018  9790 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:52.704252  6595 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:52.686353  9929 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:52.693605  6637 leader_election.cc:336] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.734s (SENT)
I20260709 13:17:52.694298  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:52.706090  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.614534  9968 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:52.701709  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 92838.8us
I20260709 13:17:52.673512  9845 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:52.692025  6515 rpcz_store.cc:276] 0709 13:17:51.582573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.582721 (+   148us) service_pool.cc:224] Handling call
0709 13:17:51.592713 (+  9992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.592927 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:51.765429 (+172502us) log.cc:844] Serialized 14 byte log entry
0709 13:17:52.664903 (+899474us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b1800499d80. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287
0709 13:17:52.666734 (+  1831us) log.cc:844] Serialized 14 byte log entry
0709 13:17:52.666923 (+   189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.666969 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:52.667008 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.667073 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.667183 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44000,"prepare.queue_time_us":257888,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":31473,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":17311,"spinlock_wait_cycles":3609984,"thread_start_us":269195,"threads_started":3,"wal-append.queue_time_us":32159}
I20260709 13:17:52.693303  9898 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:52.710714  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:52.701373  9517 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:52.697888  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.713002  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.704672  9714 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:52.714200  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.716385  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.718778  6518 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:17:52.719120  6518 pending_rounds.cc:85] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 3
I20260709 13:17:52.719327  6518 pending_rounds.cc:107] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:17:52.719537  6518 raft_consensus.cc:2889] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.716593  6596 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:52.704000  9530 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:52.703922  9635 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.717520  9517 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:52.717129  6588 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:52.703332  9910 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:52.724565  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:52.752342  9635 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.771495  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.803084  6595 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.779228  6667 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:17:52.803911  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79450.4us
I20260709 13:17:52.728454  9517 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.744024  9714 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:52.752980  6590 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:17:52.809913  6590 pending_rounds.cc:85] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 1
I20260709 13:17:52.810190  6590 pending_rounds.cc:107] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:52.810554  6590 raft_consensus.cc:2889] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.791342  9530 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.725183  9661 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:52.791790  6669 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:17:52.815033  6669 pending_rounds.cc:85] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 7
I20260709 13:17:52.815281  6669 pending_rounds.cc:107] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:17:52.815508  6669 raft_consensus.cc:2889] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.813990  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:52.816344  6588 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:17:52.792397  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 137467us
I20260709 13:17:52.761696  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:52.775142  9963 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:17:52.692998  9913 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.726423  9795 raft_consensus.cc:2764] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:17:52.804407  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 124191us
W20260709 13:17:52.879949  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.884307  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:52.811805  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55199.8us
I20260709 13:17:52.814877  9661 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:52.820530  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:52.887550  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 67089.8us
I20260709 13:17:52.871757  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:52.876345  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.780387  9972 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:52.810084  9921 raft_consensus.cc:2764] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:17:52.876912  6517 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:52.888792  6588 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:52.886093  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.889976  6596 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:52.896788  6595 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
W20260709 13:17:52.902220  9974 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:17:52.905167  9658 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.906610  9920 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:52.909263  9920 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:52.912931  9913 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:52.882524  9646 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 13:17:52.906953  9971 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:17:52.916281  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.917833  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:52.919447  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:52.922600  9899 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:52.927103  6595 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:52.891613  9661 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:52.892683  9735 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:52.938024  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:52.938797  9634 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:52.938820  9912 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:52.948032  6517 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:52.948158  9912 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:52.948809  9912 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.898038  9867 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:52.898957  9898 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:52.953455  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:52.955832  6589 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:52.817544  9530 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:52.969444  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 51963.4us
I20260709 13:17:52.966244  6595 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:17:52.970917  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:52.958596  6588 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:52.972487  6596 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:52.975097  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:52.949350  9912 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:52.956377  9929 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.993090  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:52.956894  9898 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.947124  9735 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:52.885392  9910 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.948288  9944 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:17:52.967814  9867 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:52.977471  9896 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:52.980341  9795 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:52.964246  9968 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:52.991168  9635 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.978081  9801 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:17:52.996153  9898 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.999289  9921 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:53.012725  9969 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:17:53.001772  9901 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.001936  9896 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:53.005775  9635 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.008450  9920 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:52.996549  9957 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:17:53.024948  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77035.4us
I20260709 13:17:53.025848  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:53.035188  9977 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:17:53.035682  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 990) took 1722 ms. Trace:
I20260709 13:17:53.038443  7233 rpcz_store.cc:276] 0709 13:17:51.305095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.358150 (+ 53055us) service_pool.cc:224] Handling call
0709 13:17:53.027108 (+1668958us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54004de254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:53.027247 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372949248}
I20260709 13:17:53.044482  9635 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.046909  9867 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.049273  9917 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.050725  9915 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.051231  9915 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.051710  9822 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.052825  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:53.053583  9944 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:53.053932  9944 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:53.054255  9944 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.054708  9735 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.023439  9918 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:53.056174  9918 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.058243  9918 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:53.027534  9896 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.059103  9688 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.066001  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:53.035847  9921 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:53.050441  9917 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.069483  6521 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:53.080956  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 59855.7us
I20260709 13:17:53.082484  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:53.083230  9901 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.083837  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:53.072976  9799 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:53.089062  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.089421  6592 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:53.089900  9854 raft_consensus.cc:2764] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:17:53.090688  9854 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:53.090984  6592 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:53.073491  9968 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:53.072194  9921 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.096215  9944 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:53.097569  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.094137  9968 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.081017  9928 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:53.099454  9928 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:53.099769  9968 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.106979  6669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:17:53.124637  6669 raft_consensus.cc:606] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:53.098345  9944 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.114032  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:53.129503  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.088608  9517 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.106298  9912 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.130990  6662 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:53.131940  9915 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.134135  9845 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:53.099809  9928 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.135730  9932 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:53.144222  9929 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:53.145730  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.146044  6520 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:53.149109  9661 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.224666  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.231854  7236 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.232074  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.232617  7230 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:53.150409  6669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:17:53.157296  9972 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:53.157644  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 43677us
I20260709 13:17:53.238435  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.213197  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.216979  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:53.224678  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.218997  7235 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:53.223994  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.223905  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.225344  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.155861  9778 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:53.227355  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.233451  6669 raft_consensus.cc:606] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:53.150883  9896 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.146088  9898 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.239500  7237 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 3.
I20260709 13:17:53.216609  9932 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:53.240511  7232 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.241935  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:53.242530  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.242849  7229 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:53.256925  7235 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:53.251936  7231 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:53.258131  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.247045  6669 raft_consensus.cc:614] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:53.247987  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.243768  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.260147  6590 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:53.260144  6666 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:53.220542  9688 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.247314  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:53.262902  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:53.238878  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:17:53.247615  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 144835us
I20260709 13:17:53.254798  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.220927  9944 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:17:53.239183  6667 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.242866  7233 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.258913  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:53.243239  6517 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:53.252985  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 178895us
I20260709 13:17:53.265934  7237 raft_consensus.cc:2376] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9. Re-sending same reply.
I20260709 13:17:53.271368  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.272054  7237 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:53.219669  9929 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:53.267164  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.279659  6665 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.279740  9929 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.281831  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:53.271658  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:53.242383  9658 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:17:53.269352  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.283691  7234 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:17:53.283942  9658 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:53.284276  9658 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:53.242111  9886 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:53.284489  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:53.284808  9886 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:53.272078  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.285259  9658 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.284562  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.268405  9799 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:17:53.286844  7234 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.285228  9886 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.288106  9661 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.271883  9917 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:17:53.242246  9635 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:17:53.290868  9661 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:53.291285  9661 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:53.263468  6669 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:53.296213  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:53.296815  6591 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:53.276631  7230 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:53.250499  9972 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.277988  9896 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.278712  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.271553  9854 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:17:53.283051  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:53.284332  9912 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.285771  9517 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:53.296967  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.272244  9688 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.267372  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:17:53.297026  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49793.8us
I20260709 13:17:53.299821  6594 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:53.247924  9921 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.300261  7232 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:53.301294  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.323033  7228 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:53.292189  9661 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.301308  9968 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:53.333087  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.301959  9778 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:53.308192  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:53.303165  7237 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:53.336509  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.337677  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.338160  7237 raft_consensus.cc:2376] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 8. Re-sending same reply.
I20260709 13:17:53.338945  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.339355  9886 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.340181  7237 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:17:53.312656  6667 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:53.313282  7236 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:53.269035  9931 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:53.342420  9931 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:53.343293  9822 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:53.343482  9931 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.344424  9822 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:53.344797  9822 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:53.345615  9822 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.286574  9658 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:53.322350  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 83560.1us
I20260709 13:17:53.293568  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.324146  7229 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:53.328231  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:53.325591  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58257.7us
I20260709 13:17:53.329442  9688 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.329979  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.330271  6597 raft_consensus.cc:2410] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:53.331326  9517 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:17:53.331629  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.331665  9635 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:53.333096  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:53.306730  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.334868  7232 raft_consensus.cc:2410] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:53.347808  7236 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:53.360433  9822 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.348335  7230 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:53.361291  9973 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.557s	user 0.004s	sys 0.007s
I20260709 13:17:53.362517  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2356) took 1062 ms. Trace:
I20260709 13:17:53.362912  6514 rpcz_store.cc:276] 0709 13:17:52.300418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.325001 (+ 24583us) service_pool.cc:224] Handling call
0709 13:17:52.415996 (+ 90995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:52.460099 (+ 44103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.460119 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:52.532716 (+ 72597us) log.cc:844] Serialized 26 byte log entry
0709 13:17:52.703003 (+170287us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:52.703039 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.703108 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:53.361016 (+657908us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b2400118cb8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5e1a9c4 00007f04b5e18acb 00007f04b5fd3dc9 00007f04b5fd3a55 00007f04b5fd309d 00007f04b8c0510e 00007f04b8c1c88c 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221
0709 13:17:53.362248 (+  1232us) raft_consensus.cc:1685] finished
0709 13:17:53.362272 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.362461 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13588,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":126,"spinlock_wait_cycles":52855040,"thread_start_us":161062,"threads_started":1,"wal-append.queue_time_us":206037}
I20260709 13:17:53.365172  9898 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.303651  9867 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:17:53.346998  9968 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:53.350589  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:53.353140  6663 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:53.353716  7233 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:53.328394  9912 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:53.314272  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:53.375205  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:53.375344  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:53.371520  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:53.372036  7228 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:53.348686  7234 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:53.375881  6520 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:53.376860  7237 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:53.375948  6514 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.377480  9989 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:17:53.377750  9993 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:53.378116  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:53.381008  9635 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:53.381544  9635 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:53.377779  9898 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:53.383018  9898 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:53.384202  6514 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:53.383916  9898 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.384347  9972 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:53.370940  9812 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:17:53.385953  9972 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:53.386319  9972 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:53.386535  9968 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.387406  9972 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.388652  9972 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.389873  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.390602  9778 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:53.391726  9778 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:53.392136  9778 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:53.393159  9778 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.368732  9661 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:53.397491  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 83309.6us
I20260709 13:17:53.396664  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:53.393785  9915 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:53.333624  9921 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.403565  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:53.404312  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.405447  9611 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:53.405787  9932 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.407507  9932 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:53.407953  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.410079  9927 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:53.410571  9845 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.411311  9845 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.413133  9985 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:17:53.378343  9993 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.413770  9845 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.410535  9927 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:53.415385  9517 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:53.414989  9927 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.415836  9517 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:53.416244  9517 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:53.416345  9927 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.421290  9929 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.401693  9688 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:17:53.423342  9688 consensus_queue.cc:1243] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:53.423868  9688 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:53.424178  9688 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:53.409063  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:53.404685  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.426841  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:53.427374  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.427986  6667 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.428280  6665 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:53.428139  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.428989  6668 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:53.429486  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.430986  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:53.437896  6667 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:17:53.394609  9778 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:53.416198  9992 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 13:17:53.441202  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 66134.3us
I20260709 13:17:53.442653  9867 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.443615  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:53.400655  9915 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:53.446393  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:53.446762  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.459998  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.460879  7234 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:53.462420  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.464358  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.465010  7236 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:17:53.468364  6668 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:53.469058  9915 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.460604  9989 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:53.431705  9886 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.440181  9966 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:53.432425  9899 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:53.472983  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.433933  9845 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:53.474357  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:53.471335  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.482201  7235 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:53.479977  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.472889  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.475390  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.410192  9943 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:17:53.440454  9775 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:53.432440  9812 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:53.441610  9854 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.479172  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:53.480871  7230 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:53.483670  7231 raft_consensus.cc:2410] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 10 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:53.485750  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.421463  9928 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.486665  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:53.487931  9918 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:53.462292  9993 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.397166  9635 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.440725  9611 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:17:53.454025  9896 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:17:53.491497  9909 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:53.509069  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.512192  7234 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:17:53.530589  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56303.2us
I20260709 13:17:53.486562  9886 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:53.476787  9925 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:53.537820  7232 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:53.541203  7230 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:53.461207  9867 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.555992  9886 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:53.444868  9688 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.576401  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 89853.1us
I20260709 13:17:53.577208  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.488952  9931 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
W20260709 13:17:53.588482  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 100464us
I20260709 13:17:53.547099  9917 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:53.571678  9994 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:17:53.527069  9854 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.559031  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 291988us
I20260709 13:17:53.591346  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:53.494230  9899 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:53.544152  9909 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:53.560810  9972 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:53.555204  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 108561us
I20260709 13:17:53.526820  9812 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.572319  9635 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.603256  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:53.605443  9812 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.607404  9845 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:53.608927  9802 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:53.578117  9966 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:53.584054  9886 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.578485  9925 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:53.570576  9989 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:53.596421  9714 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.600476  9909 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:53.599406  9917 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:53.612277  9688 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:53.609463  9802 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.623944  9968 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:17:53.651055  9921 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:53.705247  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.630954  9896 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:17:53.626775  9957 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:17:53.626843  9790 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:17:53.661351  9918 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:53.683718 10000 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:17:53.683104  9998 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:17:53.682463 10003 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:17:53.735988  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:53.682560  9996 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.376s	user 0.001s	sys 0.001s
I20260709 13:17:53.686455  9802 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.662345 10004 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.686803  9966 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.671705  9899 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.814479  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.797717  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2182) took 1697 ms. Trace:
I20260709 13:17:53.631125  9714 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.851725  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.795859  9854 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:53.869287  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.853132  9989 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.732013  9634 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.795378  9611 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:53.852226  6671 rpcz_store.cc:276] 0709 13:17:52.100223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.255754 (+155531us) service_pool.cc:224] Handling call
0709 13:17:52.256103 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:52.259464 (+  3361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.259498 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.424105 (+1164607us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b18001977a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a
0709 13:17:53.424628 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:17:53.656259 (+231631us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:53.656294 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.656361 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:53.797416 (+141055us) raft_consensus.cc:1685] finished
0709 13:17:53.797441 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.797652 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":435,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":3271,"spinlock_wait_cycles":2957056,"thread_start_us":231192,"threads_started":1,"wal-append.queue_time_us":305582}
W20260709 13:17:53.878111  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 286811us
I20260709 13:17:53.795138  9928 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.805090  9635 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:53.890184  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:53.874977  9658 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.853191  9918 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:53.810493  9915 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.904663  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 299880us
W20260709 13:17:53.904897  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 281573us
I20260709 13:17:53.906738  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:53.883783  9917 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.909813  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 764) took 1749 ms (client timeout 2127 ms). Trace:
W20260709 13:17:53.910367  6597 rpcz_store.cc:269] 0709 13:17:52.160713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.161175 (+   462us) service_pool.cc:224] Handling call
0709 13:17:53.909614 (+1748439us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002fc1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:53.909756 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2662179712}
I20260709 13:17:53.857950 10004 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:53.912004  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.906762  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:53.920840  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 343667us
I20260709 13:17:53.883836  9611 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:53.913267  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 773) took 1040 ms. Trace:
I20260709 13:17:53.903937  9972 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:53.925149  6589 rpcz_store.cc:276] 0709 13:17:52.872761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.996958 (+124197us) service_pool.cc:224] Handling call
0709 13:17:53.006754 (+  9796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.011202 (+  4448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.011224 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.244771 (+233547us) spinlock_profiling.cc:243] Waited 7.01 ms on lock 0x7b2c0012b1d8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:17:53.245180 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 13:17:53.430408 (+185228us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:53.430460 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.430549 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:53.904869 (+474320us) raft_consensus.cc:1685] finished
0709 13:17:53.904891 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.913220 (+  8329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8122,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"spinlock_wait_cycles":16132864,"thread_start_us":147459,"threads_started":1,"wal-append.queue_time_us":199849}
I20260709 13:17:53.928483  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:53.931622  6520 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:53.894711  9918 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.926900 10008 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:53.884240  9634 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:53.924840  9822 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.564s	user 0.006s	sys 0.005s
I20260709 13:17:53.798727  9925 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.939626  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:17:53.934511  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.904315  9658 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:53.940742  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.953981  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:53.954591  6671 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.956097  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:53.941072 10018 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:53.929539  9836 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.947839  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:53.950032  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:53.932806  9972 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:53.953680  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:53.985453  6596 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:17:53.944617  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.986198  6597 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:53.901634  9635 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:53.957672  6592 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:53.957741  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:53.937652  9867 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:53.944818  9714 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:53.986013  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:53.956888  6667 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:53.996315  6591 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:53.942607  9530 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:53.959472  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.005537  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.936196  9801 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:53.995437  6519 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:53.993678  7200 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:53.944796  9921 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:54.006259  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:54.008684  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 60328.7us
I20260709 13:17:54.010269  6671 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:53.985961  9972 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.015084  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:54.015918  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61020.9us
I20260709 13:17:54.015858  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.016657  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.017163  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:54.000478  9932 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.006s
I20260709 13:17:54.019284  6588 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:53.990190  9957 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.022934  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.023566  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:54.024413  7235 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:53.998289 10005 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.990742 10018 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:53.958148  9928 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:17:54.035750  6513 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:17:54.023527  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 75762.4us
I20260709 13:17:54.026324  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:54.007598  9867 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:54.041755  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:54.042426  7230 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:54.025367  9944 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:54.022501  9989 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:54.042400  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:53.958647  9611 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.059751  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:54.045941 10021 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 13:17:54.045595  9517 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.629s	user 0.006s	sys 0.006s
I20260709 13:17:54.060462  6513 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:54.065265  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:54.066068  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.043802  9867 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:54.030821  9957 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.045320  9932 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:17:54.035414 10005 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.064978  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:54.058187  9801 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:54.058338  9989 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:54.068941  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 59767.1us
I20260709 13:17:54.142272  6597 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
W20260709 13:17:54.233393  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 168498us
I20260709 13:17:54.108855  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:54.236361  7233 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
W20260709 13:17:54.157110  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.238489  6597 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:54.059332  9944 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.174942  6521 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:54.056540 10004 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.236488  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:17:54.100056 10024 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:17:54.068307  9949 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:54.087692  9915 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:54.107530  9929 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.000s
W20260709 13:17:54.101500 10029 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:17:54.175995  9898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.003s	sys 0.015s
W20260709 13:17:54.262768  9898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.006s	sys 0.021s
I20260709 13:17:54.069802  9517 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.265641  9836 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:54.253353  9688 consensus_meta.cc:287] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.001s
I20260709 13:17:54.176846 10041 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:17:54.267602  9517 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.102377 10018 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.068303 10032 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.207s
I20260709 13:17:54.270977  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2177) took 2173 ms. Trace:
I20260709 13:17:54.068177  9611 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:54.100097 10019 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:17:54.251328  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:54.100752 10028 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:17:54.124697 10037 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:54.105484  9778 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.003s
I20260709 13:17:54.230609  9867 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:54.059469  9886 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.101944  9635 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.107712  9634 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.094468 10014 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.329s
I20260709 13:17:54.254411  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:54.317214  9944 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.274766  9929 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:54.252650  9925 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:54.318693  9929 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.107218  9658 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.240662  9989 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.319717  9929 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.322926  9898 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:54.313336  9915 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.324033  9915 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.326198  9915 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.235152  9917 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:54.232296  9801 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:54.272537  6518 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:17:54.276443 10046 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:17:54.276530  9918 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.277132 10036 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.277561 10013 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:54.278208  9735 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:54.279331  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.271132  9795 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:54.279431  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:54.341055  7233 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.100203  9909 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.096251 10017 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.328s
W20260709 13:17:54.286682  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.084826 10023 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:54.347069 10023 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:54.315495  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 250308us
W20260709 13:17:54.340523  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 104086us
I20260709 13:17:54.315721  9949 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.314090  9790 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:54.318145  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63795.3us
I20260709 13:17:54.271381  6662 rpcz_store.cc:276] 0709 13:17:52.097532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.097696 (+   164us) service_pool.cc:224] Handling call
0709 13:17:52.098052 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.157176 (+1059124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.270720 (+1113544us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540011dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:54.270917 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2374552320}
I20260709 13:17:54.174512 10034 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:17:54.327000  9917 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.328913  9801 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.101914 10033 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.183s
I20260709 13:17:54.334215  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1061) took 1114 ms. Trace:
I20260709 13:17:54.336702  9984 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:17:54.275313  9972 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:54.340201 10012 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.012s	sys 0.039s
I20260709 13:17:54.390184  7229 rpcz_store.cc:276] 0709 13:17:53.220249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.285963 (+ 65714us) service_pool.cc:224] Handling call
0709 13:17:53.323990 (+ 38027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.356783 (+ 32793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.334071 (+977288us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54000ba750. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:54.334173 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86746496}
I20260709 13:17:54.353631  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:54.356535  7235 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:54.368383  7233 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:17:54.349200  6669 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:54.361790  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.388387  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:54.359004  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1007) took 1113 ms. Trace:
I20260709 13:17:54.387809  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:54.349640  9987 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.017s	user 0.002s	sys 0.000s
I20260709 13:17:54.397030  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:54.330761  9989 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.389183  9915 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:54.354045 10014 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.398219  9646 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:54.341224 10013 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.405052 10050 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:54.398331  9794 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:54.406039  9955 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:17:54.407830  9921 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.412015  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1066) took 1190 ms. Trace:
I20260709 13:17:54.411896  9921 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.412432  7228 rpcz_store.cc:276] 0709 13:17:53.221699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.330391 (+108692us) service_pool.cc:224] Handling call
0709 13:17:53.371870 (+ 41479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.400343 (+ 28473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.411821 (+1011478us) spinlock_profiling.cc:243] Waited 41.1 ms on lock 0x7b54005195d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:54.411957 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94591360}
I20260709 13:17:54.408946  9918 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:54.417110  9921 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.399755  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:54.399922  6597 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:54.389940  9996 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:17:54.434118 10025 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.436s	user 0.000s	sys 0.000s
W20260709 13:17:54.435019 10042 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:17:54.349465  9909 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.437022 10005 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.438553  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.439198  6667 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:54.439595  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.440824  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.441537  6671 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.401147  7237 rpcz_store.cc:276] 0709 13:17:53.245168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.340945 (+ 95777us) service_pool.cc:224] Handling call
0709 13:17:53.376634 (+ 35689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.396570 (+ 19936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.358772 (+962202us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b540051d1d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:54.358936 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81338752}
I20260709 13:17:54.386546  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:54.340785  9795 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:54.402496  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.454805  6595 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
W20260709 13:17:54.396684 10012 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.015s	sys 0.040s
I20260709 13:17:54.407423 10013 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.395190  9790 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.423470  9802 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:54.389467  9917 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:54.424777  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.389794  9926 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:17:54.425041  9972 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.400884  9931 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:17:54.418196 10032 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:17:54.425909  9957 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.417975 10037 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:54.427208 10052 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:17:54.427667  9989 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.350236 10023 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.352321  9867 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.476028  6671 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
W20260709 13:17:54.495842  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 109350us
I20260709 13:17:54.486056  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:17:54.496729  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 97044.7us
I20260709 13:17:54.450764 10014 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.501721  9795 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.445518  9886 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.502743  9790 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.502691  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.455549 10050 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:54.506282 10001 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.885s	user 0.000s	sys 0.001s
I20260709 13:17:54.342736 10036 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.500555 10005 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:54.500938  9802 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:54.534722  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.536540  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.554296  6662 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:54.554425  6668 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:17:54.477231  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.537201  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.555336  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:54.555868  7233 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:54.556128  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.556821  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:54.537103  7230 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:17:54.507099 10049 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.541077  6671 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:54.543083 10032 consensus_queue.cc:1243] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:54.546028  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.547724  6595 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:17:54.548210  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45639us
I20260709 13:17:54.561764  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.549437  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.562723  6514 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:17:54.550536  9997 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.184s	user 0.002s	sys 0.000s
I20260709 13:17:54.551730  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.553073  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:54.537645  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.567835  6513 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:54.568164  6666 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.476869  9949 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.556922  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.559361  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.560493  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.560608  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.563838  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.563935  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.562503  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.576803  6662 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:54.546869  9989 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:54.568003  7237 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:54.540433  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.550863 10050 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.560854  9972 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.567142  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:54.557526  6520 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.561254  9886 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:54.533529 10037 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.570377  9802 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.571117  9867 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.572413  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.596485  6663 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:54.550884 10026 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:54.573881  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 1.20353e+06us
I20260709 13:17:54.552949  9944 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:54.574787  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.580845  6664 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:54.561275 10018 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:54.582657  9658 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:54.568846  7228 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:17:54.559535  9993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.015s	sys 0.024s
I20260709 13:17:54.582118  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.587604  6513 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:54.584456  9989 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.591602  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.639724  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.599640  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.602150  9926 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.602870  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.603499  6593 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:17:54.603940 10014 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.583031  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.660560  6671 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:54.617408  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:54.632930  9993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.015s	sys 0.024s
I20260709 13:17:54.618129  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.582214 10013 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:54.613085  6666 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:54.561041 10005 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.643939  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.606506  9804 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:54.652577  6663 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:54.657142  6520 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:17:54.611337  9912 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:54.617957  9944 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:54.662322  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44994.8us
I20260709 13:17:54.677917  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:54.582340  9966 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:54.615748  9886 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:54.620877 10018 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.632189  9658 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.670571  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.634227  9873 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.667640  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:54.675434  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.596876  9972 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.676854  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:54.678311  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.720799  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:54.721064  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44297.2us
I20260709 13:17:54.721558  6666 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:54.681414  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.684077  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:54.712671  6514 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:54.714512  6663 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:54.718775  7228 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:54.722481  6666 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:54.675830  9804 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:54.711850  9955 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:54.714653 10018 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.681777  9944 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.607206 10026 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.710924  9932 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:54.734606  6667 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:54.661106  9926 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.677394  9912 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.711112  9909 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:54.717829  9658 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:54.679302 10013 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.730413  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 622) took 3357 ms. Trace:
I20260709 13:17:54.673538 10023 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
W20260709 13:17:54.740351  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 56353.4us
I20260709 13:17:54.758266  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:54.717980  9928 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.720103 10033 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.762152 10008 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:54.757339  6670 rpcz_store.cc:276] 0709 13:17:51.372950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.383958 (+ 11008us) service_pool.cc:224] Handling call
0709 13:17:51.384828 (+   870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:51.489524 (+104696us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:51.567151 (+ 77627us) log.cc:844] Serialized 14 byte log entry
0709 13:17:53.142951 (+1575800us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b18003ecf20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:17:53.256423 (+113472us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.270361 (+ 13938us) log.cc:844] Serialized 26 byte log entry
0709 13:17:53.270589 (+   228us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:53.270649 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.270711 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.516256 (+1245545us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:54.730162 (+213906us) raft_consensus.cc:1685] finished
0709 13:17:54.730189 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.730345 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36513,"prepare.queue_time_us":155439,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":41469,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":14796,"spinlock_wait_cycles":60809856,"thread_start_us":250304,"threads_started":3,"wal-append.queue_time_us":1749412}
I20260709 13:17:54.723402  9966 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.726853  9775 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:54.749215  9909 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.745297  9634 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:54.753613  9932 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:54.753115 10005 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.754181  9926 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:54.754006 10047 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:54.728860  9635 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.754681  6671 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:17:54.755087  9912 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.714264  9984 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:17:54.748071  9804 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.713605  9801 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.744968 10032 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:54.723846  9873 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.769136  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.773352  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.821174  6597 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:54.821810  6667 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:17:54.796094  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:54.823427  6592 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.796201  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.825917  6597 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:17:54.826431  6597 pending_rounds.cc:85] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 7
I20260709 13:17:54.826854  6597 pending_rounds.cc:107] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:54.828398  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:54.832979  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:54.828341 10013 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.829926  6662 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:17:54.840481  6662 pending_rounds.cc:85] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 3
I20260709 13:17:54.840766  6662 pending_rounds.cc:107] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:17:54.841084  6662 raft_consensus.cc:2889] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:54.830175  9635 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.833146  6670 raft_consensus.cc:2376] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8. Re-sending same reply.
I20260709 13:17:54.844722  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.833731  6597 raft_consensus.cc:2889] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:54.849879  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.850008  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.850487  6670 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:54.850601  6667 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:54.852365  9801 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.832011  9909 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:54.854864  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.768751  9966 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.768626 10008 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.855876  6520 raft_consensus.cc:1217] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Deduplicated request from leader. Original: 1.1->[3.2-5.3]   Dedup: 5.3->[]
I20260709 13:17:54.856447  6520 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:17:54.857769  6520 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:54.856336  6637 connection.cc:256] RPC call timeout handler was delayed by -1.6265748729999814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 13:17:54.861934  6637 connection.cc:256] RPC call timeout handler was delayed by -1.6175748729999668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:17:54.862237  6637 connection.cc:256] RPC call timeout handler was delayed by -1.3700748730000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
I20260709 13:17:54.863075  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:54.864920  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.849300  9688 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:17:54.865779 10057 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:54.866415  9688 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:54.866828 10026 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:54.803368 10033 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.866312  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.820569  9634 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.868489  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.772955  9949 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:54.870975  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.875408  6520 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:54.880162  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.843875  9989 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.885368  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.887480  9993 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:54.888407  9993 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:54.888729  9993 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:54.890769  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:54.891700  6589 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:54.893381  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.768193  9928 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.895392  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:54.849291  9917 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:54.895272 10026 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.899143  9917 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.899264 10026 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.870119  9867 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:54.901546  9867 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.901901 10026 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:54.904690  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2337) took 2239 ms (client timeout 1895 ms). Trace:
W20260709 13:17:54.905201  6594 rpcz_store.cc:269] 0709 13:17:52.665314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.665447 (+   133us) service_pool.cc:224] Handling call
0709 13:17:54.904481 (+2239034us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54002f71d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:54.904608 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228457216}
I20260709 13:17:54.906430  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.906869  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:54.801465  9886 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.908172  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:54.908633  7232 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:17:54.909134  7229 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:54.910002  7228 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:17:54.910509  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.897217  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:54.911501  6596 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.914028  6594 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:54.915053  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.866858  9688 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:54.876307  9634 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.889519  9993 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.917343  9799 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:54.917649  6594 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:54.921784 10018 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.922935  9932 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.924899  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.932387  6521 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:54.934762  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:54.927194  9988 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.618s	user 0.000s	sys 0.004s
I20260709 13:17:54.874313 10032 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:54.939033  6592 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
W20260709 13:17:54.938215  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 198357us
I20260709 13:17:54.940127  6594 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:17:54.943254  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.930465  9799 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:54.941072  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:54.949198  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.896384  9949 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 13:17:54.954151  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.954362  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:54.911763 10037 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:54.961263  6596 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:54.962571  6592 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:54.945662  9993 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.941383  9972 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:54.965296 10064 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:17:54.966403  6561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.988s (SENT) [suppressed 28 similar messages]
I20260709 13:17:54.962582  9688 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:54.953742  9611 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.971542  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:55.115134  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2200) took 1166 ms. Trace:
I20260709 13:17:54.978070  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:54.991427  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.118474  6669 rpcz_store.cc:276] 0709 13:17:53.949039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.949166 (+   127us) service_pool.cc:224] Handling call
0709 13:17:53.987431 (+ 38265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.395648 (+408217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.114775 (+719127us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54000cbf54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:55.115040 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831583104}
I20260709 13:17:55.119518  7229 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:55.118916  6596 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:55.116736  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:17:55.127570  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.127404  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:55.137035  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.154913  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.142529 10056 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.155699  6596 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:55.159533  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:55.091657 10067 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:17:55.002228  9972 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:55.162657  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.166368  6596 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:55.126543  9688 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.126137  9780 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:55.173243  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 153479us
I20260709 13:17:55.175179  6596 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:55.126703  9611 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.190732  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:55.191691  6516 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:55.170814  9775 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:55.189441 10035 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.175271  9928 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:55.205837  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.207320  6514 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:55.212317  6514 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:55.208853  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2003) took 1296 ms. Trace:
I20260709 13:17:55.162358 10056 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.168802  9972 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.213603  6519 rpcz_store.cc:276] 0709 13:17:53.912301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.912456 (+   155us) service_pool.cc:224] Handling call
0709 13:17:53.912842 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.007962 (+ 95120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.208496 (+1200534us) spinlock_profiling.cc:243] Waited 82.4 ms on lock 0x7b5400044554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:55.208754 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189555456}
I20260709 13:17:55.217511  6514 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:55.220083  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 103435us
I20260709 13:17:55.221259  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.223266  7237 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:55.223398  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:55.223655  7237 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:55.222466  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.237998  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.222764  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:55.193511 10018 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.183980  9780 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.244225  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.212150 10033 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.216454  9944 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:55.250463  7229 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:55.224541  7237 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.275684  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.245352  9775 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.276683  7237 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:55.280268  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56951.4us
I20260709 13:17:55.246985 10050 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:55.210843 10035 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.321329  6596 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:17:55.321174  9886 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:55.321156  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.277058 10071 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:17:55.307968 10018 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:55.378401  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57321.8us
W20260709 13:17:55.390512  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.392678 10080 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:17:55.393324  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.396085  6589 raft_consensus.cc:1217] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-5.3]   Dedup: 5.3->[]
I20260709 13:17:55.396651  6589 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [3.2-5.3]
W20260709 13:17:55.464058  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.465981  9886 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.395570 10018 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:55.466486  9886 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.467002 10018 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:55.468643  9886 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.469153  9886 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:55.469699  9886 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:55.470767 10082 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:17:55.471812  9780 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:55.473246 10072 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:17:55.474256  9917 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.320217 10032 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:55.324008 10084 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.379810 10050 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:55.466499  9944 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:55.395591  9929 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:17:55.477692  9910 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:55.478236  9966 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.001s
I20260709 13:17:55.491751  7237 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
W20260709 13:17:55.500097  6594 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.029s	sys 0.045s
I20260709 13:17:55.508705  7237 pending_rounds.cc:85] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 5
W20260709 13:17:55.508935  6594 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.030s	sys 0.045s
I20260709 13:17:55.509192  7237 pending_rounds.cc:107] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:17:55.509613  7237 raft_consensus.cc:2889] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:55.492951 10067 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 13:17:55.494421 10075 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
W20260709 13:17:55.511291  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 235657us
I20260709 13:17:55.469693  9972 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.494470 10023 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:55.505084 10077 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:17:55.521341 10086 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:55.521486 10056 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:55.528225  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.516717  9929 consensus_queue.cc:1243] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:17:55.515185 10032 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:55.531411 10049 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.961s	user 0.013s	sys 0.000s
W20260709 13:17:55.548163  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 226633us
I20260709 13:17:55.531781  9611 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:55.514858  9966 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.551416  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.533939 10035 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:55.500499 10084 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.539232 10032 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:55.556674  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 163417us
I20260709 13:17:55.559705  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.527617 10023 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.549142 10049 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:55.561338 10008 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.006s
W20260709 13:17:55.585875  9634 consensus_meta.cc:287] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.001s
I20260709 13:17:55.568089  9949 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:55.596949  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 126692us
I20260709 13:17:55.565129 10035 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:55.541615 10086 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.599908 10093 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:55.600801  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.601898 10023 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.607543  6514 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.615053  7236 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:55.596661  9929 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:55.598408 10073 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.588734 10032 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.586555  9634 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:55.631793  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.584127 10008 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:55.645064  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.647104  6486 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.648569  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.648648  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.650259  7229 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:55.652540  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.653190  7236 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:17:55.656490  6517 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:55.609745 10035 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.637799  9915 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.627020 10092 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:17:55.680130  9658 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.886s	user 0.012s	sys 0.000s
I20260709 13:17:55.680707  9658 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:55.649534  9944 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:55.655624 10032 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:55.632664  9929 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.741082  6514 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:17:55.733861  9899 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.611824 10093 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:55.744624  9913 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:17:55.744724  9899 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:55.750012  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 190709us
I20260709 13:17:55.750857  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.752990  9929 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.753574  9929 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:55.753930  9822 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:55.755378  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:17:55.757368  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.758175 10094 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:17:55.682076  9915 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.754056  9929 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.675735 10089 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:17:55.762707  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:55.765974  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:55.765587 10097 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:55.767483  9658 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:55.767267  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.635355 10073 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.767832  9822 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:55.763566 10056 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.771695 10087 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:55.771063 10088 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:55.786742  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:55.785331  9822 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:55.785482 10087 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:55.779752  9658 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:55.790776  9822 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:55.795042  9658 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:55.795481  9658 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:55.790686 10087 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:55.800410  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.806928 10086 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.807760 10084 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.807932  9929 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.788604 10088 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.808765 10088 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:55.810923  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 162875us
I20260709 13:17:55.811997  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.813757  6595 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:55.768082 10092 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:17:55.816731  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.817274  6595 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 4.
I20260709 13:17:55.817981  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.818480  6595 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:55.819183  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.819969  6595 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
W20260709 13:17:55.811205  9688 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.005s
I20260709 13:17:55.826761  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:55.788918 10056 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.832654  9530 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:55.829571  9658 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.839753  9530 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.841361  9658 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.841763  9530 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:55.842002  9915 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:55.843048 10100 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 13:17:55.843712 10018 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.845017 10039 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:55.845409 10039 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:55.845854 10018 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.830686  9634 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:55.834602  9778 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.850046  9899 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:55.849932  9778 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.851017  6665 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:17:55.851227  9799 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:55.852468  9799 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:55.854694  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.860042  6519 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:55.864135  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.866297  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.867095  7233 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:55.869462  6516 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:55.870292  6513 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:55.870409 10087 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:55.871315  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:55.872346  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2346) took 1037 ms. Trace:
I20260709 13:17:55.872697  6589 rpcz_store.cc:276] 0709 13:17:54.834854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.907703 (+ 72849us) service_pool.cc:224] Handling call
0709 13:17:55.395936 (+488233us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:55.401551 (+  5615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.872170 (+470619us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b540019e1d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:55.872291 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122019712}
I20260709 13:17:55.853363  9778 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:55.877550  7232 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:55.874018  9896 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.874176 10071 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.883189  9896 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.884208  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:55.884748  6515 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:55.885206 10089 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:17:55.888440  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.888967  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.889492  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.889760  6562 connection.cc:256] RPC call timeout handler was delayed by -1.077205339000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 13:17:55.890043  6562 connection.cc:256] RPC call timeout handler was delayed by -1.0448053390000496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
I20260709 13:17:55.863968  9929 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:55.891381  9929 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.892392  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.890758  9836 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:55.893028  7230 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:55.902904  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:55.846235 10039 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.903527  7233 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.906524  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:55.910661  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:55.912081  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:55.913275  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:55.914578  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.915099  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.914975  6515 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:55.916487  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.917744  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:55.872412 10099 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:55.902565  9836 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:55.902732  9899 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:55.942127  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.947939  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:55.948707  7237 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:55.948630  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:55.954473  7233 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:17:55.914372 10092 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:55.919018 10099 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.960460  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:55.909190  9822 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.972280  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:55.917205  9658 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:55.943246  9688 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:55.947466 10039 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:55.980611  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 108237us
I20260709 13:17:56.008428 10105 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.902412 10071 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.983090 10086 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:56.017253  9921 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:17:55.978806  9822 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:55.942698  9836 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.021054  9944 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:56.023804 10078 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.705s	user 0.001s	sys 0.000s
I20260709 13:17:55.970255  9899 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.025573  7235 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:17:56.015210 10056 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.019245 10008 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.041324  9867 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.062914  7232 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:55.979491 10073 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:56.051337  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.056612  9896 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:56.056703  6588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 782) took 2099 ms (client timeout 1500 ms). Trace:
I20260709 13:17:56.027019 10086 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.043045  9920 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:56.064960  6588 rpcz_store.cc:269] 0709 13:17:53.957364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.957496 (+   132us) service_pool.cc:224] Handling call
0709 13:17:56.056459 (+2098963us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5400080a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:56.056625 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59664768}
I20260709 13:17:56.010079 10105 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.067795  9944 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:17:56.068854  9899 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.070564  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2012) took 1196 ms. Trace:
I20260709 13:17:56.075632  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2204) took 1639 ms. Trace:
I20260709 13:17:56.075320 10088 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:56.076054  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.076690  9912 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.067716 10086 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.067673 10099 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.080575  6665 rpcz_store.cc:276] 0709 13:17:54.435808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.457717 (+ 21909us) service_pool.cc:224] Handling call
0709 13:17:56.075443 (+1617726us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54001cacd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:56.075543 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2980228352}
I20260709 13:17:56.060554 10093 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.079763  6521 rpcz_store.cc:276] 0709 13:17:54.873797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.933166 (+ 59369us) service_pool.cc:224] Handling call
0709 13:17:54.933594 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:54.934776 (+  1182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:54.934802 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:54.935215 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 13:17:55.173467 (+238252us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:55.173505 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.173569 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:56.023105 (+849536us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:17:56.070270 (+ 47165us) raft_consensus.cc:1685] finished
0709 13:17:56.070289 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:56.070505 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":666,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4971,"spinlock_wait_cycles":128864128,"thread_start_us":215941,"threads_started":1,"wal-append.queue_time_us":302059}
I20260709 13:17:56.132772 10041 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.066744  9822 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.452126  9836 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:56.452766 10105 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:56.312871  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.082556  9929 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.455636 10088 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:56.457155  6522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 635) took 2448 ms (client timeout 1894 ms). Trace:
I20260709 13:17:56.302168  9899 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:56.458895  6522 rpcz_store.cc:269] 0709 13:17:54.008286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.008545 (+   259us) service_pool.cc:224] Handling call
0709 13:17:56.456953 (+2448408us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540031fbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:56.457087 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985395968}
I20260709 13:17:56.464505  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:56.465114  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.466034  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.315255  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.472872  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.473016  9804 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:56.473332  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.474874  9929 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.475584  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.317041  9896 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:56.465973 10105 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:56.466467  9915 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.476428  9915 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.476493 10105 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.476871  9915 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.478582 10105 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.478945  9966 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.479014  6522 raft_consensus.cc:1217] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Deduplicated request from leader. Original: 1.4->[4.5-5.6]   Dedup: 4.5->[5.6-5.6]
I20260709 13:17:56.479691  6522 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [4.5-5.6]
I20260709 13:17:56.480203 10036 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.481751  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.482008  9955 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:56.482430  6522 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:56.483023  9795 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.482259  9896 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.078573  9949 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:56.484923  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.486137  6522 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:56.487718  9873 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.487670  9915 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.477011  9635 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.453632 10041 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.524852  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2212) took 1927 ms. Trace:
I20260709 13:17:56.525761  9915 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.525251  6664 rpcz_store.cc:276] 0709 13:17:54.597718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.721313 (+123595us) service_pool.cc:224] Handling call
0709 13:17:54.721543 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.721725 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:55.118523 (+396798us) log.cc:844] Serialized 14 byte log entry
0709 13:17:56.524482 (+1405959us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:56.524545 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:56.524608 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.524676 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:56.524798 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1401639,"prepare.queue_time_us":278961,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":1406008,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"thread_start_us":294771,"threads_started":1}
I20260709 13:17:56.527454  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.527520  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:56.528195 10041 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.528719 10041 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.528792  9735 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:56.529444  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.529213 10041 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:56.494807  9799 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.642s	user 0.013s	sys 0.000s
I20260709 13:17:56.497997  9802 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.499938 10032 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:56.501281  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:56.536826  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:56.538091  6664 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:56.503765  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 783) took 2544 ms. Trace:
W20260709 13:17:56.505828  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.506783  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.539173  6593 rpcz_store.cc:276] 0709 13:17:53.959171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.960040 (+   869us) service_pool.cc:224] Handling call
0709 13:17:54.000983 (+ 40943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:54.658043 (+657060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.503519 (+1845476us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54002fd5d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:56.503705 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384962816}
I20260709 13:17:56.540871  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.541127  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.507465  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.508819  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.538107  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:56.541997  6515 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:56.513125  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:54.317170  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:56.514132  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.543905  6513 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:56.517026  9873 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:56.546643  6669 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:56.543776  9611 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:17:56.548085 10026 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:56.548501 10026 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.548832 10026 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:56.544148  9931 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.549645  9931 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.539685 10036 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.550981 10036 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.551321 10036 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.551549  9291 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.552227  9291 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:56.518244  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.554826  9925 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.555661  6522 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:56.555439  9925 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.556248  9886 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.556550  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:56.475423  9929 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.556761  9886 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.557472  9925 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.557976  9925 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.558823  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.546289  9873 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.559156  9886 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.559357  9804 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.559635  6666 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:56.485253  9896 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:56.521793  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.523571 10086 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.560949  9873 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.528136  6516 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:56.561930 10086 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.482498  9955 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.563088  6662 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:56.459682  9899 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.528730  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:56.527539  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.532522  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2215) took 1890 ms. Trace:
I20260709 13:17:56.568128  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.530601  9915 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.545500  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.570043  6666 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:56.570398  6667 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:56.558936  9929 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.559903  9804 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.561146  6588 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:17:56.467854  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2101) took 1706 ms. Trace:
I20260709 13:17:56.561167  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:56.529970  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.574258  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45582.6us
W20260709 13:17:56.574878  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.574159  6590 rpcz_store.cc:276] 0709 13:17:54.761756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.769354 (+  7598us) service_pool.cc:224] Handling call
0709 13:17:56.058343 (+1288989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:56.058521 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.058543 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:56.058580 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:56.058636 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.058688 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:56.467698 (+409010us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400080a50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:56.467799 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2890483072}
W20260709 13:17:56.575500  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.575820  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.576215  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.564591  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.565114  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.577239  6665 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:56.551226  9778 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.571664  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.569417  9931 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.566373  9291 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.568375  6663 rpcz_store.cc:276] 0709 13:17:54.641865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.729299 (+ 87434us) service_pool.cc:224] Handling call
0709 13:17:55.116331 (+387032us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:17:55.116543 (+   212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:55.116735 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:55.116971 (+   236us) log.cc:844] Serialized 14 byte log entry
0709 13:17:56.520199 (+1403228us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b1800429ca0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:17:56.532064 (+ 11865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:56.532141 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:56.532202 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.532293 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:56.532440 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11061,"prepare.queue_time_us":3755,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":1407389,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":10991360}
I20260709 13:17:56.574369  6670 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:56.578994  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:56.580300  6666 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:56.580713  6593 raft_consensus.cc:2376] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8. Re-sending same reply.
I20260709 13:17:56.584061  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:56.558382  9925 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.584879  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.567231  9790 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:56.586117  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.601579  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:56.603207  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.578490  9812 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:56.601928  6662 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:56.612123  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:56.607221  9291 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:17:56.602669  9790 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.617482  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.618083  6662 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:56.619517  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.620179  6666 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:56.620615  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.621500  6670 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:56.621666  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.622771 10041 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.623505  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.620998  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.623850  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.624537  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.624701  9790 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:56.585583  9910 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:17:56.645613  9910 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:56.646104  9910 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:56.646427  9910 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:56.647269  9910 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.648474  9910 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:56.648699 10036 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.649914  9921 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.584702  9778 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.651057  9899 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.650723  9921 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.610002  9931 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:56.578455  9915 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.627328  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.659324  6667 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:56.643602  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.615439  9799 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.645009  9804 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.653807  9778 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.611487 10049 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.582926  9901 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:17:56.624563  6664 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:56.624712  6665 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:56.660410  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.624986  6662 raft_consensus.cc:2410] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:17:56.661164  9921 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:56.685055  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.685657  6665 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:56.685730  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.686326  6516 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
W20260709 13:17:56.686466  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 148453us
I20260709 13:17:56.687183  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:56.689661  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:56.681114  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:56.692046  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:56.662098  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.662153  9915 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.664716  9931 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:56.665760  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.666460  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.694252  6588 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:56.694391  6590 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:56.613057 10027 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:56.614395 10033 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.671020  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.695948  6520 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:56.673370  9799 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.676307  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:56.677486  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.678082  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.678077  6666 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.700397  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.680474  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.700976  6662 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.696774  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.672063  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.698868  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:56.672940  9912 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.700348  6589 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:56.675470  9972 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.698310  9804 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.698413 10027 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.702558  6519 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:56.711254  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:56.706262  9812 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:56.695815  9845 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:56.698644 10033 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.698788 10005 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.734472  9854 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.682961 10049 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.736054 10027 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:56.737360 10102 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.750s	user 0.003s	sys 0.009s
I20260709 13:17:56.700255  9896 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.712414 10041 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:17:56.712281  9778 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:17:56.702831  9921 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.739838  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1869) took 2200 ms. Trace:
I20260709 13:17:56.708086  9915 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.761511  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.759346  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.730724  6666 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:56.750963  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.710531  9912 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.752187  9925 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.812462  6591 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:17:56.803432  9778 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:56.813162  9778 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:56.813529  9896 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.813618  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2372) took 1954 ms. Trace:
I20260709 13:17:56.814036  6517 rpcz_store.cc:276] 0709 13:17:54.858942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.929047 (+ 70105us) service_pool.cc:224] Handling call
0709 13:17:54.938925 (+  9878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:55.657020 (+718095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.813372 (+1156352us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b5400044a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:56.813560 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649510656}
I20260709 13:17:56.805326  7228 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:56.714377  9972 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.815543  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.783234  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:56.810961  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.816814  6671 rpcz_store.cc:276] 0709 13:17:54.539390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.669510 (+130120us) service_pool.cc:224] Handling call
0709 13:17:56.739637 (+2070127us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b540020fdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:56.739777 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81317376}
I20260709 13:17:56.817499  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:56.817929  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.818820  6671 raft_consensus.cc:2410] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:17:56.742138 10033 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:56.777210  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.820667  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 144127us
I20260709 13:17:56.821223  6522 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 4: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:56.822002  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:56.819618  6670 raft_consensus.cc:2410] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:56.781910  9921 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.817036  6519 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:56.744225 10005 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.730070  6662 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:56.816202  6517 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:56.828944  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:56.829174 10005 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.829452  6517 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.703130  9799 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.814028  9896 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.823879  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:56.824291  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:56.824486  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:56.825419  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 92545.1us
I20260709 13:17:56.825690  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:56.826749  6665 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:17:56.834332  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:56.834841  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.827356  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.835628  6522 raft_consensus.cc:2410] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:56.750555 10035 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.823215  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:56.803037 10041 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:56.852396  6517 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:56.854202  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.834870  6516 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:56.820411 10027 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:56.864790 10027 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:56.865175  9780 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.865710  9780 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.865763 10027 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.819140  9925 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:56.752431  9931 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.867971  9780 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.869897  9845 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.870581  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.870436  9845 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.871371  6591 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:56.872342  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.872651  9845 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.872874  6591 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:56.873687  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:56.876056  6591 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.877338  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:56.878191  7233 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:17:56.877338  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.879110  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:56.879767  7232 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.882282  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:56.837220  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.837160  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.884804  6519 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:56.801287  9915 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:56.885546  9915 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.885838  9899 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.886336  9899 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.888339  9899 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.752365  9804 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:56.842348  9972 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.843575  9955 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.892771  9955 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.814365  9778 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.847761 10049 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.847898  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:17:56.893851  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.830224  9910 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:56.848615  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:56.894366  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.894639  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:56.894930  6594 raft_consensus.cc:2376] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6. Re-sending same reply.
I20260709 13:17:56.895212  6590 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:56.849064  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.895794  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:56.896051  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.896081  6589 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:56.896732  6590 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:56.896936  7235 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:56.850997  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.848110  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.898234  6595 raft_consensus.cc:2410] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:56.862541  6597 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:17:56.898355  6588 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:56.855370  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2108) took 2000 ms. Trace:
I20260709 13:17:56.854417  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:56.898980  6592 rpcz_store.cc:276] 0709 13:17:54.854766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.939756 (+ 84990us) service_pool.cc:224] Handling call
0709 13:17:54.962397 (+ 22641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.963177 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:56.855187 (+1892010us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b54002fd0d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:56.855314 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18194176}
I20260709 13:17:56.899492  6595 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:56.899688  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 45330us
I20260709 13:17:56.863533 10037 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:56.900200  9812 consensus_queue.cc:1175] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "921c423406da4f289884afca51ca198f" responder_term: 8 status { last_received { term: 8 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 8 index: 7 } }
I20260709 13:17:56.900815 10037 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.901203 10037 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.901572 10037 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.863953 10039 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:56.902032  6522 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:56.801414  9790 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.903136  6588 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:56.863162  9836 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:56.893816  9778 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.904803  9790 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:56.831218  9921 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:56.894376  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.901468  6592 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:56.906039  6593 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:56.907007  7236 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 5: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:56.907316  6592 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:56.895799  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:56.914980  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:56.915751  7237 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:17:56.922353  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:56.923151  7229 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.924147  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:56.924889  7235 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:56.854585  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:56.926349  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71817.4us
I20260709 13:17:56.928115  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:56.935122  6595 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:56.937664  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:56.938081  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.744760  9854 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:56.938922  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:56.940071  6595 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:56.851940 10041 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:56.940774  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:56.944515  9822 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:56.945102  9822 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:56.929618  9804 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:56.947481 10071 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.947223  9804 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.948264 10049 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.910156  9845 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.950732  6595 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:56.894641  9912 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:56.945969  9822 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:56.943608  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:56.952899  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:56.959775  6595 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:56.935220  9836 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.953166  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:56.948668 10049 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.941617  9812 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.964541  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.965206  6595 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:17:56.967036  9910 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.969390  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:56.977396  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:56.972040  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:56.979470  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:56.972550  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:56.980355  6595 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:56.982939  7232 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:56.941846  9854 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.938231 10026 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.028198  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.028939  6522 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:57.029729  7229 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:17:56.965184  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:56.972788  6589 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:56.986862  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:56.992820  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.993439  9802 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:56.973291 10039 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.960807  9845 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:56.974177 10073 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:57.022681  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.044836  6588 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 6: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:17:57.022909 10121 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:17:57.023252  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.973454  9812 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:56.964125  9836 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.986337  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:56.983527  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:56.975358 10049 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.031879  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:57.031985  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45201.4us
W20260709 13:17:57.032280  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.050436  6521 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:57.100346  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:56.990651  9790 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:56.990355 10088 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:57.116736  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:57.099644  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:57.102892  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.110358  7233 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:57.072052  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.115314  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:57.043478  6595 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:17:57.027154  9291 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:57.114326  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.124727  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:57.126140  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.126577  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.126995  6595 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:57.019762  9899 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.986037 10093 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.123780  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:57.133981  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:57.130988  6592 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.044755 10035 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:57.035574 10039 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.134533  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.131145  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.126134  9291 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:57.142156  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.154191  6595 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:57.158763  6595 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:17:57.026886  9931 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.096958  9802 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:57.101737  9812 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.157080  6589 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:57.108517  9836 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.160214  6595 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:57.120918  9790 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:57.131047  9972 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:57.131582  9795 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:57.239117  6588 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:17:57.138953  9611 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:57.237268  9915 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.045635  9873 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 13:17:57.123224 10088 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:57.156636  9925 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:57.153450  9780 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:57.155586 10035 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.252516 10112 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:17:57.253922 10110 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.552s	user 0.002s	sys 0.000s
I20260709 13:17:57.257917  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.260748  6664 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:57.240697  9790 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:57.263789  6669 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:57.356752  6669 pending_rounds.cc:85] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
I20260709 13:17:57.357077  6669 pending_rounds.cc:107] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:57.328573  9913 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.267433  9799 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.238288  9291 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.139987  9801 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:57.359119  6521 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:57.155050  9899 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.268774 10033 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.283623  9802 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.238761  9972 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.242602  9795 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.290009 10026 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.291669  9845 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:57.297127 10122 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
W20260709 13:17:57.298929 10113 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.362s	user 0.000s	sys 0.002s
I20260709 13:17:57.128608 10041 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.229622 10032 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
W20260709 13:17:57.303411  7228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 949) took 2082 ms (client timeout 1836 ms). Trace:
I20260709 13:17:57.304878  6665 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:57.305989  9921 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.379940  9972 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:57.313715  9915 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.314329  7234 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:17:57.318889  6417 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 249) took 27154 ms (client timeout 15000 ms). Trace:
I20260709 13:17:57.320050 10036 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:57.326233  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 425894us
I20260709 13:17:57.328701  6591 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:17:57.153685  9804 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.330374  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.479300  6520 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:57.332063  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.333148  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.334190  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:57.152684  6592 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
W20260709 13:17:57.480859  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 146738us
I20260709 13:17:57.331727  9925 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.348150  6418 catalog_manager.cc:5697] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 1 to 4, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 4 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.483880  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:57.349258  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.484419  6517 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.349332  9635 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:57.351178  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.154699 10086 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.175s
I20260709 13:17:57.261381  6662 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:17:57.507336  6662 pending_rounds.cc:85] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 4
I20260709 13:17:57.507679  6662 pending_rounds.cc:107] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:17:57.507957  6662 raft_consensus.cc:2889] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:57.357362  6669 raft_consensus.cc:2889] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:57.264415  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2375) took 2405 ms. Trace:
I20260709 13:17:57.332231 10088 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.155262 10073 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:57.158144 10039 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:57.362812  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 337092us
I20260709 13:17:57.375196  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.351963 10091 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.346477  9780 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.537992  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:57.376778  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 411660us
I20260709 13:17:57.377724  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.244063  9931 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.353184 10035 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.381561  6667 raft_consensus.cc:2368] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8.
W20260709 13:17:57.381909  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 398443us
I20260709 13:17:57.545966  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.548911  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:57.385602  6666 raft_consensus.cc:2368] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8.
I20260709 13:17:57.370767  9801 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.389859  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:57.390672  7228 rpcz_store.cc:269] 0709 13:17:55.220876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:55.220994 (+   118us) service_pool.cc:224] Handling call
0709 13:17:57.303236 (+2082242us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540049d254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:57.303341 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3640144768}
I20260709 13:17:57.297282 10037 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.406077  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.381946  9795 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:57.354907  9836 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.431324  9921 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:57.519491 10027 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:57.384608  9291 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.556265  9804 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:17:57.439205  9799 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.544512  9780 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.371080  9913 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.568456  9913 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:57.568986  9913 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.443034  9812 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.446923 10033 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.571905  9836 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.390610 10005 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:17:57.452484 10093 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:57.387324  9896 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.574916  9896 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:57.467716  9790 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.576674  9790 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.472908  9912 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:57.577392  9912 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.578936  9804 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.579525  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.579708  9912 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:57.509450  6514 rpcz_store.cc:276] 0709 13:17:54.859386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.918357 (+ 58971us) service_pool.cc:224] Handling call
0709 13:17:54.918651 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:55.602031 (+683380us) spinlock_profiling.cc:243] Waited 58.4 ms on lock 0x7b58005a71d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fdd0aa 00007f04b905f447 00007f04b9077fe7 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4
0709 13:17:55.741519 (+139488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.264190 (+1522671us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540014e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:57.264337 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797852416}
I20260709 13:17:57.480665  6516 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:17:57.491008  9925 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.581423  9931 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.535655  6517 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:57.519441  6418 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.544727  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:57.582563  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:57.405779  6417 rpcz_store.cc:269] 0709 13:17:30.164083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.184371 (+ 20288us) service_pool.cc:224] Handling call
0709 13:17:30.840868 (+656497us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8feb090 00007f04b9089f3a 00007f04bee401cc 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc 00007f04bd3fe392 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:31.469510 (+628642us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b5400002490. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5d640 00007f04bee34b76 00007f04bef3ec3e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc 00007f04bd3fe392 00007f04b73bc48c 00007f04b73bb8c6
0709 13:17:57.318778 (+25849268us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:51.935202 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:52.226427 (+291225us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:52.233795 (+  7368us) write_op.cc:435] Acquired schema lock
0709 13:17:52.235248 (+  1453us) tablet.cc:662] Decoding operations
0709 13:17:52.301455 (+ 66207us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:52.301536 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:52.301563 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:17:52.302665 (+  1102us) tablet.cc:701] Row locks acquired
0709 13:17:52.302719 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:17:52.302926 (+   207us) write_op.cc:270] Start()
0709 13:17:52.303199 (+   273us) write_op.cc:276] Timestamp: P: 1783603072302850 usec, L: 0
0709 13:17:52.303223 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:17:52.303957 (+   734us) log.cc:844] Serialized 4330 byte log entry
0709 13:17:52.583685 (+279728us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b180034d680. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:17:53.027451 (+443766us) op_driver.cc:464] REPLICATION: finished
0709 13:17:53.307323 (+279872us) write_op.cc:301] APPLY: starting
0709 13:17:53.313948 (+  6625us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:53.365759 (+ 51811us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:53.365801 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:53.596071 (+230270us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:53.664580 (+ 68509us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:17:53.664633 (+    53us) write_op.cc:312] APPLY: finished
0709 13:17:53.680680 (+ 16047us) log.cc:844] Serialized 83 byte log entry
0709 13:17:53.680876 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:53.681028 (+   152us) write_op.cc:454] Released schema lock
0709 13:17:53.681104 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:53.681526 (+   422us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:53.681555 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":787129,"reported_tablets":27,"spinlock_wait_cycles":1204546560,"child_traces":[["op",{"apply.queue_time_us":267345,"apply.run_cpu_time_us":6121,"apply.run_wall_time_us":374216,"num_ops":11,"prepare.queue_time_us":235581,"prepare.run_cpu_time_us":27056,"prepare.run_wall_time_us":524078,"raft.queue_time_us":402677,"raft.run_cpu_time_us":70549,"raft.run_wall_time_us":230896,"replication_time_us":705856,"spinlock_wait_cycles":62354176,"thread_start_us":647057,"threads_started":4,"wal-append.queue_time_us":163108,"wal-append.run_cpu_time_us":13586,"wal-append.run_wall_time_us":2217329}]]}
I20260709 13:17:57.476794  9972 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.481105 10026 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:57.437474  9899 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.579802  9804 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:57.536919 10073 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:57.584815  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.586562  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.523794  9635 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:57.588505  6418 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.590420  6668 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:57.556262  9801 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:57.579865 10126 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:57.545643  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 411694us
I20260709 13:17:57.572021  9812 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.584571 10127 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:57.610426  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 493892us
I20260709 13:17:57.587857 10026 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:57.618428  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.421669 10049 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.544344 10091 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.631537  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.570560  9913 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:17:57.610000 10129 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:57.636579  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:57.615024 10126 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.637364  6662 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:57.610044  9804 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:57.643874  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.629097 10086 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:57.646675  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:57.662315  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.633854  9775 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.675489  9775 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.676735  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.635254 10127 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:57.677331  6668 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:57.678021 10127 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.636569 10129 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:57.680616 10041 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.650688 10086 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.681299 10086 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.684706  6418 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 4 to 5, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 5 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.649906  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:57.640141  9920 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:57.668011  9804 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:57.687721  9804 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.688128  9804 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:57.688234  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:57.688830  9920 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:17:57.689282  9920 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.690846  9915 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:57.691094 10032 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:57.691294  9915 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:57.691748  9915 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.667912  9635 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.632982  9899 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:57.693943  9899 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.695550 10036 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:17:57.696056  9899 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:57.696146 10091 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.696534  9921 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:57.696934  9921 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:57.696993 10026 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.686508  6418 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 2 to 4, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 4 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.698076 10026 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.697846  9921 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.698778  6418 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.674782  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.674924  9896 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:57.678454  9910 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.671424  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.701191 10049 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.719249  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.701447  9845 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:57.702793  9972 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.691699 10032 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:57.704289  9921 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.704934  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.728540  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:57.705396  9804 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.705843  9780 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.706656  9795 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.707638  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.747516  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.625849 10039 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.708653  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.709316  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:17:57.754251  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.709528  7232 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:57.649003 10131 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:57.710280  9801 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:57.710951 10125 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:17:57.629925  9822 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:17:57.769428  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:57.674149  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:57.712131  6662 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:17:57.780202  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.678295  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.780979  6668 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:57.781754  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:57.782269  6668 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.782560  7231 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:57.714075  6418 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 3 to 5, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 5 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.716853  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.783565  7229 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:57.716629  9896 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.716809  9898 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.644999 10126 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.720021  7236 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:17:57.664529  9790 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:57.803028  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.803826  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.805218  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:57.806109  6588 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:57.806682  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.807462  7231 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:57.799680  9912 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.809259  9912 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.810504  9898 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.813580 10129 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.816383  9804 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.819023  9802 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:57.727141  9910 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.821241  9775 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:57.792882  9896 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:57.730577  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:57.853149  7235 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:57.717288  9854 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.002s
I20260709 13:17:57.733141  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.739724  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.754489  7233 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:57.664582  6667 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:17:57.862392  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.866660  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:57.867261  6589 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.867726  6667 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:57.872413  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.876323  6561 leader_election.cc:400] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8.
I20260709 13:17:57.876727  6561 leader_election.cc:403] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:17:57.875090  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.800232 10133 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.767253  7232 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:57.767729  7230 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:57.772311  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 187555us
I20260709 13:17:57.721772 10027 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.892869  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:57.775029  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.776134  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:57.784199  7237 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:57.784453  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.788275  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:57.789479  6665 raft_consensus.cc:2368] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
W20260709 13:17:57.895481  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 119428us
I20260709 13:17:57.896785  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:57.791287  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 224653us
I20260709 13:17:57.748766 10036 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:17:57.781983  9972 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.793846  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:17:57.795714  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.748358  9899 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.797837  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.761303 10049 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.802603  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:57.729272  6663 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:57.807505  6593 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: [4.3-7.4]
I20260709 13:17:57.850032  6418 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.886471  6668 raft_consensus.cc:2410] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:17:57.890175  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:57.890399  6596 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:57.790659  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 288678us
I20260709 13:17:57.913991  7236 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:17:57.897985  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:57.728467 10073 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.899720  6561 leader_election.cc:400] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8.
I20260709 13:17:57.882289  9910 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.858673 10037 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:57.944988  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:57.945763  6515 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:57.946209  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:57.946774  6518 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.906399 10036 consensus_queue.cc:1243] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:57.897115 10027 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:57.948863  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 809) took 1100 ms. Trace:
I20260709 13:17:57.904399  7235 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:57.904728  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.957769  6669 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:57.950677  6590 rpcz_store.cc:276] 0709 13:17:56.848910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.897371 (+ 48461us) service_pool.cc:224] Handling call
0709 13:17:56.897693 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:56.929734 (+ 32041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.929765 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:56.930364 (+   599us) log.cc:844] Serialized 52 byte log entry
0709 13:17:57.329632 (+399268us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:57.329675 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.329745 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:57.948650 (+618905us) raft_consensus.cc:1685] finished
0709 13:17:57.948675 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:57.948804 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31880,"raft.queue_time_us":32520,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":7614,"thread_start_us":355073,"threads_started":1,"wal-append.queue_time_us":645754}
I20260709 13:17:57.904906  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.958446  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.886119  9804 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:17:57.910506  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:57.911270  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.922725  6588 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:57.924345  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.967861  6668 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:57.968453  6589 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:57.968636  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.925124  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:57.969293  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:57.969856  6520 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:17:57.960356 10056 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:57.916179  6594 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.920879  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:17:57.975317  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 54550.3us
I20260709 13:17:57.976033  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.930964  6561 leader_election.cc:403] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:17:57.752904  9925 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:57.977267  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77664.5us
I20260709 13:17:57.978150  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.978765  9925 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:57.931850  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:57.979118 10132 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:57.979662  6593 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.981882  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:17:57.909703  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:57.982962  6670 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:57.935209  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 195575us
W20260709 13:17:57.958856  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 54031.4us
I20260709 13:17:57.875978  9854 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.962275  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:57.963055  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:57.913793 10093 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:57.915047  6664 raft_consensus.cc:2410] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:57.979871 10132 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:57.949663 10071 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:57.907744  9873 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8.
I20260709 13:17:57.992331  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:57.930681 10008 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.983830  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:17:58.040132  6596 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:57.923600 10143 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:57.929414  9780 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:58.056475  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:57.917654  9972 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:58.030591  9925 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:58.080693  6667 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:57.931213 10091 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
W20260709 13:17:58.096307  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.066900 10158 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:58.050810 10071 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.932674  9899 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:57.906685 10133 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:57.913697 10134 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.148s
I20260709 13:17:57.932518 10049 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:58.223819  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 241967us
W20260709 13:17:58.224733  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 430913us
W20260709 13:17:58.227517  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 249428us
I20260709 13:17:58.146727 10037 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:57.950981 10005 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 13:17:58.041589 10131 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:57.897329 10032 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.190958  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:58.191876  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:58.121547  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.040766 10139 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8.
I20260709 13:17:58.201225  9795 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:58.246749  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 271102us
I20260709 13:17:58.278594  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:58.282506  6596 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
W20260709 13:17:58.118916  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 135155us
I20260709 13:17:58.255766  6663 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:57.948773 10013 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.177s
I20260709 13:17:58.114674 10132 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:58.176235 10136 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:17:58.042151 10056 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 13:17:58.160801  6669 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:17:58.234717  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.240299  6594 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:17:58.200384 10071 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:58.242233  6590 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
W20260709 13:17:58.243501  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:58.246119  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 247073us
I20260709 13:17:58.206957  9873 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:58.225664  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:58.437465  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 211880us
I20260709 13:17:58.284780  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:58.450825  6588 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:17:58.299671  6668 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:17:58.302670  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.456502  6663 raft_consensus.cc:2368] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
W20260709 13:17:58.327440 10152 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.287s	user 0.000s	sys 0.000s
W20260709 13:17:58.334648  9921 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.594s	user 0.017s	sys 0.005s
I20260709 13:17:58.226459  9912 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:58.332190  9790 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.529s	user 0.036s	sys 0.015s
W20260709 13:17:58.472461  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 187729us
I20260709 13:17:58.245990  9801 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:58.473084  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.474694  6667 raft_consensus.cc:2410] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:17:58.335877 10041 consensus_meta.cc:287] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.655s	user 0.013s	sys 0.017s
I20260709 13:17:58.340072  9896 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.344195 10049 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:58.353452  9899 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:58.359594 10088 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.823s	user 0.005s	sys 0.019s
W20260709 13:17:58.367807  9915 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.675s	user 0.005s	sys 0.003s
I20260709 13:17:58.375474  9972 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:58.496029  6662 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:17:58.376161 10086 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.694s	user 0.015s	sys 0.010s
W20260709 13:17:58.378402 10135 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.558s	user 0.002s	sys 0.001s
W20260709 13:17:58.382884 10137 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.438s	user 0.000s	sys 0.001s
I20260709 13:17:58.225633  9898 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:58.396584  6593 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:58.559736  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:58.560472  6593 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:58.406941  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.564311  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:58.565167  7237 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:17:58.163651 10008 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.423377  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:58.464228  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.567127  7229 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:17:58.475374  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:58.483424  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:58.568367  6664 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:58.569124  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.569808  6664 raft_consensus.cc:2410] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:58.500424  9778 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.500672  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:58.243692 10091 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.249s
I20260709 13:17:58.513818  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.502005  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:58.452641 10005 consensus_queue.cc:1243] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:58.516300  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:17:58.530756  6486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.006s	sys 0.000s
W20260709 13:17:58.590422  6486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.006s	sys 0.000s
I20260709 13:17:58.467931 10131 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:58.465495 10158 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.437158  9873 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:58.540465  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.036s	sys 0.058s
I20260709 13:17:58.513366  9915 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.471438 10112 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:58.474268 10143 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:58.493822  9790 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:58.517697  9802 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:58.555209  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 810) took 1706 ms. Trace:
I20260709 13:17:58.463151  9845 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:58.562083  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:58.487646  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:17:58.650635  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 163036us
I20260709 13:17:58.578387  6667 raft_consensus.cc:2376] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 9. Re-sending same reply.
I20260709 13:17:58.652714  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.591987  9822 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:58.592566  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 169243us
W20260709 13:17:58.556090  9920 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.866s	user 0.007s	sys 0.012s
I20260709 13:17:58.495410 10155 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:17:58.595970  7233 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:17:58.599138  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.036s	sys 0.058s
I20260709 13:17:58.289322 10037 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.495452  9912 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:17:58.501716 10088 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:58.607582  6593 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:17:58.613346 10131 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.615624  9896 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:58.616273 10138 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.467s	user 0.001s	sys 0.000s
I20260709 13:17:58.592860  9886 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:58.601166  9873 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:58.620424  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.620585  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2132) took 1595 ms. Trace:
I20260709 13:17:58.600925 10158 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:58.589273 10049 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.606657  9913 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.292s
I20260709 13:17:58.898429 10049 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:58.898916  9913 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:58.899485  9913 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:58.899585  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:58.900221  6515 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:58.900671 10049 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:58.622061  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.900518  9913 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.901515  6663 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:58.901959  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 671) took 1122 ms. Trace:
I20260709 13:17:58.902259  6663 rpcz_store.cc:276] 0709 13:17:57.779744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.593887 (+814143us) service_pool.cc:224] Handling call
0709 13:17:58.901902 (+308015us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:58.904105  6668 raft_consensus.cc:2410] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:58.622370 10146 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:58.623008 10035 consensus_meta.cc:287] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.041s	user 0.011s	sys 0.000s
I20260709 13:17:58.573606  9972 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.624191  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:17:58.691071 10147 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:58.540100 10086 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.641549  6597 rpcz_store.cc:276] 0709 13:17:56.849195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.898862 (+ 49667us) service_pool.cc:224] Handling call
0709 13:17:56.899150 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:56.932162 (+ 33012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:56.932200 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:56.932735 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.295555 (+362820us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b1800469480. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:17:57.593616 (+298061us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:57.593670 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.593756 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:58.548776 (+955020us) raft_consensus.cc:1685] finished
0709 13:17:58.548797 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.555140 (+  6343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46088,"raft.queue_time_us":31264,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5614,"spinlock_wait_cycles":4501248,"thread_start_us":297879,"threads_started":1,"wal-append.queue_time_us":845025}
I20260709 13:17:58.651759  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:58.587472  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.689791  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.697414  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:58.701349  7235 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:17:58.702993  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:58.564756 10093 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:58.599004  9921 raft_consensus.cc:2764] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:17:58.958449  7237 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:17:58.592110  9778 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.640113 10143 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:58.745481  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1897) took 2102 ms. Trace:
I20260709 13:17:58.689265 10091 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.261s
W20260709 13:17:58.817349  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.797156  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:58.709880  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1905) took 1594 ms. Trace:
I20260709 13:17:58.704335  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:58.978937  6670 rpcz_store.cc:276] 0709 13:17:57.115969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.257665 (+141696us) service_pool.cc:224] Handling call
0709 13:17:58.709683 (+1452018us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54000cc954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:58.709825 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444867840}
I20260709 13:17:58.980415  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:17:58.649755  9802 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:58.827416  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:58.985543  6522 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:17:58.987810 10158 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:58.710285  6666 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:58.965788  9915 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:58.725063  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:58.836149  6669 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
W20260709 13:17:58.850346  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:58.991573  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 294323us
I20260709 13:17:58.649688  9635 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.902114  9913 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:58.825512  6595 rpcz_store.cc:276] 0709 13:17:57.024673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.233873 (+209200us) service_pool.cc:224] Handling call
0709 13:17:57.234212 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:57.585566 (+351354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:57.585607 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:57.586027 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.894078 (+308051us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:57.912547 (+ 18469us) log.cc:844] Serialized 14 byte log entry
0709 13:17:57.932209 (+ 19662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.932311 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:58.620306 (+687995us) raft_consensus.cc:1685] finished
0709 13:17:58.620336 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.620510 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19362,"prepare.queue_time_us":12217,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":140259,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":29257,"thread_start_us":616711,"threads_started":2,"wal-append.queue_time_us":1010495}
I20260709 13:17:59.012327  6595 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:59.013079  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2380) took 1150 ms. Trace:
I20260709 13:17:59.013476  6595 rpcz_store.cc:276] 0709 13:17:57.862777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.011741 (+1148964us) service_pool.cc:224] Handling call
0709 13:17:59.012112 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.012775 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.013014 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:58.918305  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:58.919127  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:58.659444 10112 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:58.688998 10005 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:58.924724  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:17:59.029596  6588 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:58.929445  6518 consensus_meta.cc:287] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.976s	user 0.010s	sys 0.000s
I20260709 13:17:58.933732  6662 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:59.032986  6518 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:59.033257  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 679) took 1251 ms. Trace:
I20260709 13:17:59.033772  6662 rpcz_store.cc:276] 0709 13:17:57.781704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.623932 (+842228us) service_pool.cc:224] Handling call
0709 13:17:59.033199 (+409267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:58.624775 10167 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.938017  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:58.650015 10073 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:17:58.948797  6515 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:17:58.839999 10026 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.141s	user 0.007s	sys 0.007s
I20260709 13:17:58.958542  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 669) took 1179 ms. Trace:
I20260709 13:17:58.963008 10147 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:58.925704 10027 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.933s	user 0.012s	sys 0.000s
I20260709 13:17:58.966907  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2229) took 1294 ms. Trace:
I20260709 13:17:58.967422  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1907) took 1081 ms. Trace:
W20260709 13:17:58.967845  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 316151us
I20260709 13:17:58.703207  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:58.970660  6671 rpcz_store.cc:276] 0709 13:17:56.642567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.834977 (+192410us) service_pool.cc:224] Handling call
0709 13:17:56.835257 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:56.915914 (+ 80657us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:56.938633 (+ 22719us) log.cc:844] Serialized 14 byte log entry
0709 13:17:57.321017 (+382384us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b180010b3a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:17:57.633121 (+312104us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:57.635085 (+  1964us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.635247 (+   162us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:57.635292 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.635409 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:58.703368 (+1067959us) raft_consensus.cc:1685] finished
0709 13:17:58.745219 (+ 41851us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.745399 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80314,"prepare.queue_time_us":22550,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":83578,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":17809,"spinlock_wait_cycles":3528448,"thread_start_us":310447,"threads_started":1,"wal-append.queue_time_us":947316}
I20260709 13:17:58.923882  9912 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.922724 10146 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:59.067620 10173 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:17:59.067353 10146 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.068383 10146 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:58.915864 10049 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:59.070878 10049 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.072365 10088 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:58.982237  6589 raft_consensus.cc:2368] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7.
I20260709 13:17:58.707996  9822 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.704298  9920 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:58.825467  9886 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:17:58.984503  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2230) took 1311 ms. Trace:
I20260709 13:17:58.926834  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:17:59.004418  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.029289 10005 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.015324  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 825) took 1153 ms. Trace:
I20260709 13:17:59.081153  6594 rpcz_store.cc:276] 0709 13:17:57.861716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.787078 (+925362us) service_pool.cc:224] Handling call
0709 13:17:59.015248 (+228170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:58.961388 10142 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:59.017449  6596 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:17:58.980448 10041 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.950553 10035 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.955924  9972 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:58.692610  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:59.028738 10013 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
W20260709 13:17:59.041373  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 103425us
I20260709 13:17:59.047662  6664 rpcz_store.cc:276] 0709 13:17:57.886463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.587144 (+700681us) service_pool.cc:224] Handling call
0709 13:17:58.967354 (+380210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.108824  6486 leader_election.cc:400] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
I20260709 13:17:59.109355  6486 leader_election.cc:403] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:59.050102  6671 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:17:59.047397  6663 rpcz_store.cc:276] 0709 13:17:57.672797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.922179 (+1249382us) service_pool.cc:224] Handling call
0709 13:17:58.966369 (+ 44190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.966568 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:58.966605 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:58.966667 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:58.966696 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.966751 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.966855 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.041143  6667 rpcz_store.cc:276] 0709 13:17:57.779225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.689530 (+910305us) service_pool.cc:224] Handling call
0709 13:17:58.958480 (+268950us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.110092  6671 pending_rounds.cc:85] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
I20260709 13:17:59.110514  6671 pending_rounds.cc:107] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:17:59.110854  6671 raft_consensus.cc:2889] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:59.111735  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2235) took 1437 ms. Trace:
I20260709 13:17:59.112144  6671 rpcz_store.cc:276] 0709 13:17:57.674354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.049524 (+1375170us) service_pool.cc:224] Handling call
0709 13:17:59.049790 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.111389 (+ 61599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.111663 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.056789  6662 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:59.114490  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2232) took 1441 ms. Trace:
I20260709 13:17:59.062861  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 675) took 1139 ms. Trace:
I20260709 13:17:59.063441 10145 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.064265 10037 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.047246 10167 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.070715 10093 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.075191  6670 rpcz_store.cc:276] 0709 13:17:57.673041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.981510 (+1308469us) service_pool.cc:224] Handling call
0709 13:17:58.984030 (+  2520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.984200 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:58.984225 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:58.984265 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:58.984297 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.984347 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.984452 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:58.839394  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:17:59.007184  6516 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.016s	user 0.014s	sys 0.004s
W20260709 13:17:59.076723  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.124872  6516 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:17:59.049993 10175 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:17:59.053798 10158 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:58.997684 10172 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:59.051023  6595 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:59.114725 10036 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.156342  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2385) took 1177 ms. Trace:
I20260709 13:17:59.160693  6595 rpcz_store.cc:276] 0709 13:17:57.978888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.042435 (+1063547us) service_pool.cc:224] Handling call
0709 13:17:59.043661 (+  1226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.156009 (+112348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.156266 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.162093  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:59.121178  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:59.121605  6518 rpcz_store.cc:276] 0709 13:17:57.908732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.911912 (+  3180us) service_pool.cc:224] Handling call
0709 13:17:59.043588 (+1131676us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.181392  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:59.181807 10166 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.182590 10166 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.122449  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:59.122529  6662 rpcz_store.cc:276] 0709 13:17:57.673545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.042951 (+1369406us) service_pool.cc:224] Handling call
0709 13:17:59.056406 (+ 13455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.114141 (+ 57735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.114415 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.027325  9635 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.125032  6593 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:17:59.125439  6516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 673) took 1217 ms (client timeout 1560 ms). Trace:
I20260709 13:17:59.042290  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:59.087625 10005 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.128361  7234 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
W20260709 13:17:59.220574  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 528035us
W20260709 13:17:59.153077 10160 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:59.132982 10158 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:59.133877 10168 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:17:59.127561 10163 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:59.134941  6597 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:59.137784 10037 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:59.139458  9854 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:59.142046 10159 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.594s	user 0.003s	sys 0.013s
I20260709 13:17:59.047848 10143 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.242950  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 826) took 1381 ms. Trace:
I20260709 13:17:59.243049 10143 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.243408  6589 rpcz_store.cc:276] 0709 13:17:57.861955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.969095 (+107140us) service_pool.cc:224] Handling call
0709 13:17:59.242754 (+1273659us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002fc6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:59.242884 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2327789440}
I20260709 13:17:59.047446 10073 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.153766 10172 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:59.249004  9635 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.249133 10172 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.251116 10177 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.251456 10166 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:59.251616 10177 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.253026 10129 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:59.253734 10177 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:59.255105  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 971) took 1487 ms. Trace:
I20260709 13:17:59.255744 10158 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.230702 10039 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.153463 10112 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.127657 10161 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7.
I20260709 13:17:59.107215 10041 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.176644  6595 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:17:59.273080  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:17:59.274065  6595 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:17:59.177464  6663 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:59.096858 10142 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.083470  9912 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.131312 10145 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.108901  6664 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:59.219971  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 143977us
I20260709 13:17:59.292351  6595 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:17:59.203022  6516 rpcz_store.cc:269] 0709 13:17:57.908209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.910234 (+  2025us) service_pool.cc:224] Handling call
0709 13:17:59.125377 (+1215143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.076066 10027 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:17:59.222020  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 113456us
I20260709 13:17:59.222476  6588 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
W20260709 13:17:59.223548  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 181335us
I20260709 13:17:59.295333  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:59.295434  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:17:59.296332 10027 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:59.296242  9913 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:59.297011 10169 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:59.298086 10177 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:59.230702  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:59.298460 10169 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:59.298800  9913 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:59.299047 10169 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:59.299242  9913 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:17:59.299319  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68751us
I20260709 13:17:59.299789 10154 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.300427 10154 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.300006 10169 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.241272 10037 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:59.300840 10154 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.234748  9795 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:59.261523  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.302352  6521 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:59.307991  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.255478  7231 rpcz_store.cc:276] 0709 13:17:57.767874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.808306 (+ 40432us) service_pool.cc:224] Handling call
0709 13:17:57.808716 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:57.970578 (+161862us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:57.970608 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:57.971889 (+  1281us) log.cc:844] Serialized 26 byte log entry
0709 13:17:58.466329 (+494440us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:58.466416 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.466579 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.254852 (+788273us) raft_consensus.cc:1685] finished
0709 13:17:59.254880 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.255043 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84826,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":14993,"thread_start_us":608881,"threads_started":2,"wal-append.queue_time_us":1040939}
I20260709 13:17:59.308738  6516 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.309995  6594 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:17:59.275309  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 676) took 1494 ms. Trace:
I20260709 13:17:59.310799  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2386) took 1081 ms. Trace:
I20260709 13:17:59.310902  6663 rpcz_store.cc:276] 0709 13:17:57.780994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.132004 (+1351010us) service_pool.cc:224] Handling call
0709 13:17:59.177055 (+ 45051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.274967 (+ 97912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.275238 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.312345  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.306330  7198 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:59.300138  9913 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.264096  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:59.301829 10037 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.311379  6588 rpcz_store.cc:276] 0709 13:17:58.229012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.787462 (+558450us) service_pool.cc:224] Handling call
0709 13:17:59.310720 (+523258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.303058 10154 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:17:59.297036 10013 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:17:59.266234 10049 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.298485 10177 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:17:59.306018  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2236) took 1631 ms. Trace:
I20260709 13:17:59.349255  6664 rpcz_store.cc:276] 0709 13:17:57.714172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.108433 (+1394261us) service_pool.cc:224] Handling call
0709 13:17:59.108741 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.305790 (+197049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.305964 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:59.269130  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.270735  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:59.350870  6520 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:59.351148  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 80534.1us
I20260709 13:17:59.236375 10086 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.329434  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.355188  6522 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:59.335024  6514 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:59.336417  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:17:59.358219  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 431555us
I20260709 13:17:59.359380  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:59.337067  6516 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:17:59.241667  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.337966  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.360888  6518 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:17:59.359210  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:59.312379  9795 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:59.363436  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:59.364490  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:17:59.366824  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:17:59.342592 10026 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.373093  7229 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 5. Current term is 7. Ops: []
I20260709 13:17:59.304267  9921 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:59.293869 10035 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:59.352922  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:59.331259 10073 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.370908  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:59.339097  6588 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:59.371206  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:59.354641 10126 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.552s	user 0.017s	sys 0.025s
I20260709 13:17:59.376973  6519 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.362236  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:59.387720  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:17:59.388392  7233 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:59.389619  7237 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:59.389679  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:59.390918  7229 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:17:59.391623  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.355338 10177 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.394240  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:59.395319  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:59.360548  9913 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:59.264071  9886 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.360671  9920 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:59.361260 10013 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:59.398765  7231 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:17:59.418067  7230 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:59.374303  9790 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:59.369740 10154 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:59.404096  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:59.399684  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 104270us
I20260709 13:17:59.406896  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:59.396374 10027 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:59.434504  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:59.375578  9915 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.373045 10049 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:59.412196  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:17:59.443288  7230 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
W20260709 13:17:59.415657  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 117339us
I20260709 13:17:59.444005  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:59.444484  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:17:59.418331  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:59.418107  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:17:59.420660  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 125383us
I20260709 13:17:59.445605  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:59.373617  9822 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.412309  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:17:59.431824  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.431701  6519 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:17:59.448843  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.433686  6518 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:59.402734 10039 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:59.372350 10036 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.401247 10147 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.375465 10157 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:59.446309  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:17:59.457713  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:17:59.424566  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:59.415310 10035 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:59.451417  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:17:59.419817 10184 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:59.463558  6520 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:17:59.428669 10154 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.397358 10126 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.462847  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:59.416831  9854 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:17:59.473469  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:59.474051  7231 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:17:59.474817  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:59.465940  6514 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:59.441635  9886 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:59.483533  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:17:59.478793  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.417650 10158 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:17:59.488181  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:59.443112 10049 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.451632  9896 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:59.494340  6588 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:17:59.469918 10132 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.032s	user 0.008s	sys 0.004s
I20260709 13:17:59.501894  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:59.508167  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:17:59.509229  7231 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:59.452983  9972 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.457263 10039 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:17:59.455502 10027 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:59.510433  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2228) took 1837 ms. Trace:
I20260709 13:17:59.473220 10154 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.467746  9921 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:59.515679  6592 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:17:59.514869  6665 rpcz_store.cc:276] 0709 13:17:57.672570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.787287 (+1114717us) service_pool.cc:224] Handling call
0709 13:17:58.793034 (+  5747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:58.793498 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:58.922019 (+128521us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.305982 (+383963us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.306232 (+   250us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:59.306702 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:17:59.334112 (+ 27410us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:59.334171 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.334267 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.510191 (+175924us) raft_consensus.cc:1685] finished
0709 13:17:59.510216 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.510363 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6475,"prepare.queue_time_us":122980,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":79,"raft.queue_time_us":82314,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":140498,"spinlock_wait_cycles":8738688,"thread_start_us":378470,"threads_started":2,"wal-append.queue_time_us":232895}
I20260709 13:17:59.521711  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:17:59.522503  7229 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:59.524447  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 78899us
I20260709 13:17:59.525382  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:17:59.525573 10188 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:17:59.483639  9795 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.526280  9912 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.527724 10157 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:59.528219 10157 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:59.452069 10013 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:17:59.484452 10184 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.448127 10183 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:17:59.529456  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:17:59.505803 10049 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.516237 10132 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:59.514362 10185 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.520407 10039 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:59.470677 10035 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.473773  9802 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:59.791390  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.796077  6588 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:59.520112  9896 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:59.533820  9912 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:59.536057  9921 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:17:59.872553  9921 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:59.878865  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.878971 10158 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.879973 10158 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.886566  6588 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:59.892824  9802 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:59.587133 10008 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.894656  9920 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:59.895283  9920 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:17:59.897109  9920 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:17:59.897259  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 367684us
I20260709 13:17:59.599429 10027 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.899915  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:17:59.537698 10041 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.566097 10056 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:59.808648 10035 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:17:59.601825 10126 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.544179 10142 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:59.970778  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.982512  7229 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
W20260709 13:17:59.989864  6592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2374) took 2192 ms (client timeout 1854 ms). Trace:
I20260709 13:17:59.990159  6593 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:59.559485 10145 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:17:59.573024 10150 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.542s	user 0.004s	sys 0.009s
I20260709 13:17:59.558151 10157 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.993466  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2220) took 2385 ms. Trace:
I20260709 13:17:59.993912  6666 rpcz_store.cc:276] 0709 13:17:57.607661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.607784 (+   123us) service_pool.cc:224] Handling call
0709 13:17:58.710119 (+1102335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.988767 (+278648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.992885 (+1004118us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000cc950. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:59.993179 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534421888}
I20260709 13:17:59.573717 10161 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:59.996418 10161 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:17:59.998662  9845 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:59.983029  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.999482  9845 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.571678 10167 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.534381 10184 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:00.000887 10167 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:00.001551 10167 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.789558  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.003170  6518 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
W20260709 13:18:00.004572 10071 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.531s	user 0.004s	sys 0.021s
I20260709 13:18:00.005071 10071 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:17:59.568017 10185 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.006631  6513 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:18:00.007954  7234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 961) took 2302 ms (client timeout 1537 ms). Trace:
I20260709 13:18:00.008548 10185 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:59.557667 10190 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:17:59.556102 10195 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:00.011006 10195 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:00.012665 10195 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:17:59.540217 10194 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:00.015133 10194 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.015213  9899 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.565155 10199 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:59.826738 10183 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.864967  9896 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.028332 10183 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.030227  9778 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:17:59.907095 10056 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:17:59.947098  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 489293us
W20260709 13:17:59.963052 10005 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.730s	user 0.014s	sys 0.028s
W20260709 13:17:59.953784  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54594.2us
I20260709 13:17:59.954818 10205 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.955065 10035 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:17:59.957047 10146 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.888s	user 0.009s	sys 0.032s
I20260709 13:17:59.958761 10158 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:17:59.999814  6669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2241) took 2071 ms (client timeout 1843 ms). Trace:
I20260709 13:18:00.010404  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 978) took 1449 ms. Trace:
I20260709 13:18:00.043586  7235 rpcz_store.cc:276] 0709 13:17:58.561265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.561367 (+   102us) service_pool.cc:224] Handling call
0709 13:18:00.010005 (+1448638us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b54004e16d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:00.010189 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116909440}
I20260709 13:17:59.789558  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:00.023185  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.015731 10194 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:59.954999  6595 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.009s	sys 0.012s
I20260709 13:18:00.045136  6514 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.948732  9920 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:00.045147  6595 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.009s	sys 0.013s
W20260709 13:18:00.008328  7234 rpcz_store.cc:269] 0709 13:17:57.705579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.707393 (+  1814us) service_pool.cc:224] Handling call
0709 13:18:00.007492 (+2300099us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400684554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:00.007736 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3175077760}
I20260709 13:18:00.027467  9635 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:00.027804 10195 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:59.914669  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.894511 10086 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.030581 10183 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:17:59.990463  6592 rpcz_store.cc:269] 0709 13:17:57.797205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.797597 (+   392us) service_pool.cc:224] Handling call
0709 13:17:59.989291 (+2191694us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400210f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:59.989583 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3697701760}
I20260709 13:18:00.042400 10035 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.999940 10039 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.067222 10035 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:00.068874 10039 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:00.068811  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2136) took 1010 ms. Trace:
I20260709 13:18:00.069938  6594 rpcz_store.cc:276] 0709 13:17:59.058679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.232169 (+173490us) service_pool.cc:224] Handling call
0709 13:17:59.309699 (+ 77530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.310433 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.068453 (+758020us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b54002fe4d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:00.068648 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157164928}
W20260709 13:18:00.043032  6669 rpcz_store.cc:269] 0709 13:17:57.928282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.436843 (+508561us) service_pool.cc:224] Handling call
0709 13:17:59.999326 (+1562483us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54000e65d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:17:59.999542 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617473408}
I20260709 13:18:00.002271  9845 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.072180  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 827) took 2209 ms. Trace:
I20260709 13:18:00.072445  6520 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:00.072598  6597 rpcz_store.cc:276] 0709 13:17:57.862188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.965011 (+1102823us) service_pool.cc:224] Handling call
0709 13:17:58.965364 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.234693 (+269329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.071712 (+837019us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54003a43d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:00.071967 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389511424}
I20260709 13:18:00.073654  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:00.074707 10154 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.075842  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2387) took 1833 ms. Trace:
I20260709 13:18:00.076067  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.076922 10154 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.077497  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.078864  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:00.079532  6518 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:00.080006  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.080760  6516 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:18:00.083209 10177 consensus_meta.cc:287] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.663s	user 0.000s	sys 0.026s
I20260709 13:18:00.086041 10086 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.086642 10177 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.088625 10086 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:00.086035  9915 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.640s	user 0.036s	sys 0.076s
W20260709 13:18:00.089848 10176 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.027s	user 0.001s	sys 0.001s
I20260709 13:18:00.089990  9915 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:00.091008  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:00.091629 10163 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.092212 10163 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:00.093122  7200 leader_election.cc:400] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7.
I20260709 13:18:00.093557  7200 leader_election.cc:403] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:17:59.998056 10161 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.095181 10161 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:00.083771  9913 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.009s
I20260709 13:18:00.084522  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2035) took 2155 ms. Trace:
I20260709 13:18:00.076887  6519 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:00.076332  6590 rpcz_store.cc:276] 0709 13:17:58.242429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.083743 (+841314us) service_pool.cc:224] Handling call
0709 13:17:59.089337 (+  5594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.094906 (+  5569us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:59.266794 (+171888us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.418337 (+151543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:59.418414 (+    77us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:59.418511 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.075318 (+656807us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b54002f71d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:00.075394 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.075634 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":366888,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":12701,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":10390,"spinlock_wait_cycles":32000768,"thread_start_us":309861,"threads_started":2,"wal-append.queue_time_us":188351}
I20260709 13:18:00.100593  6514 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:00.099450  6513 rpcz_store.cc:276] 0709 13:17:57.928649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.928762 (+   113us) service_pool.cc:224] Handling call
0709 13:17:57.965881 (+ 37119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:00.007130 (+2041249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.084152 (+ 77022us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54002161d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:00.084442 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4691972992}
W20260709 13:18:00.105278  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.031263 10133 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.108673  6516 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:00.111915  9873 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.193s	user 0.025s	sys 0.025s
I20260709 13:18:00.051365 10005 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.062515  9899 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.121681 10133 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.124713 10167 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.156158  9899 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:00.112891 10154 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:00.157879 10146 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:18:00.157553  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45499.1us
I20260709 13:18:00.158955  6520 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:00.154304 10049 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.609s	user 0.004s	sys 0.014s
I20260709 13:18:00.161026 10049 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.162915  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:00.163376 10049 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.165019 10158 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:00.166016  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 828) took 1694 ms. Trace:
I20260709 13:18:00.166484  6593 rpcz_store.cc:276] 0709 13:17:58.471557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.202246 (+730689us) service_pool.cc:224] Handling call
0709 13:17:59.989541 (+787295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.991161 (+  1620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.165778 (+174617us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b5400210f50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:00.165956 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1809204992}
I20260709 13:18:00.160209 10146 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:00.167742 10146 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:00.127319  6521 consensus_meta.cc:287] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.010s
I20260709 13:18:00.168396  6521 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:00.168864 10146 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:00.128453 10026 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.742s	user 0.011s	sys 0.012s
I20260709 13:18:00.170986 10026 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:00.129102  6668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2242) took 2200 ms (client timeout 1914 ms). Trace:
I20260709 13:18:00.107537 10205 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:00.171994  6668 rpcz_store.cc:269] 0709 13:17:57.928736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.477827 (+549091us) service_pool.cc:224] Handling call
0709 13:18:00.128596 (+1650769us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54000c5dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:00.128887 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721325824}
I20260709 13:18:00.173182  6513 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:18:00.173308  7233 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:00.173781  6513 pending_rounds.cc:85] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 7
I20260709 13:18:00.173879 10205 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.174196  6513 pending_rounds.cc:107] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:18:00.131925  6597 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:18:00.175182  9886 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.683s	user 0.006s	sys 0.006s
W20260709 13:18:00.135893 10129 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.864s	user 0.016s	sys 0.011s
I20260709 13:18:00.176162  6596 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 4.
I20260709 13:18:00.176234 10129 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:00.176991 10129 consensus_queue.cc:1243] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:00.137470  9845 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.179069  9845 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.179837 10201 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:00.180277 10201 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:00.181051 10071 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.181504  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:00.181715  9845 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.110363  9915 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:00.182475  7235 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:00.182957  6591 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:00.170475 10146 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:00.184119  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2376) took 2385 ms (client timeout 2454 ms). Trace:
I20260709 13:18:00.184302 10093 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.178293  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:00.184553  6591 rpcz_store.cc:269] 0709 13:17:57.799155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.802813 (+  3658us) service_pool.cc:224] Handling call
0709 13:18:00.183911 (+2381098us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54003bbfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:00.184068 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5470488960}
I20260709 13:18:00.184932 10093 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.187022  9845 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:00.187558  9845 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.187402 10184 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.188051  9845 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:00.184141  7237 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:00.190948  7228 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:00.190840  9886 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:00.192003  9886 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:00.192458  9886 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:00.193135 10056 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.193617  9886 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.195123  9886 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:00.147248  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.148351  7234 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:00.150547  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.101397  9635 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.199407  9635 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.150908 10133 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.202297 10133 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.203230 10183 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:00.204334 10093 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.204468  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:00.205080  7233 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:00.153024 10143 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.909s	user 0.018s	sys 0.024s
I20260709 13:18:00.205753  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:00.187883  7236 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:00.206775  9898 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:00.208254  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:00.208880  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2138) took 1149 ms. Trace:
I20260709 13:18:00.209336  6596 rpcz_store.cc:276] 0709 13:17:59.059369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.181073 (+121704us) service_pool.cc:224] Handling call
0709 13:18:00.208623 (+1027550us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b540019cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:00.208809 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285395328}
I20260709 13:18:00.211042  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.154265 10194 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.214711 10194 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.116890  9873 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:00.140707  6561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.817s (SENT) [suppressed 25 similar messages]
I20260709 13:18:00.216004  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:00.216739  6590 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:00.217267  9873 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.218433  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:00.205631 10154 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:00.181350 10201 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.219113  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:00.219416 10154 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:00.219856 10201 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.220510 10143 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:00.220999 10143 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.220578 10154 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.221336 10143 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:00.221868  9845 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.206935  6515 raft_consensus.cc:2410] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:00.174605  6513 raft_consensus.cc:2889] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:00.193549  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.224668  7237 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:00.212678  7235 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:00.140218 10005 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.229912 10183 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:00.231029  9925 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:00.231514 10183 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:00.232053 10183 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:00.233182 10183 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.235271 10183 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:00.206676 10056 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.238900 10177 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.238950  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:00.240989  9899 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:00.241595  9899 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:00.246745  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:00.248747  9790 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.812s	user 0.006s	sys 0.006s
I20260709 13:18:00.249238  9790 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:00.208823  9898 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:00.250509  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.251577  7234 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:00.251955  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.252383  9898 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:00.252794  7235 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:00.253165  9845 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.206851  6592 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:00.202541 10163 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.260385 10163 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.264626 10184 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.265369  9925 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:00.266041  9925 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.253443  9898 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.243038  9899 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:00.255064  6636 connection.cc:256] RPC call timeout handler was delayed by -1.054805480999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 13:18:00.224355  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:00.280772  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:00.281467  6515 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:18:00.282544  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.283210  6593 raft_consensus.cc:2410] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:00.283048  9915 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7.
I20260709 13:18:00.269424  9925 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:00.285362  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61168.9us
I20260709 13:18:00.287214  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:00.287972  6516 raft_consensus.cc:2368] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9.
I20260709 13:18:00.288496  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.289228  7235 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:00.288304 10184 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:00.290792 10184 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.291256 10184 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:00.290611 10143 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.299901 10035 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.303256 10035 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.303766  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:00.303782 10143 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.304775  9920 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:00.304946  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:00.305927 10049 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:00.306937  9920 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:00.307762  6668 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:00.306475 10049 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:00.309089 10049 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.309942  6668 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:18:00.310873  9920 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.311713  6668 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:18:00.323179  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.326076  7228 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:00.350914  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:00.352159  6514 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:00.358244  6515 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:00.360628  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:00.361583  6519 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:00.366706  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.356798  6513 pending_rounds.cc:85] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 6
I20260709 13:18:00.367447  6513 pending_rounds.cc:107] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:18:00.367787  6513 raft_consensus.cc:2889] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:00.378329  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.380923  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.377652  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.388334  6521 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:00.366530  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:00.394400  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 253836us
W20260709 13:18:00.398941  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.325266  9886 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:18:00.397553  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:00.402853  6663 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:00.398384  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:00.403314  6517 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:00.403939  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:00.403734  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.409049  6519 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:00.418110  6593 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:00.412066  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:00.421533  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.416059  6666 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:18:00.424400  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:00.436233  6590 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:00.452884  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:00.445380  6486 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.843s (SENT)
I20260709 13:18:00.456494  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.430962  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:00.459848  6596 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:00.463316  6592 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:00.461202  7235 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:18:00.465322  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.398092  9845 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 13:18:00.458976  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.468566  6597 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:00.467373  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2141) took 1407 ms. Trace:
I20260709 13:18:00.466063  6669 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:00.474730  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:00.475195  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.469887  6486 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:00.483654  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.484807  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.515568  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:00.577935  6597 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:18:00.524921  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.519762  7228 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:18:00.527125  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 92099.9us
I20260709 13:18:00.471071  6595 rpcz_store.cc:276] 0709 13:17:59.060222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.293276 (+233054us) service_pool.cc:224] Handling call
0709 13:17:59.293533 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.293725 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:00.050121 (+756396us) log.cc:844] Serialized 14 byte log entry
0709 13:18:00.404437 (+354316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:00.438839 (+ 34402us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:00.438943 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.455580 (+ 16637us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.467312 (+ 11732us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103,"prepare.queue_time_us":192354,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":6372,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":27079,"thread_start_us":1037987,"threads_started":2,"wal-append.queue_time_us":369648}
W20260709 13:18:00.582818  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.587244  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:00.468902 10129 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.521409  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:00.518884  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:00.486421 10035 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:00.598109  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.520984  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:00.520520  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:00.546715  9845 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:00.604885  6671 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:00.611146  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.558723 10161 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:00.527197  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:00.612120  6486 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.914s (SENT)
I20260709 13:18:00.606036  6667 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:00.519152 10207 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:00.519629  9873 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.562721 10184 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:00.785200  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.668243  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.377704  7233 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:18:00.684599  6522 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.329s	user 0.015s	sys 0.026s
I20260709 13:18:00.847234  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.378722  6522 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:01.379503  7232 raft_consensus.cc:2368] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10.
W20260709 13:18:01.379662  6522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2399) took 2114 ms (client timeout 1536 ms). Trace:
W20260709 13:18:01.380250  6522 rpcz_store.cc:269] 0709 13:17:59.264984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.266107 (+  1123us) service_pool.cc:224] Handling call
0709 13:18:01.379530 (+2113423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.380862  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:00.641474  6668 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
I20260709 13:18:00.925992  6596 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:18:00.784502  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.640622  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:00.675081  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 74916.6us
I20260709 13:18:01.383935  6520 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:00.933063 10169 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.637903 10035 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.384855 10169 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:00.932488  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:01.387421 10169 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:00.793659  9845 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:00.928331  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 475223us
I20260709 13:18:00.611934 10129 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.401998  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:00.668920 10049 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.640545  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:00.934496  6636 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.001s	sys 0.053s
W20260709 13:18:00.897090 10231 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.305s	user 0.002s	sys 0.000s
I20260709 13:18:01.388689 10184 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:00.890992 10213 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:00.600728  9886 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:01.389047  9873 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:00.670344 10230 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:01.390301  7200 leader_election.cc:400] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9.
W20260709 13:18:00.865494  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.773998 10207 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:01.389279  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 844384us
I20260709 13:18:00.668658  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.019129  6515 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:18:01.431516  6515 pending_rounds.cc:85] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 5
I20260709 13:18:01.112166  6416 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:01.431844  6515 pending_rounds.cc:107] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 4.6
W20260709 13:18:01.432073  6418 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 326) took 30324 ms (client timeout 15000 ms). Trace:
I20260709 13:18:01.432209  6515 raft_consensus.cc:2889] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:00.914842  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.433398  6416 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 8, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 8 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:00.682811  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.435125  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1919) took 1070 ms. Trace:
I20260709 13:18:00.669108  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:00.622597  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.837082  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.436264  7229 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:00.826987  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:00.774667  6670 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:01.429723  6596 pending_rounds.cc:85] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 4
W20260709 13:18:01.437067  6517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.011s	sys 0.035s
I20260709 13:18:01.437171  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1924) took 1070 ms. Trace:
I20260709 13:18:01.437335  6596 pending_rounds.cc:107] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:18:01.437887  6596 raft_consensus.cc:2889] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:01.399701  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2043) took 1321 ms. Trace:
W20260709 13:18:01.438299  6517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.011s	sys 0.035s
W20260709 13:18:00.978569  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:00.669023 10222 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 13:18:00.928277 10201 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.003s
I20260709 13:18:01.401175  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2052) took 1076 ms. Trace:
I20260709 13:18:01.006029  9899 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:01.407980  6636 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.004s	sys 0.071s
I20260709 13:18:00.768299  6665 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:18:01.409051  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.390609  9915 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.452565  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:01.410557  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:01.421505  6522 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:01.425240  6514 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:18:00.670502 10143 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:01.419638  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:01.431905  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.392246 10035 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:01.427696  7200 leader_election.cc:403] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:18:00.675628  6663 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:18:01.430239  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.457739  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1922) took 1092 ms. Trace:
W20260709 13:18:01.458019  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:00.688791 10133 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:01.437804  6670 rpcz_store.cc:276] 0709 13:18:00.366579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.407069 (+ 40490us) service_pool.cc:224] Handling call
0709 13:18:01.437105 (+1030036us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:01.406092  9925 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.116s	user 0.023s	sys 0.011s
I20260709 13:18:01.444113  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:01.446780  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 1.13237e+06us
I20260709 13:18:01.446473  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 1920) took 1081 ms. Trace:
I20260709 13:18:01.435597  6662 rpcz_store.cc:276] 0709 13:18:00.364463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.404709 (+ 40246us) service_pool.cc:224] Handling call
0709 13:18:01.435047 (+1030338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.467348  6665 rpcz_store.cc:276] 0709 13:18:00.364919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.404758 (+ 39839us) service_pool.cc:224] Handling call
0709 13:18:01.446413 (+1041655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.450992  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:01.468286  6595 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.451507  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.452210  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.438366  6518 rpcz_store.cc:276] 0709 13:18:00.072889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.080450 (+  7561us) service_pool.cc:224] Handling call
0709 13:18:00.080964 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:00.081438 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:00.081477 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:00.082120 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 13:18:00.461977 (+379857us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:00.462018 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.462102 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:01.391290 (+929188us) raft_consensus.cc:1685] finished
0709 13:18:01.391310 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.393798 (+  2488us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":379346,"threads_started":1,"wal-append.queue_time_us":605809}
I20260709 13:18:01.420259 10093 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.397254  7237 raft_consensus.cc:2376] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 8. Re-sending same reply.
I20260709 13:18:01.458245  6663 rpcz_store.cc:276] 0709 13:18:00.365810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.366039 (+   229us) service_pool.cc:224] Handling call
0709 13:18:01.457656 (+1091617us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.460341  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.437171 10184 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:01.439354  9898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.010s	sys 0.040s
I20260709 13:18:01.434533 10154 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:01.449261  6513 rpcz_store.cc:276] 0709 13:18:00.324490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.324632 (+   142us) service_pool.cc:224] Handling call
0709 13:18:00.327215 (+  2583us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:00.411636 (+ 84421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:00.411695 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:00.446438 (+ 34743us) log.cc:844] Serialized 52 byte log entry
0709 13:18:00.668113 (+221675us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:00.668249 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.668390 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:01.400958 (+732568us) raft_consensus.cc:1685] finished
0709 13:18:01.400981 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.401122 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9678,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":32051,"thread_start_us":164680,"threads_started":1,"wal-append.queue_time_us":907433}
I20260709 13:18:01.471172  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.471489  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.451931  9873 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.473227  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:01.514451  7228 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:01.474999  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.428179 10049 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.516285  6663 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:01.452510  9886 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.517643  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.461390 10133 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:01.558961  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:01.539338  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64967.4us
I20260709 13:18:01.564065  6665 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:18:01.556764  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 166589us
I20260709 13:18:01.533844  7229 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:01.542157  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:01.475564  6593 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:01.472590 10169 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:18:01.548074  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2051) took 1224 ms. Trace:
I20260709 13:18:01.523375  7230 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:01.515522  7231 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:01.518515 10035 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.565097  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:01.474717  9898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.011s	sys 0.040s
W20260709 13:18:01.472390 10056 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.235s	user 0.011s	sys 0.008s
I20260709 13:18:01.549320  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.457432 10213 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:01.539070 10230 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:18:01.600767 10239 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:01.577250  6670 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:01.469079 10201 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.540087  9873 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:01.432457  6418 rpcz_store.cc:269] 0709 13:17:30.893399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.917369 (+ 23970us) service_pool.cc:224] Handling call
0709 13:18:01.186561 (+30269192us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:58.745976 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:59.052836 (+306860us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:59.073175 (+ 20339us) write_op.cc:435] Acquired schema lock
0709 13:17:59.073195 (+    20us) tablet.cc:662] Decoding operations
0709 13:17:59.074086 (+   891us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:59.074137 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:59.074166 (+    29us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:17:59.074827 (+   661us) tablet.cc:701] Row locks acquired
0709 13:17:59.074844 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:17:59.074994 (+   150us) write_op.cc:270] Start()
0709 13:17:59.075103 (+   109us) write_op.cc:276] Timestamp: P: 1783603079074947 usec, L: 0
0709 13:17:59.075127 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:17:59.163298 (+ 88171us) log.cc:844] Serialized 3210 byte log entry
0709 13:17:59.730406 (+567108us) op_driver.cc:464] REPLICATION: finished
0709 13:17:59.955607 (+225201us) write_op.cc:301] APPLY: starting
0709 13:17:59.955833 (+   226us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:59.956286 (+   453us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:59.956491 (+   205us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:59.958309 (+  1818us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:00.105678 (+147369us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:18:00.110306 (+  4628us) write_op.cc:312] APPLY: finished
0709 13:18:00.132965 (+ 22659us) log.cc:844] Serialized 65 byte log entry
0709 13:18:00.133217 (+   252us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:00.133472 (+   255us) write_op.cc:454] Released schema lock
0709 13:18:00.133533 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:00.134111 (+   578us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:00.134147 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1582637,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":224698,"apply.run_cpu_time_us":5260,"apply.run_wall_time_us":178592,"num_ops":8,"prepare.queue_time_us":417629,"prepare.run_cpu_time_us":14157,"prepare.run_wall_time_us":697338,"raft.queue_time_us":268518,"raft.run_cpu_time_us":12236,"raft.run_wall_time_us":238571,"replication_time_us":655105,"spinlock_wait_cycles":83712,"thread_start_us":932020,"threads_started":4,"wal-append.queue_time_us":216348,"wal-append.run_cpu_time_us":132250,"wal-append.run_wall_time_us":1773454}]]}
I20260709 13:18:01.582374  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:01.571854  6516 rpcz_store.cc:276] 0709 13:18:00.324004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.324155 (+   151us) service_pool.cc:224] Handling call
0709 13:18:00.367639 (+ 43484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:00.367912 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:00.367948 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:00.399782 (+ 31834us) log.cc:844] Serialized 26 byte log entry
0709 13:18:00.687863 (+288081us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:00.687905 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.687984 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.896717 (+208733us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b2400118cb8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5e1a9c4 00007f04b5e18acb 00007f04b5fd3dc9 00007f04b5fd3a55 00007f04b5fd309d 00007f04b8c0510e 00007f04b8c1c88c 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221
0709 13:18:01.541786 (+645069us) raft_consensus.cc:1685] finished
0709 13:18:01.541816 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.547981 (+  6165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27652480,"thread_start_us":229528,"threads_started":1,"wal-append.queue_time_us":182985}
I20260709 13:18:01.520690  9920 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:01.607800  7237 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:01.599490  7228 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:01.606098  7234 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:01.459924  9915 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.608938  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:01.464145  9899 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.606792 10157 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.464028 10143 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.533682 10154 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:01.613691  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 156343us
I20260709 13:18:01.613826  6662 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:18:01.622515  7236 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:01.474156  9925 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.514555 10184 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.626176  7231 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:01.543695 10093 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.604403 10244 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.622785  7229 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:01.605840  9873 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.623983  6663 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:01.621980  6416 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 7, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 7 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:01.578367  9886 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.617076 10056 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:01.634408  7235 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.635006  6668 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:01.616730  9920 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:01.632251  6665 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.636577  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:01.638072  6416 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.608340 10035 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.609709 10201 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.641839 10213 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.644603  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:01.634302  9873 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:01.637727 10169 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.651222 10049 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.646636  6416 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.631039 10183 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:01.652887  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:01.640810 10207 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:01.658805  6663 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:18:01.643425 10056 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:01.659185  6416 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 5, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:01.665958 10245 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.663596  6416 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:01.633967 10133 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:01.652647  9845 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.687441  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1117) took 1224 ms. Trace:
I20260709 13:18:01.688282  7234 rpcz_store.cc:276] 0709 13:18:00.462569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.468570 (+  6001us) service_pool.cc:224] Handling call
0709 13:18:01.687081 (+1218511us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540025c454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:01.687313 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354261888}
I20260709 13:18:01.680693  6664 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:01.691429  7233 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:01.643936 10244 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:01.694159  7198 connection.cc:256] RPC call timeout handler was delayed by -1.3137998570002765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3049s for a response
I20260709 13:18:01.665963  9886 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.704200  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:01.698367 10133 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:01.711401  6595 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:01.712625  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 692) took 1275 ms. Trace:
I20260709 13:18:01.713914  6662 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:01.714814  6416 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 5, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 5 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:01.697093  9845 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:01.717767  7235 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:01.673702 10245 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:01.721716  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.672730 10169 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.713867  6517 rpcz_store.cc:276] 0709 13:18:00.437439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.452356 (+ 14917us) service_pool.cc:224] Handling call
0709 13:18:00.475884 (+ 23528us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:00.580808 (+104924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:00.580851 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:00.619152 (+ 38301us) log.cc:844] Serialized 52 byte log entry
0709 13:18:01.440080 (+820928us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:01.440120 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.440183 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:01.712407 (+272224us) raft_consensus.cc:1685] finished
0709 13:18:01.712427 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.712566 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2250,"raft.queue_time_us":26360,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4588,"thread_start_us":764548,"threads_started":1,"wal-append.queue_time_us":932268}
I20260709 13:18:01.719412  7236 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:18:01.720280  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.723009  9899 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:01.723945  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.731271  7237 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:01.690076 10183 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.755076  7229 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:01.756484  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.758167  6662 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:01.762454  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 1.09332e+06us
I20260709 13:18:01.763262 10252 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.765524 10252 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:01.768512 10252 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:01.774657  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.773423  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.776393  7231 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
W20260709 13:18:01.768996  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.770444 10247 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.772261  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.747010  6664 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:01.749893  6416 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.705139  9886 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:01.678959 10056 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.779942  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:01.720305 10243 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.783532  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.790570  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:01.790941  6416 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 8, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 8 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:01.788939  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:01.787793  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 340746us
I20260709 13:18:01.778666  9915 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:01.798722  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:01.799376  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.792196 10154 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:01.822080  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:01.720868  9920 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:01.825580  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:01.829082  6663 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:01.827438 10245 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:01.826937  6416 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 7, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 7 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:01.834823  6662 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:01.836432  6416 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 6, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 6 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:01.837996  6416 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 1 to 3, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.802677 10093 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.823859 10035 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:01.847419  6416 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 4, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 4 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:01.860069  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.823621  9925 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:01.852834  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 74934.9us
I20260709 13:18:01.847628 10245 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:18:01.839082 10154 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:01.858952 10248 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.859458 10184 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.830974  9886 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.862345  7234 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
W20260709 13:18:01.852698  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 212127us
W20260709 13:18:01.865218  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 300181us
W20260709 13:18:01.848807 10161 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.227s	user 0.011s	sys 0.003s
I20260709 13:18:01.787917 10247 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.835033 10201 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:01.871038  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 264763us
I20260709 13:18:01.868216  6416 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.872771  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:01.804225 10243 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.879522  6416 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:01.892627  6416 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 5, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:01.885895  6663 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:01.895780  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:01.849704  9873 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.871783 10161 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:01.914294  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 149387us
I20260709 13:18:01.916229  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:01.916965  6667 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:01.880765 10244 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.895072 10157 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9.
I20260709 13:18:01.923372  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:01.930546  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:01.931210  6519 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:01.871626 10035 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:18:01.930893  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:01.909377 10201 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.940824  6665 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:01.879148  9915 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:18:01.956502  6513 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:01.934582  6517 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:01.967783  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:01.971589  6521 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:01.896780  9925 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:18:01.982903  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47028us
I20260709 13:18:01.974813  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:01.984326  6522 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:01.975427  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:01.975983 10246 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:01.976780  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:01.977164 10129 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.977792  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:01.978778 10184 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:01.978914  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:01.980228 10262 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:01.980566  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.980667 10252 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:01.934790 10244 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:01.877012 10253 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:18:01.982251  6664 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:01.905560  9886 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:01.985425  6515 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:01.948061 10183 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.919261 10154 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:18:01.948276 10169 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:01.960559 10093 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:01.960587 10161 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.879503 10248 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.005460  6520 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:02.014986  7234 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:02.014842  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.988598 10213 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:02.025962  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:02.029114  7236 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
W20260709 13:18:02.029608  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 84169.9us
I20260709 13:18:02.033092  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:02.034361  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:01.989526 10184 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.039057  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:02.039515  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.043357  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.044056  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.044814  7237 raft_consensus.cc:2376] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 5. Re-sending same reply.
I20260709 13:18:02.046568 10161 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.017486  9898 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:01.991825 10133 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:18:02.017567 10207 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.017026 10035 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.055658  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:02.055706 10035 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.056510  6595 raft_consensus.cc:606] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:18:02.056565 10265 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:02.056897  6595 raft_consensus.cc:614] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:02.057183 10265 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:01.989979 10201 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:02.085587  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 112253us
I20260709 13:18:02.043295 10184 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:02.049896  9896 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:02.088143  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 159799us
W20260709 13:18:02.089826  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 180797us
I20260709 13:18:02.007493 10262 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.058403 10035 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:02.052726  9898 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:02.087286 10056 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:18:02.090951  9896 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:02.100909  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.112936  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.141752  6597 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:02.090327 10143 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.053627 10244 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:02.085008 10183 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 13:18:02.136969  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 97951us
I20260709 13:18:02.058116 10265 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.090149 10093 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:18:02.088874 10207 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.085821 10246 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:18:02.154579  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a321aae654f82803540bffcf7968f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:02.169145  6595 raft_consensus.cc:606] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:02.169380  6595 raft_consensus.cc:614] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:02.101161  9899 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.149369  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:02.222342  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 108301us
I20260709 13:18:02.143023  9896 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:02.113469 10213 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:02.166867  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 141005us
I20260709 13:18:02.225039  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.226553  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:02.232831  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 299140us
I20260709 13:18:02.235572  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:02.235996  6517 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:02.236505  6522 raft_consensus.cc:2410] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:02.236997  6597 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:02.233050  6594 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:02.238313  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:02.239743  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 225648us
I20260709 13:18:02.238257  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.254398  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:02.168251 10247 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.255129  6520 raft_consensus.cc:2376] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8. Re-sending same reply.
I20260709 13:18:02.255820  6487 leader_election.cc:400] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10.
I20260709 13:18:02.261498  6487 leader_election.cc:403] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:02.240489  6592 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:18:02.239822  6513 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
W20260709 13:18:02.257009  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.113390 10270 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 13:18:02.270507  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.272262  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:02.239008 10183 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:18:02.271782  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 232343us
I20260709 13:18:02.274653  6521 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:02.274655  6517 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:02.228010 10276 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:18:02.235934  9925 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:02.297106  9898 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.263737  9873 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:02.313321  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 164021us
I20260709 13:18:02.313476  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.314143  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.315600  7230 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:02.272673 10129 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:02.339428  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:02.290633 10255 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 13:18:02.276834 10183 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:02.228463  9899 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.277815 10267 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10.
W20260709 13:18:02.358232  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:02.370052  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.302795  9925 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:02.371439  6516 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:18:02.371832  6516 pending_rounds.cc:85] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 6
I20260709 13:18:02.292434  9920 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 13:18:02.368211  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 138505us
I20260709 13:18:02.338930  9898 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:02.372067  6516 pending_rounds.cc:107] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:18:02.376165  6516 raft_consensus.cc:2889] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:02.353144  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:02.298277 10243 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:02.354105 10260 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:02.391862  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 121811us
W20260709 13:18:02.398383  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 85818.3us
I20260709 13:18:02.399259  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.399546  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.400358  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.400986  7236 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:02.403554  6520 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:02.408092 10243 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:02.424119  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:02.430426  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:02.431543  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:02.434720  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.383841 10260 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.383206 10247 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:02.392517 10248 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:02.319015 10244 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.447593 10260 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:02.492759  6519 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.561s	user 0.005s	sys 0.005s
I20260709 13:18:02.493216  6519 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:02.443492 10049 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:02.482698  9925 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.374437 10213 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.446074 10247 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:02.455199 10201 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.477490  9886 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:02.537166  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 158668us
W20260709 13:18:02.537302  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 131338us
I20260709 13:18:02.456037 10251 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:02.552820  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 237772us
I20260709 13:18:02.552562  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:02.553524  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 113788us
I20260709 13:18:02.455072 10129 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:18:02.537886 10247 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:02.539861  9925 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:02.553534  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:02.541953  6486 connection.cc:256] RPC call timeout handler was delayed by -1.0358103870000832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 13:18:02.555395  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:02.557821  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:02.558091  6486 connection.cc:256] RPC call timeout handler was delayed by -1.0334578709998823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260709 13:18:02.538461 10262 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.551149 10251 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:02.476373 10183 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.573904 10183 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:02.572615  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:02.543776 10253 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.564978  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:02.565351 10275 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:18:02.565716 10248 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:02.580657 10277 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:18:02.581404 10266 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:02.538865 10287 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:18:02.658063 10282 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:02.658298 10287 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:02.658761 10287 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:02.590580  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:02.659564  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 116258us
I20260709 13:18:02.595326  9899 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:02.660346  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:02.555877 10201 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:02.656739 10253 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.661561 10201 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.662021 10201 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:02.578054 10251 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:02.663094 10253 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:02.667783 10184 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 13:18:02.668236 10184 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:02.656021 10258 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.007s	sys 0.000s
W20260709 13:18:02.678809 10258 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.007s	sys 0.000s
I20260709 13:18:02.602721 10262 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:02.679396 10262 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.679831 10262 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:02.682801  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.616135 10296 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:02.616705  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:02.625403 10207 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.631922 10257 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:18:02.634627  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62059.1us
W20260709 13:18:02.642256  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:02.647429  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 382785us
I20260709 13:18:02.696002  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:02.653072  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 91701.4us
I20260709 13:18:02.535571 10049 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.675266  6517 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:02.653578 10248 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.659598 10287 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.674995 10295 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:02.598546  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:02.597813  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:02.687194  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 122213us
I20260709 13:18:02.699918  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:02.700990  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:02.699009  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.696607  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:02.700273  6594 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
W20260709 13:18:02.705209  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 88589.2us
W20260709 13:18:02.696578 10265 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.004s	sys 0.003s
I20260709 13:18:02.688417 10266 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:02.695111 10246 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:02.695396 10257 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:02.662520 10260 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.699532 10248 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:02.703084 10295 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.716568  9886 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.719388 10270 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:02.720075 10276 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:02.710242 10258 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:02.716159 10265 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.005s	sys 0.004s
I20260709 13:18:02.685711  9896 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.726692  6513 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:02.710490  9899 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.737277  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:02.711199  9873 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:18:02.726730 10270 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.746265  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:02.724749 10299 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.738580  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:02.726989 10266 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:02.725092 10295 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:02.727121 10276 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:02.730824 10258 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:02.775969  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.732560 10246 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:18:02.748370 10270 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:02.730201 10201 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.730728  9886 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.756899  9920 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:02.805855  9920 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:02.806871  9920 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.808017 10260 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.764065 10296 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:02.750221 10297 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:18:02.811352 10297 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:02.811792 10251 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.813890 10251 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.765126 10276 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:02.817991 10266 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.754789 10299 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.819862  9886 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.766356 10258 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:02.820510  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 160263us
W20260709 13:18:02.820883  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 94156us
I20260709 13:18:02.824710  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:02.825100  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:02.827097  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.784314 10252 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:02.748137  9899 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.831658  6593 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:02.831686  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:02.782766 10184 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:02.857394  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:02.883606  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52035us
I20260709 13:18:02.835144  9886 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.881554 10247 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:02.855326 10265 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:02.891652  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:02.891986  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.893253  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:02.886758  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 148309us
I20260709 13:18:02.902433  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:02.892906  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.894568  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:02.928591  6588 raft_consensus.cc:2410] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:02.896051  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.892289  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.935092  6591 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:02.938956  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:02.962185  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:02.962201  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:02.967785  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76176.7us
I20260709 13:18:02.886871 10184 raft_consensus.cc:2764] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:02.891142 10049 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:02.979362  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:02.980188  6515 raft_consensus.cc:2368] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:02.980856  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.981027  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:02.981618 10297 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:02.981920  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.857276 10257 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.983304  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.983451 10257 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:02.983963  6593 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
W20260709 13:18:02.905187 10305 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:02.927688 10035 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:18:02.912294 10201 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:02.886073 10312 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:02.974792 10272 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:02.975181  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.975601 10299 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:02.975859  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:02.985158  6637 leader_election.cc:400] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:03.075014  6637 leader_election.cc:403] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:18:02.973325 10288 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:02.894223 10295 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.912659 10093 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:03.063961  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:02.966256  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:03.067772  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 350132us
I20260709 13:18:02.903695 10247 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:02.899036  9896 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:02.985570  6515 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:02.916249  9886 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:02.935153 10270 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:03.168318 10259 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.085s	user 0.005s	sys 0.000s
I20260709 13:18:03.178601  9886 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.087122  9920 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:03.179319 10093 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:03.124457  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.064091  9925 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.000s
I20260709 13:18:02.974834 10313 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.209s
I20260709 13:18:03.546305 10313 consensus_queue.cc:1243] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:03.546881 10313 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:03.547246 10313 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:03.548137 10313 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:03.549326 10313 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:03.063309 10263 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:03.076822 10312 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.552287 10312 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:03.325398  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:03.558136  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.558221  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 233063us
I20260709 13:18:03.559800  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:03.560101  9925 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:03.560921  9925 raft_consensus.cc:2764] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:03.561281  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:03.392104 10248 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.004s
I20260709 13:18:03.561587 10295 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.562107 10248 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.562846 10247 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:18:03.405325  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.460546  6514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 699) took 1493 ms (client timeout 1865 ms). Trace:
W20260709 13:18:03.563936  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.564126  6514 rpcz_store.cc:269] 0709 13:18:01.967501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.967625 (+   124us) service_pool.cc:224] Handling call
0709 13:18:03.459652 (+1492027us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b540029dea0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b9084e18 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:03.460353 (+   701us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400044054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:03.460464 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1750581504}
I20260709 13:18:03.462460  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:03.566242 10312 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.568337 10312 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.562515  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:03.569391 10247 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:03.090040 10272 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:03.570629  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:03.471971  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.571385 10247 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:03.495724  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 529701us
I20260709 13:18:03.572453  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:03.572125 10272 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.573874  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:03.575685 10248 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.576555 10248 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:18:03.507973 10262 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.828s	user 0.014s	sys 0.001s
I20260709 13:18:03.587224 10248 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.590566 10248 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:03.542258 10252 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:03.497157 10280 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:03.592330 10252 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:03.579694 10295 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:03.593343 10295 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.593729 10295 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:03.601581  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.608332  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:03.586323  6592 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:18:03.587837  6518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.005s	sys 0.004s
W20260709 13:18:03.609658  6518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.005s	sys 0.004s
I20260709 13:18:03.587028 10262 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:03.591277  9920 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:03.591985  6520 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:18:03.516394  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 441824us
I20260709 13:18:03.578334  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:03.619807 10248 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:03.621030  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:03.621806  9925 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:03.602547 10280 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:03.622892 10247 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:03.623276  6522 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:03.623399 10247 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.623811 10247 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:03.619881  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:03.613607  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:03.622332  9925 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:03.627405  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:03.627611 10201 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:03.629226 10201 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:18:03.511520  6521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.015s	sys 0.028s
W20260709 13:18:03.631474  6521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.015s	sys 0.028s
I20260709 13:18:03.632565 10213 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:03.632571 10262 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:03.622371  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:03.622978 10280 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:03.640318 10280 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:03.627414  9925 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.646606 10201 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:03.649384  6594 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:03.649125 10201 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.636094 10213 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:03.651944 10213 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:03.642066  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:03.616178  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:03.658170  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:03.675949  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:03.679066  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:03.679347  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:03.678625 10262 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:03.678956 10280 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:03.680862 10184 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:03.693269 10262 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:03.688925  9920 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:03.699297 10252 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.702589 10252 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.703281  6518 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:03.707505 10332 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.708073 10332 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.709235  9899 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:03.709913  9899 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:03.710171 10332 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.711135 10332 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:03.711650 10313 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:03.711817 10332 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.712178 10313 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:03.712330 10332 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:03.712687 10313 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:03.713901 10207 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:03.714293 10207 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:03.700158 10262 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:03.715118 10207 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.716140 10207 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:03.698283  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:03.697297 10295 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.727978 10295 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.718042 10213 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:03.729555 10258 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.893s	user 0.001s	sys 0.010s
W20260709 13:18:03.759754  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 80794.3us
W20260709 13:18:03.763875  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 84635.1us
I20260709 13:18:03.765475  6592 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:03.766243  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:03.758991 10258 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:03.773437  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54412.9us
I20260709 13:18:03.773929  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:03.774564  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.774843  6516 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:03.776618  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:03.771956  9899 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.781689 10258 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:03.089850 10169 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.701s
I20260709 13:18:03.792513  6515 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:03.795332  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:03.811012  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.811920  6522 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:18:03.809932  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.812371  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.812021  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:03.814736  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.815186  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:03.815871  6592 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:03.779078  6407 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 921c423406da4f289884afca51ca198f (127.5.72.129:34121) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 921c423406da4f289884afca51ca198f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:03.823169  6407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44283us
I20260709 13:18:03.823076  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:03.826783  6514 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:03.831763  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.832597  6521 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:03.837411  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.839591  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:03.834362 10335 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.834689  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:03.840988  6592 raft_consensus.cc:2376] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7. Re-sending same reply.
I20260709 13:18:03.837090  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:03.816334 10340 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:03.795374  9899 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.849392  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:03.813390 10336 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:18:03.850040  6665 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:03.840466 10335 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:03.852047  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 83921.7us
I20260709 13:18:03.827498 10313 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.861128  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:03.829461 10262 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:03.848310 10248 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:03.882078  6592 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 4.
I20260709 13:18:03.882153  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:03.869246  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:03.869616  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:03.840798  9873 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:18:03.884341  7234 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:03.867974  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:03.885840  7228 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:03.879590  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.887059  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.887735  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.887892  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:03.888002  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:03.859493  9925 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:03.889660  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:03.887780  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:03.890589  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:03.898564  6667 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:03.897256  7231 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:03.897365  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:03.884629  7236 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:03.897934  6665 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:03.900244  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:03.900950  6662 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:03.901609  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:03.902325  7230 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:03.904105  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:03.904894  6665 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:03.882400 10248 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.911041  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:03.884809 10257 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.901s	user 0.009s	sys 0.005s
I20260709 13:18:03.912767  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:03.913540  6665 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:03.920109  7228 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:03.922092  6522 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:18:03.919767  7236 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:03.924854  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:03.897027 10244 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:03.930076  7234 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:03.932996  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:03.933722  7234 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:03.904542  9925 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:03.907279 10348 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.907455 10213 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:03.917619 10248 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:03.938999  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:03.943252  6665 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:03.946506  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:03.947540  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.948632  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:03.949479  6667 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:03.950580  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:03.956244  6515 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:03.955525  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:03.936954 10336 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:03.965670  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.998857  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:03.999699  7229 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:18:04.000355 10335 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:04.000849  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:04.001691  7229 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:04.004124 10313 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:04.007452 10343 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:18:04.008096  9899 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:03.997884  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:04.002506  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:04.030869  7229 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:04.185169  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.215224  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.506s (SENT)
I20260709 13:18:04.215716  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:04.239147  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 241320us
I20260709 13:18:03.996008 10347 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.241598  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:03.937203 10244 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.202730  6451 debug-util.cc:398] Leaking SignalData structure 0x7b080008b740 after lost signal to thread 5413
I20260709 13:18:03.934638 10247 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.242794  7200 leader_election.cc:336] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.517s (SENT)
I20260709 13:18:04.247128  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:04.238510 10295 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:04.266139  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:04.247921 10345 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 13:18:04.263397  7198 leader_election.cc:336] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.647s (SENT)
W20260709 13:18:04.274392  7200 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.668s (SENT)
I20260709 13:18:03.954283 10348 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.287209  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:04.287961  7229 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:04.267634 10295 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:04.268886 10331 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:04.211474 10336 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.322968  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.283058 10262 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.330850  7234 raft_consensus.cc:2376] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7. Re-sending same reply.
W20260709 13:18:04.346390  6486 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.536s (SENT)
W20260709 13:18:04.351980  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.261430 10347 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.373585  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.303804 10295 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:04.301118 10280 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:04.462549  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.395596  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.413991 10247 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.273043  9925 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.501713  7200 leader_election.cc:336] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.781s (SENT)
W20260709 13:18:04.714090  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 212419us
W20260709 13:18:04.715658  7200 leader_election.cc:336] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.813s (SENT)
I20260709 13:18:04.716312  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:04.563022  7198 leader_election.cc:336] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.899s (SENT)
I20260709 13:18:04.717641  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:04.703918  7229 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:04.377424  9873 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:04.314786 10035 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:04.424125 10348 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:04.432430 10280 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.332247 10331 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.713696 10262 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.725049 10351 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.727216 10247 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.133689  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:04.756220  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:04.757040  7229 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:04.738489 10355 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:04.759819  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2422) took 1777 ms (client timeout 1638 ms). Trace:
I20260709 13:18:04.745740 10352 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:04.759624 10247 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:04.755141 10351 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.794066 10247 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:03.067165 10276 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:04.794499 10351 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:04.795475  6596 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:04.798745 10357 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:04.798998  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 236011us
I20260709 13:18:04.799300 10357 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.801318 10357 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.802765 10336 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:04.790273  6593 rpcz_store.cc:269] 0709 13:18:02.976533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.037392 (+ 60859us) service_pool.cc:224] Handling call
0709 13:18:04.753686 (+1716294us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:04.806699  6562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.015s	sys 0.007s
W20260709 13:18:04.807020  6562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.015s	sys 0.007s
I20260709 13:18:04.807161  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:04.807605  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 541506us
I20260709 13:18:04.807755  6593 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:04.808818 10248 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.851s	user 0.010s	sys 0.000s
I20260709 13:18:04.809103  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:04.809185 10248 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:04.795513  6670 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 13:18:04.812178  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 422334us
I20260709 13:18:04.791822 10355 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.814200  6519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 693) took 3882 ms (client timeout 2019 ms). Trace:
I20260709 13:18:04.815009 10355 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:04.816339 10276 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:04.816933 10276 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:04.817389 10276 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:04.820371  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.814599  6519 rpcz_store.cc:269] 0709 13:18:00.931973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.932159 (+   186us) service_pool.cc:224] Handling call
0709 13:18:04.814023 (+3881864us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54002166d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:04.814129 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250905472}
W20260709 13:18:04.769382 10346 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.528s	user 0.000s	sys 0.000s
I20260709 13:18:04.769810 10348 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:04.743800 10280 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:04.773365 10288 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:04.775420  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:04.776177 10331 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.776906 10244 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:04.777257  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2393) took 1884 ms (client timeout 1662 ms). Trace:
W20260709 13:18:04.837633  6594 rpcz_store.cc:269] 0709 13:18:02.892550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.892668 (+   118us) service_pool.cc:224] Handling call
0709 13:18:04.777066 (+1884398us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b54001b6f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:04.777185 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1658429184}
I20260709 13:18:04.750931 10262 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.779834  7229 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:04.784128  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:04.785290 10332 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.072s	user 0.010s	sys 0.004s
I20260709 13:18:04.786718 10347 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:04.828332  6520 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.012s	sys 0.014s
W20260709 13:18:04.811172  7198 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.920s (SENT)
W20260709 13:18:03.075497  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 90482.3us
I20260709 13:18:04.833086 10357 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:04.840219 10295 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.822753 10248 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:04.844400  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:04.844913  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:04.859826  7229 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:04.860340  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:04.836447  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.853113  6520 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.012s	sys 0.014s
I20260709 13:18:04.835531 10260 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:04.856427  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:04.863056  7231 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:04.865936 10276 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:04.860713  9873 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:04.873523  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.876188 10262 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:04.876619  6561 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items
I20260709 13:18:04.848721 10332 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.878597  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 67447us
W20260709 13:18:04.879240  7198 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.923s (SENT)
I20260709 13:18:04.865875  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:04.879923  6593 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:04.880412  6519 raft_consensus.cc:2410] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:18:04.880908  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 843) took 1758 ms (client timeout 1781 ms). Trace:
W20260709 13:18:04.881330  6594 rpcz_store.cc:269] 0709 13:18:03.122103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.845129 (+1723026us) service_pool.cc:224] Handling call
0709 13:18:04.880856 (+ 35727us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.882534  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:04.883761  7231 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:04.885022  7231 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:04.885448  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2248) took 1033 ms. Trace:
I20260709 13:18:04.885936  6663 rpcz_store.cc:276] 0709 13:18:03.852338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.782884 (+930546us) service_pool.cc:224] Handling call
0709 13:18:04.783226 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:04.783585 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.783616 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:04.784124 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:18:04.879054 (+ 94930us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:04.879096 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.879180 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.885234 (+  6054us) raft_consensus.cc:1685] finished
0709 13:18:04.885254 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.885391 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35900,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":60160,"thread_start_us":94551,"threads_started":1,"wal-append.queue_time_us":99437}
I20260709 13:18:04.891224  7229 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:18:04.880447  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2403) took 1983 ms (client timeout 1979 ms). Trace:
W20260709 13:18:04.892159  6593 rpcz_store.cc:269] 0709 13:18:02.897301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.806847 (+1909546us) service_pool.cc:224] Handling call
0709 13:18:04.880381 (+ 73534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.893824  7229 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:18:04.873656  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:03.185309 10287 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:04.897447 10332 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.847328 10288 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:04.900321 10288 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:04.903260  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:04.882138 10260 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.904443  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:04.861795 10357 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:04.876905  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2418)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2187)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2425)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2408)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2427)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2189)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2396)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2407)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 847)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
W20260709 13:18:04.918582  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.090268  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:04.913285  9898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.729s	user 0.012s	sys 0.020s
I20260709 13:18:04.920321  6589 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
W20260709 13:18:04.912333 10274 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 2.681s	user 0.011s	sys 0.025s
I20260709 13:18:04.950708  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:04.951372  6669 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:04.951630  6664 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:04.951907  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2260) took 1009 ms. Trace:
I20260709 13:18:04.952221  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2245) took 1104 ms. Trace:
I20260709 13:18:04.905409 10260 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.952613  6664 rpcz_store.cc:276] 0709 13:18:03.847511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.950985 (+1103474us) service_pool.cc:224] Handling call
0709 13:18:04.951391 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.952009 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.952171 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:04.920636 10281 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.936s	user 0.013s	sys 0.023s
I20260709 13:18:04.919665 10270 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.909611 10282 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:04.954236 10282 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:04.953804 10281 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.969s	user 0.013s	sys 0.027s
W20260709 13:18:04.949034  9898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.765s	user 0.015s	sys 0.022s
I20260709 13:18:04.952405  6669 rpcz_store.cc:276] 0709 13:18:03.942556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.950375 (+1007819us) service_pool.cc:224] Handling call
0709 13:18:04.951830 (+  1455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:03.090272  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:04.953788 10357 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:04.974898  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 117105us
W20260709 13:18:04.975723  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.975944  6407 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 921c423406da4f289884afca51ca198f (127.5.72.129:34121) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 921c423406da4f289884afca51ca198f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:04.976197  6637 connection.cc:256] RPC call timeout handler was delayed by -1.4146938290000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
W20260709 13:18:04.976580  6637 connection.cc:256] RPC call timeout handler was delayed by -1.3218951710000511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
W20260709 13:18:04.976872  6637 connection.cc:256] RPC call timeout handler was delayed by -1.3051457919998484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
W20260709 13:18:04.977161  6637 connection.cc:256] RPC call timeout handler was delayed by -1.2569402530000389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 13:18:04.977432  6637 connection.cc:256] RPC call timeout handler was delayed by -1.2401392509998459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 13:18:04.977694  6637 connection.cc:256] RPC call timeout handler was delayed by -1.2229938289999609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 13:18:04.977946  6637 connection.cc:256] RPC call timeout handler was delayed by -1.0801619799999571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
W20260709 13:18:04.978219  6637 connection.cc:256] RPC call timeout handler was delayed by -1.0747420939999301s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3669s for a response
I20260709 13:18:04.978883 10332 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.979377 10332 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:04.962312  6561 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2418)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2187)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2425)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2408)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2427)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2189)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2396)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2407)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 853)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 847)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
I20260709 13:18:04.981505 10332 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.981865  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:04.983050 10332 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:04.983817 10332 consensus_queue.cc:1243] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:04.984261  9925 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:04.984591  9925 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:04.985217  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:04.982144  6561 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2418)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2187)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2425)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2408)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2427)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2189)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2396)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2407)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 853)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 847)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
I20260709 13:18:04.985317  9925 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:04.986754  9925 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:04.910449 10295 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:04.907537 10184 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:04.988307 10184 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:04.957106  6591 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
W20260709 13:18:04.990355  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2415) took 2015 ms (client timeout 1662 ms). Trace:
W20260709 13:18:04.991065  6591 rpcz_store.cc:269] 0709 13:18:02.974828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.089893 (+115065us) service_pool.cc:224] Handling call
0709 13:18:04.990280 (+1900387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40960}
W20260709 13:18:04.992076  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2418) took 2016 ms (client timeout 1941 ms). Trace:
W20260709 13:18:04.992425  6591 rpcz_store.cc:269] 0709 13:18:02.975636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.991887 (+2016251us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:04.991998 (+   111us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:18:04.993151  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2187) took 2032 ms (client timeout 1976 ms). Trace:
W20260709 13:18:04.993424  6591 rpcz_store.cc:269] 0709 13:18:02.961413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.993014 (+2031601us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:04.993106 (+    92us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:18:04.994165  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2425) took 2016 ms (client timeout 1979 ms). Trace:
I20260709 13:18:04.994797  6669 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:04.995343  6588 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:04.986776  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:04.996670  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:04.919277  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1015) took 1047 ms. Trace:
W20260709 13:18:04.949340  7198 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.932s (SENT)
I20260709 13:18:04.998939  6515 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:04.998878  7232 rpcz_store.cc:276] 0709 13:18:03.871617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.918251 (+1046634us) service_pool.cc:224] Handling call
0709 13:18:04.918645 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.918866 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.918902 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:04.918958 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:04.919002 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.919058 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.919180 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.948897  6590 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:05.000196  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2428) took 2022 ms (client timeout 1675 ms). Trace:
I20260709 13:18:03.183405 10093 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.000811  7232 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:05.001003  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2182) took 2265 ms. Trace:
I20260709 13:18:05.001483  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1143) took 1074 ms. Trace:
I20260709 13:18:05.001390  6596 rpcz_store.cc:276] 0709 13:18:02.735180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.735297 (+   117us) service_pool.cc:224] Handling call
0709 13:18:02.748228 (+ 12931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.795876 (+2047648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.000769 (+204893us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54002fcbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:05.000923 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4707683200}
W20260709 13:18:05.000597  6590 rpcz_store.cc:269] 0709 13:18:02.978101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.089983 (+111882us) service_pool.cc:224] Handling call
0709 13:18:05.000131 (+1910148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32000}
W20260709 13:18:05.003777  6486 leader_election.cc:336] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.908s (SENT)
I20260709 13:18:05.003683 10293 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:04.940263 10280 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:05.004890  6637 leader_election.cc:336] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.555s (SENT)
W20260709 13:18:04.962091  6451 debug-util.cc:398] Leaking SignalData structure 0x7b080008b940 after lost signal to thread 6454
I20260709 13:18:05.005339  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.007229  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.007875  6669 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:05.008829  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1133) took 1090 ms. Trace:
W20260709 13:18:04.999460  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50153.3us
I20260709 13:18:05.009410  7234 rpcz_store.cc:276] 0709 13:18:03.918540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.955221 (+ 36681us) service_pool.cc:224] Handling call
0709 13:18:05.008676 (+1053455us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540051a254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:05.008773 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859664256}
W20260709 13:18:05.010660  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.008872  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:05.003628  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2427) took 2025 ms (client timeout 2024 ms). Trace:
I20260709 13:18:05.004338  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:05.004603 10254 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:05.012099  6671 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:05.012421 10254 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:05.013978  6486 leader_election.cc:336] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.915s (SENT)
I20260709 13:18:05.007257 10247 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:05.014416  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 722) took 1143 ms. Trace:
I20260709 13:18:04.905485  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:05.016911  6519 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:05.017900  6637 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.638s (SENT)
I20260709 13:18:05.018602  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:04.982524  6514 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
W20260709 13:18:05.020432  6637 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.641s (SENT)
I20260709 13:18:05.021183  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.017184 10253 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260709 13:18:05.011845  6590 rpcz_store.cc:269] 0709 13:18:02.977808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.003465 (+2025657us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:05.003573 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:05.026398 10284 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:05.027388 10284 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:05.028014 10280 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.028801 10284 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:05.000082  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:05.015625 10252 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:18:05.031252  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2408) took 2132 ms (client timeout 2067 ms). Trace:
I20260709 13:18:05.032187  6515 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:04.964320  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.918318  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.016882  6486 leader_election.cc:336] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.955s (SENT)
I20260709 13:18:05.037153  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:04.978246  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:05.032222  6596 rpcz_store.cc:269] 0709 13:18:02.898525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.002591 (+2104066us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:05.031064 (+ 28473us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:18:05.040134  7198 leader_election.cc:336] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.229s (SENT)
W20260709 13:18:04.994449  6591 rpcz_store.cc:269] 0709 13:18:02.977273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.994010 (+2016737us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:04.994095 (+    85us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:05.043252 10252 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.055563  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1138) took 1131 ms. Trace:
W20260709 13:18:05.052210  7200 leader_election.cc:336] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.229s (SENT)
I20260709 13:18:05.059237  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:05.013798 10254 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.044407  6670 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:05.016055  6521 rpcz_store.cc:276] 0709 13:18:03.870861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.870990 (+   129us) service_pool.cc:224] Handling call
0709 13:18:03.898940 (+ 27950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.274047 (+375107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.274100 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:04.274156 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:04.274200 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.013996 (+739796us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b540007d0d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:05.014073 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.014304 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862099200}
I20260709 13:18:05.002058  7232 rpcz_store.cc:276] 0709 13:18:03.926939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.000252 (+1073313us) service_pool.cc:224] Handling call
0709 13:18:05.000571 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.001232 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.001428 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.060855 10254 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.062240 10379 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:05.063131 10379 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:05.043439  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:05.065802  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49022.8us
I20260709 13:18:05.026492  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:05.069267  6590 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:05.066818  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 88642.8us
I20260709 13:18:05.071133  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.046921  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:05.073110  6562 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.077497  6520 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:05.080832  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1139) took 1156 ms. Trace:
I20260709 13:18:05.091504  6588 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
W20260709 13:18:05.092466  6588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2402) took 2195 ms (client timeout 1765 ms). Trace:
I20260709 13:18:05.056301  7231 rpcz_store.cc:276] 0709 13:18:03.923980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.886527 (+962547us) service_pool.cc:224] Handling call
0709 13:18:04.886839 (+   312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:04.887401 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.887432 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:04.888032 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 13:18:05.055204 (+167172us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:05.055246 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.055314 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:05.055355 (+    41us) raft_consensus.cc:1685] finished
0709 13:18:05.055367 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.055492 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":166337,"threads_started":1,"wal-append.queue_time_us":4715}
I20260709 13:18:05.103658  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.081346  7229 rpcz_store.cc:276] 0709 13:18:03.925990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.893180 (+967190us) service_pool.cc:224] Handling call
0709 13:18:04.893580 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:04.894260 (+   680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.080749 (+186489us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:05.092983  6588 rpcz_store.cc:269] 0709 13:18:02.897063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.966878 (+ 69815us) service_pool.cc:224] Handling call
0709 13:18:05.092366 (+2125488us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:05.107259  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.103580  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.116690  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:05.117216  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65061.1us
I20260709 13:18:05.117445  6596 raft_consensus.cc:2368] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10.
W20260709 13:18:05.117812  6486 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.103s (SENT)
I20260709 13:18:05.117458  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:05.118372  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:05.119362  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.119988  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:05.099282  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2396)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2407)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 847)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
W20260709 13:18:05.139719 10261 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.943s	user 0.016s	sys 0.030s
W20260709 13:18:05.140875  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2419) took 2165 ms (client timeout 2139 ms). Trace:
I20260709 13:18:05.141983  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:05.145174  6596 rpcz_store.cc:269] 0709 13:18:02.975866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.040616 (+2064750us) service_pool.cc:224] Handling call
0709 13:18:05.140806 (+100190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.145520  7231 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:05.141304  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:05.142418  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1142) took 1215 ms. Trace:
I20260709 13:18:05.143801  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:05.144344 10129 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:05.144855 10251 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:05.147580  6664 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
W20260709 13:18:05.155153  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 134815us
I20260709 13:18:05.149001  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:05.156656  6637 leader_election.cc:336] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.662s (SENT)
I20260709 13:18:05.156939  7232 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:05.149081  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.150988  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.158209  6669 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:05.151739  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.152441  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.159437  6665 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:05.160321  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.146663  7228 rpcz_store.cc:276] 0709 13:18:03.926732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.965720 (+1038988us) service_pool.cc:224] Handling call
0709 13:18:04.966031 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:04.966348 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.966374 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:04.966864 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:18:05.142105 (+175241us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:05.142142 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.142202 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:05.142239 (+    37us) raft_consensus.cc:1685] finished
0709 13:18:05.142254 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.142361 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6196,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":174899,"threads_started":1,"wal-append.queue_time_us":171937}
I20260709 13:18:05.167426  6667 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.153832  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.154608 10268 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.157023  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.167240  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:05.167891  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:05.141319 10266 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:05.171506  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:05.179733  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1141) took 1253 ms. Trace:
I20260709 13:18:05.171882  6665 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:05.175455  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:05.116505 10295 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:05.102936 10253 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:05.174163 10268 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:05.178442  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:05.190956 10049 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:05.191421  7234 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:05.182950 10357 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:05.192499  7234 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:05.179198  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 694) took 3525 ms. Trace:
I20260709 13:18:05.180084  7237 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:05.141366 10280 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:05.194770 10251 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:05.194856  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 122899us
I20260709 13:18:05.195382  6597 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:05.156250  6664 pending_rounds.cc:85] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 3
I20260709 13:18:05.195935  6664 pending_rounds.cc:107] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:18:05.196273  6664 raft_consensus.cc:2889] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.197480  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.197647 10184 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.198355  6670 raft_consensus.cc:2410] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:05.116708 10252 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.183226  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.199913  6669 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:05.193362  6668 rpcz_store.cc:276] 0709 13:18:01.653854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.654054 (+   200us) service_pool.cc:224] Handling call
0709 13:18:01.669074 (+ 15020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:01.676545 (+  7471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.676583 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:01.701426 (+ 24843us) log.cc:844] Serialized 26 byte log entry
0709 13:18:01.879755 (+178329us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:01.929792 (+ 50037us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.930417 (+   625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.931404 (+   987us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.167816 (+1236412us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b240021b5e8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5e1a9c4 00007f04b5e18acb 00007f04b5fd3dc9 00007f04b5fd3a55 00007f04b5fd309d 00007f04b8c0510e 00007f04b8c1c88c 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221
0709 13:18:05.154960 (+1987144us) raft_consensus.cc:1685] finished
0709 13:18:05.178916 (+ 23956us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.179123 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":939,"prepare.queue_time_us":151455,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":124,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":53812096,"thread_start_us":106787,"threads_started":2,"wal-append.queue_time_us":244147}
I20260709 13:18:05.199000  6514 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:05.201159 10357 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:05.201227 10252 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.146135 10281 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:05.146431 10293 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.155340 10284 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:05.202304 10357 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.203963  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:05.204705  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 838) took 2113 ms (client timeout 2229 ms). Trace:
W20260709 13:18:05.205039  6591 rpcz_store.cc:269] 0709 13:18:03.090880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.203711 (+2112831us) service_pool.cc:224] Handling call
0709 13:18:05.204632 (+   921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.177953  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.181897  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.178212  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:05.195574  6486 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.166s (SENT)
I20260709 13:18:05.190864  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:05.185834  6637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.675s (SENT) [suppressed 69 similar messages]
I20260709 13:18:05.206661  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:05.206955  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.202432  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 715) took 2215 ms. Trace:
I20260709 13:18:05.166838 10262 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 13:18:05.208083  6637 leader_election.cc:336] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.707s (SENT)
W20260709 13:18:05.210983  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 67256us
W20260709 13:18:05.211445  6637 leader_election.cc:336] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.765s (SENT)
I20260709 13:18:05.211896  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.205904  7229 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.187726  6663 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:05.209024  7228 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:05.209793  6666 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.207830  6516 rpcz_store.cc:276] 0709 13:18:02.986799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.455682 (+468883us) service_pool.cc:224] Handling call
0709 13:18:03.455927 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.776250 (+320323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.202197 (+1425947us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b5400216bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:05.202369 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733047168}
I20260709 13:18:05.214708  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:05.217041  6513 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 2.290s	user 0.018s	sys 0.031s
I20260709 13:18:05.217419  6513 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:05.217589  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:05.217947  6513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 714) took 2320 ms (client timeout 1626 ms). Trace:
I20260709 13:18:05.218393  6667 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:05.212842  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.194060  6561 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2407)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
W20260709 13:18:05.218389  6513 rpcz_store.cc:269] 0709 13:18:02.897523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.902224 (+  4701us) service_pool.cc:224] Handling call
0709 13:18:05.217885 (+2315661us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:05.220498  7199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.034s	user 0.020s	sys 0.011s
I20260709 13:18:05.220360  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:05.221028  6513 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.221331  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.182824 10129 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:05.222049  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.231570  6668 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:18:05.232100  6637 leader_election.cc:336] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.790s (SENT)
I20260709 13:18:05.222779  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:05.232380  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.232755  7233 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
W20260709 13:18:05.232957  6637 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.854s (SENT)
I20260709 13:18:05.223428 10258 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:05.222261 10347 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:18:05.180099  7230 rpcz_store.cc:276] 0709 13:18:03.926487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.898051 (+971564us) service_pool.cc:224] Handling call
0709 13:18:04.898367 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:04.899859 (+  1492us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:04.899888 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:04.900296 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 13:18:05.179179 (+278883us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:05.179213 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.179386 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:05.179435 (+    49us) raft_consensus.cc:1685] finished
0709 13:18:05.179445 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.179681 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1157,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":278521,"threads_started":1,"wal-append.queue_time_us":271356}
I20260709 13:18:05.205900 10331 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:05.259609  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
W20260709 13:18:05.226316  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2396) took 2331 ms (client timeout 2425 ms). Trace:
I20260709 13:18:05.233404  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.191074 10268 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.239481  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.216405 10266 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:05.237098  6590 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:05.261833  7228 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:05.221922 10260 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:05.276463  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:05.282657  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2189) took 2219 ms (client timeout 1978 ms). Trace:
W20260709 13:18:05.283119  6590 rpcz_store.cc:269] 0709 13:18:03.063148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.025853 (+1962705us) service_pool.cc:224] Handling call
0709 13:18:05.282554 (+256701us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:05.264091  6596 rpcz_store.cc:269] 0709 13:18:02.895529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.145909 (+2250380us) service_pool.cc:224] Handling call
0709 13:18:05.226244 (+ 80335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.273753  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.216848 10295 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:05.258997 10253 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:05.237226 10049 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:05.263816  6513 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:05.226095 10368 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:05.233646 10281 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.236780 10287 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:18:05.287861  7232 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:05.275749  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.195868 10251 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.259955 10258 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:05.291432  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:05.292235  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:05.292497  7229 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:18:05.295346  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62403us
I20260709 13:18:05.297039  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:05.273365 10348 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:05.285190 10201 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:05.331416  6637 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.941s (SENT)
I20260709 13:18:05.298259  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.216816 10252 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:05.271888 10341 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.431s	user 0.019s	sys 0.048s
I20260709 13:18:05.276575  9925 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:05.305930  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.309219  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:05.261224 10263 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:18:05.321552 10266 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.306383 10281 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.276415 10282 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:05.328675  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.290264 10253 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:05.296659 10368 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.329599 10348 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:05.336750  6667 raft_consensus.cc:2410] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:05.335268  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.290103 10331 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:18:05.340727  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.344244  6668 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:05.358644  6662 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:18:05.348505  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.360417  6451 debug-util.cc:398] Leaking SignalData structure 0x7b0800363600 after lost signal to thread 6546
I20260709 13:18:05.345414 10281 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.333647 10201 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.272395 10129 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.336244 10270 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:05.348606  6637 leader_election.cc:336] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.979s (SENT)
I20260709 13:18:05.387909  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:05.360816  6451 debug-util.cc:398] Leaking SignalData structure 0x7b080031e000 after lost signal to thread 6547
W20260709 13:18:05.391671  6451 debug-util.cc:398] Leaking SignalData structure 0x7b0800387520 after lost signal to thread 6621
W20260709 13:18:05.391954  6451 debug-util.cc:398] Leaking SignalData structure 0x7b0800350e00 after lost signal to thread 6622
W20260709 13:18:05.392985  6451 debug-util.cc:398] Leaking SignalData structure 0x7b08003a3e80 after lost signal to thread 6695
W20260709 13:18:05.393247  6451 debug-util.cc:398] Leaking SignalData structure 0x7b08001501c0 after lost signal to thread 6696
W20260709 13:18:05.394330  6451 debug-util.cc:398] Leaking SignalData structure 0x7b0800342220 after lost signal to thread 7261
I20260709 13:18:05.342736 10287 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:05.366271  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.363106  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55990 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.350286 10253 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:05.374086  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.356951 10248 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.355502 10368 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:05.431675  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2400) took 2535 ms (client timeout 2531 ms). Trace:
W20260709 13:18:05.432085  6590 rpcz_store.cc:269] 0709 13:18:02.896592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.431501 (+2534909us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:05.431616 (+   115us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:18:05.433020  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.436497 10248 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.437830 10293 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:05.439678 10282 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:05.440986  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:05.442726  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:05.401311  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 52756.5us
W20260709 13:18:05.454165  6637 leader_election.cc:336] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.139s (SENT)
I20260709 13:18:05.454748  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.394968  6667 raft_consensus.cc:2376] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 9. Re-sending same reply.
I20260709 13:18:05.387795  9873 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:05.387856 10201 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.402810 10247 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:05.403249 10287 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:05.404119 10270 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:05.359953 10359 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.366989 10276 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:05.409341  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.540323  6662 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:05.358667 10184 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.541052 10248 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:05.418221  6451 debug-util.cc:398] Leaking SignalData structure 0x7b08000ea620 after lost signal to thread 7262
I20260709 13:18:05.422272  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.422855  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.423054  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:05.549103  6451 debug-util.cc:398] Leaking SignalData structure 0x7b0800387d40 after lost signal to thread 10290
I20260709 13:18:05.423316  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.423668  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.550300  6663 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.423988  6668 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
W20260709 13:18:05.424922 10381 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:18:05.424808  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:05.429086 10348 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.431329  6665 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:18:05.572858  6665 pending_rounds.cc:85] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 4
I20260709 13:18:05.573123  6665 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:18:05.573345  6665 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.573632  6665 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:18:05.573849  6665 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.514524  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.534991  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:05.396030  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55990 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
W20260709 13:18:05.536507  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.536455  6516 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:18:05.546656 10251 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:18:05.548760  6671 raft_consensus.cc:2376] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 9. Re-sending same reply.
I20260709 13:18:05.549211  6666 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:05.550637  6668 pending_rounds.cc:85] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 3
I20260709 13:18:05.576572  6668 pending_rounds.cc:107] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:05.576820  6668 raft_consensus.cc:2889] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.554101  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:05.576965  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 122796us
I20260709 13:18:05.553445  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 720) took 1688 ms. Trace:
W20260709 13:18:05.577611  6637 leader_election.cc:336] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.169s (SENT)
I20260709 13:18:05.577539  6522 rpcz_store.cc:276] 0709 13:18:03.865086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.865794 (+   708us) service_pool.cc:224] Handling call
0709 13:18:05.553288 (+1687494us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b5400355054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:05.553390 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77495680}
I20260709 13:18:05.546998  9925 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:05.557149  6662 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:05.576311  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55990 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.574963  6664 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:05.559020  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.515378 10332 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.567785 10258 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.571694  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.575598  6515 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 6: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:05.585127  6667 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:05.358829 10295 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.578128  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.579062  7231 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:18:05.586884  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:05.580382  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.580448  6669 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.581816  6670 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:05.560026  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.588050  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.588574  6663 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:18:05.570791 10365 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:18:05.581642  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55990 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
W20260709 13:18:05.594085  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46160.7us
W20260709 13:18:05.597110  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.594029  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.598030  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.598863  6670 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:05.599078  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.599673  6663 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:18:05.600023  6637 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.454s (SENT)
I20260709 13:18:05.600478  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.602670  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.603724  6669 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:05.604976  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.606305 10270 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:05.598091 10247 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:05.608238  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.608829  6663 raft_consensus.cc:2376] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9. Re-sending same reply.
I20260709 13:18:05.610486  9873 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.610564  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:05.611173  6590 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:05.611040  9873 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:05.611944 10347 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:05.580793 10276 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.613056  9873 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:05.614334 10272 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:05.614490 10276 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.614790 10272 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:05.615984  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:05.617206  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:05.617475 10093 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:05.617938  6519 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:05.585881 10359 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:05.619128  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2186) took 2760 ms (client timeout 2177 ms). Trace:
W20260709 13:18:05.619539  6597 rpcz_store.cc:269] 0709 13:18:02.858808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.925330 (+ 66522us) service_pool.cc:224] Handling call
0709 13:18:05.618949 (+2693619us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b5400080a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:05.619064 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5129185152}
I20260709 13:18:05.620770 10359 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:05.620888  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2194) took 1975 ms (client timeout 1729 ms). Trace:
W20260709 13:18:05.621354  6596 rpcz_store.cc:269] 0709 13:18:03.645208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.620715 (+1975507us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:05.620828 (+   113us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:05.610324  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.615676 10272 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.624511  6595 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.603430 10348 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:05.634182  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.636184  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.637383  9925 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:05.637633 10331 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:05.638821  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:05.642431  6590 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:18:05.643957  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2195) took 1938 ms (client timeout 2207 ms). Trace:
W20260709 13:18:05.644958  6590 rpcz_store.cc:269] 0709 13:18:03.705079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.610309 (+1905230us) service_pool.cc:224] Handling call
0709 13:18:05.643893 (+ 33584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.641734 10331 consensus_queue.cc:1243] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:05.601444  6561 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55990 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.652307  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.644989  6486 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.647804 10244 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:05.679787  6595 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:05.601084 10332 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:05.689536  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 89525.7us
I20260709 13:18:05.686362  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.685390  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.693919  6671 raft_consensus.cc:2376] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11. Re-sending same reply.
I20260709 13:18:05.694147  6670 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.651842 10272 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:05.705229  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.705718  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.708633 10347 consensus_queue.cc:1243] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:05.638536 10261 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:18:05.712294  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.712211  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.714917  6670 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:05.713388  6522 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:05.714654  6671 raft_consensus.cc:2376] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11. Re-sending same reply.
I20260709 13:18:05.695430 10244 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:05.719926  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:05.720631  6590 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.680874 10252 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:18:05.697444 10257 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:05.725045  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.686160 10169 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:18:05.728113  6670 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:18:05.728539  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 132148us
I20260709 13:18:05.688789 10268 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.729355  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.735653 10395 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:05.739630  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.740793  6671 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.744757  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 719) took 2142 ms. Trace:
I20260709 13:18:05.745152  6518 rpcz_store.cc:276] 0709 13:18:03.602200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.610505 (+  8305us) service_pool.cc:224] Handling call
0709 13:18:03.615266 (+  4761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.703671 (+ 88405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.744538 (+2040867us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b540014dcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:05.744700 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201988864}
I20260709 13:18:05.740347  6670 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 4.
W20260709 13:18:05.748162  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 103274us
I20260709 13:18:05.662573 10287 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.751842  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.753947 10251 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:05.754372 10251 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:05.754797 10251 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.758810  6520 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:05.760746 10332 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.730391 10257 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:05.762895 10253 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.763334 10251 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.765590 10251 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.765655 10253 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.767923  6486 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.771790  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.772266  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.772568  6671 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.773541  6590 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
W20260709 13:18:05.774150  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2198) took 1926 ms (client timeout 2192 ms). Trace:
I20260709 13:18:05.747351 10368 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.776430 10281 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.777915 10368 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:05.778573 10281 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:05.780118  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:05.783794  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 839) took 2692 ms (client timeout 1813 ms). Trace:
I20260709 13:18:05.783838  6670 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
W20260709 13:18:05.784348  6594 rpcz_store.cc:269] 0709 13:18:03.091174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.783619 (+2692445us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:05.783733 (+   114us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:05.785363  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.786140  6670 raft_consensus.cc:2410] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 8 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:18:05.781842  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.771066 10257 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.787278  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1016) took 1877 ms. Trace:
I20260709 13:18:05.787640  7235 rpcz_store.cc:276] 0709 13:18:03.872017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.872121 (+   104us) service_pool.cc:224] Handling call
0709 13:18:03.872411 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.533645 (+1661234us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b18003f8200. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b9079bfb 00007f04b908464b
0709 13:18:05.533788 (+   143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.533813 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:05.546435 (+ 12622us) log.cc:844] Serialized 26 byte log entry
0709 13:18:05.716724 (+170289us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:05.716759 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.716999 (+   240us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:05.737678 (+ 20679us) raft_consensus.cc:1685] finished
0709 13:18:05.737698 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:05.748918 (+ 11220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85458,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":6255,"spinlock_wait_cycles":33652992,"thread_start_us":198859,"threads_started":2,"wal-append.queue_time_us":139931}
I20260709 13:18:05.791006 10253 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.785987  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.738801 10244 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:05.801568 10252 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:05.803113  9899 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:05.803524  9899 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:05.805826  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.811004  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.811769  6670 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:05.774535  6590 rpcz_store.cc:269] 0709 13:18:03.847867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.682237 (+1834370us) service_pool.cc:224] Handling call
0709 13:18:05.774085 (+ 91848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:05.813921  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:05.824385  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.800410 10161 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:05.841728  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2392) took 2949 ms (client timeout 1707 ms). Trace:
I20260709 13:18:05.798848 10253 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:05.798161 10244 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:05.842206  6589 rpcz_store.cc:269] 0709 13:18:02.892225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.895588 (+  3363us) service_pool.cc:224] Handling call
0709 13:18:05.839889 (+2944301us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54003a43d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:05.841651 (+  1762us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4578258176}
I20260709 13:18:05.848778  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.831672 10266 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:05.830897 10169 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:05.883700  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.884511  6671 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.884574 10374 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:05.885442  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.886479  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.887173 10201 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.887835  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.888411  6671 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:18:05.889138  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.827250 10251 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:18:05.889833  6671 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:05.832814 10252 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:18:05.846336 10049 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:05.726178  6563 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
W20260709 13:18:05.898123  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 165515us
I20260709 13:18:05.899555  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:05.900107  6522 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:05.893522  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:05.900991  6563 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items [suppressed 12 similar messages]
I20260709 13:18:05.849243 10253 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.908514  6671 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:18:05.901337  6563 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.914816  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.915930  6671 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:18:05.804313  9899 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.920455  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.924959  6667 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:18:05.897590 10161 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.879683 10169 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:05.915452  6563 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.907976 10049 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.909207 10266 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:05.927157 10368 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:18:05.933496  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.934211  6671 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:05.928438  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:05.936194  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 845) took 2813 ms (client timeout 2019 ms). Trace:
W20260709 13:18:05.936584  6593 rpcz_store.cc:269] 0709 13:18:03.123009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.936017 (+2813008us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:05.936133 (+   116us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:05.936924  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:05.935173 10049 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.942857 10393 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.946434  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.946695  9899 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:05.947193  6665 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.947657 10201 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.947973  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.949062  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.949533  6665 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:05.950239  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.950853  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:05.969300  6592 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:05.970006  6667 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:05.969838  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.954300 10266 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:05.932670 10169 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:05.975729  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.917994 10396 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:05.969305  6665 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:05.933966  6563 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
W20260709 13:18:05.970465  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 138533us
I20260709 13:18:05.979311  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:05.980556  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:05.981577  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:05.984486  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 84770.6us
I20260709 13:18:05.971868 10261 raft_consensus.cc:2764] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:05.985472  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:05.989490  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.989730  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:05.987064  6637 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55990 dropped due to backpressure: service queue is full with 50 items
I20260709 13:18:05.933009 10161 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:05.992728  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:05.993682  6518 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:18:05.994906  6667 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:05.994889  6563 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.990420  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.999859  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:06.001174  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:06.002023  6665 raft_consensus.cc:2410] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:06.003732  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:05.985392 10270 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.998080  6563 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:05.985577 10357 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:06.009034  6488 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:06.010042  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 728) took 1190 ms. Trace:
I20260709 13:18:05.985188 10396 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:05.993431 10359 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:06.012519  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:06.017305  6518 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.991869 10268 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:05.987612 10398 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:05.993559 10368 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:06.010430  6514 rpcz_store.cc:276] 0709 13:18:04.819663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.020078 (+200415us) service_pool.cc:224] Handling call
0709 13:18:05.020687 (+   609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.200943 (+180256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.009810 (+808867us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54003bbad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:06.009986 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409617280}
I20260709 13:18:06.011883  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:06.016222  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:06.014003  6588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2420) took 3030 ms (client timeout 2169 ms). Trace:
W20260709 13:18:06.083187  6588 rpcz_store.cc:269] 0709 13:18:02.976082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.006456 (+3030374us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:06.006566 (+   110us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:06.084117  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.084798  6588 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:06.010913 10261 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:06.010705 10401 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:06.019513  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:06.002555 10276 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:18:06.020160 10334 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.012948  9925 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:18:06.007553 10270 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:06.047843  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.006282  9845 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:06.016360  9873 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.012966 10295 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:06.061470  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.100560 10260 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.100916  6665 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.101094 10260 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.085091 10368 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:06.101653 10368 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:06.103073  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2200) took 1868 ms (client timeout 1732 ms). Trace:
W20260709 13:18:06.103459  6597 rpcz_store.cc:269] 0709 13:18:04.234735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.102891 (+1868156us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:06.103014 (+   123us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:06.104380  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.015858 10049 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.017112 10359 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:06.072813  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2407) took 3174 ms (client timeout 2454 ms). Trace:
I20260709 13:18:05.976468 10253 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.074733  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:06.067589  6563 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 857)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
W20260709 13:18:06.079152  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 89728us
I20260709 13:18:06.113708  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:06.085106  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81429.1us
I20260709 13:18:06.114730  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:06.087411  6517 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:06.104454 10393 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:06.072115 10365 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:06.166442 10365 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:06.167578 10365 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.168905 10365 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.103484  6667 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:06.179945  6662 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:06.181077 10263 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:06.181484 10263 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:06.183079 10262 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:06.183486 10262 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:06.184077  6518 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:06.184326 10262 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:06.185441 10262 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:06.165222  6671 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:18:06.188042  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73419.2us
W20260709 13:18:06.189782  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.189643  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:06.190371  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.191322  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:06.193043  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:06.194099  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:06.109825  6665 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:06.110507  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:06.195329  6514 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:06.111524  6591 rpcz_store.cc:269] 0709 13:18:02.898274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.072648 (+3174374us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:06.072752 (+   104us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:06.115962  6669 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:06.117873  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.196586  6597 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:06.196760  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 860) took 1005 ms. Trace:
I20260709 13:18:06.197106  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2204) took 1334 ms. Trace:
I20260709 13:18:06.197187  6588 rpcz_store.cc:276] 0709 13:18:05.191808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.108785 (+916977us) service_pool.cc:224] Handling call
0709 13:18:06.196702 (+ 87917us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:06.118427  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.078317 10396 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:06.115854  6563 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2205)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859)
I20260709 13:18:06.122479 10253 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.126417  6668 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:06.113142 10359 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:06.127348  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:06.211140  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 83864.9us
I20260709 13:18:06.127708  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.128043 10350 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:06.128592 10257 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.129310  6663 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:18:06.134912  6666 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:18:06.214828  6666 pending_rounds.cc:85] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 9
I20260709 13:18:06.215129  6666 pending_rounds.cc:107] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:18:06.215426  6666 raft_consensus.cc:2889] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:06.136313  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.216241  6514 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:06.216423  6593 raft_consensus.cc:2368] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6.
I20260709 13:18:06.217032  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 850) took 1341 ms. Trace:
I20260709 13:18:06.217255  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:06.217445  6593 rpcz_store.cc:276] 0709 13:18:04.875805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.136064 (+1260259us) service_pool.cc:224] Handling call
0709 13:18:06.216958 (+ 80894us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:06.137158  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2196) took 2354 ms (client timeout 2240 ms). Trace:
I20260709 13:18:06.117103 10334 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.110600 10049 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.141896 10169 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:06.142715  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.143177  6670 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
W20260709 13:18:06.228281  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.194813 10276 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:06.109097 10270 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:06.150090  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.233656  6589 raft_consensus.cc:2376] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7. Re-sending same reply.
I20260709 13:18:06.155524  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:06.234131  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 855) took 1247 ms (client timeout 1517 ms). Trace:
W20260709 13:18:06.234486  6589 rpcz_store.cc:269] 0709 13:18:04.986361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.149759 (+1163398us) service_pool.cc:224] Handling call
0709 13:18:06.234078 (+ 84319us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:06.234532  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 79092us
I20260709 13:18:06.165180  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:06.235741  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 70671.2us
I20260709 13:18:06.197503  6597 rpcz_store.cc:276] 0709 13:18:04.862885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.104104 (+1241219us) service_pool.cc:224] Handling call
0709 13:18:06.197053 (+ 92949us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.206753  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:06.218508  6596 rpcz_store.cc:269] 0709 13:18:03.782815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.136969 (+2354154us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:06.137095 (+   126us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:06.214468  6663 pending_rounds.cc:85] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 7
I20260709 13:18:06.241210  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.241670  6663 pending_rounds.cc:107] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:18:06.219659  9925 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:06.219749  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.220389  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.170203 10260 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.242792  6591 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:06.251489  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 861) took 1059 ms. Trace:
I20260709 13:18:06.200798 10406 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:06.251832  6591 rpcz_store.cc:276] 0709 13:18:05.192079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.205258 (+1013179us) service_pool.cc:224] Handling call
0709 13:18:06.251436 (+ 46178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:06.221865 10407 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.110s	user 0.000s	sys 0.003s
W20260709 13:18:06.223140  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.226815  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.254175  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 866) took 1060 ms. Trace:
I20260709 13:18:06.254635  6588 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:06.255188  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 865) took 1062 ms. Trace:
I20260709 13:18:06.194823 10295 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:06.146953 10368 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.265511  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.267889  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.147873 10161 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.200999  9845 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:06.214339 10272 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:06.241943  6597 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:06.241937  6663 raft_consensus.cc:2889] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:06.255530  6588 rpcz_store.cc:276] 0709 13:18:05.193092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.205314 (+1012222us) service_pool.cc:224] Handling call
0709 13:18:06.255135 (+ 49821us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.226852 10395 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:06.254592  6593 rpcz_store.cc:276] 0709 13:18:05.193330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.218305 (+1024975us) service_pool.cc:224] Handling call
0709 13:18:06.254106 (+ 35801us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:06.256567  6407 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 921c423406da4f289884afca51ca198f (127.5.72.129:34121) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 921c423406da4f289884afca51ca198f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:06.256909  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.287629  6589 raft_consensus.cc:2368] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6.
W20260709 13:18:06.268111  6488 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:55992 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:06.270818  6514 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:06.270427  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:06.271468 10268 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.290377  6517 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:06.285583  6596 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:06.251603 10244 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.254439 10398 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:06.291361  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 863) took 1098 ms. Trace:
I20260709 13:18:06.291924  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 864) took 1099 ms. Trace:
I20260709 13:18:06.300851  6589 rpcz_store.cc:276] 0709 13:18:05.192859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.235341 (+1042482us) service_pool.cc:224] Handling call
0709 13:18:06.291870 (+ 56529us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.300783  6596 rpcz_store.cc:276] 0709 13:18:05.192592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.241493 (+1048901us) service_pool.cc:224] Handling call
0709 13:18:06.291281 (+ 49788us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.302444  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.303519  6596 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:06.255069 10406 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:06.257776 10276 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:06.304725  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.324306  6597 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:06.317811  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 862) took 1125 ms. Trace:
I20260709 13:18:06.265193 10169 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.318461  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.349314  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 858) took 1172 ms. Trace:
I20260709 13:18:06.293557 10398 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.265162 10260 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.322670  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.325380  6591 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:06.324213  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.356494  6588 raft_consensus.cc:2410] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:06.356640  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 869) took 1097 ms. Trace:
I20260709 13:18:06.357128  6589 rpcz_store.cc:276] 0709 13:18:05.259246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.301920 (+1042674us) service_pool.cc:224] Handling call
0709 13:18:06.356566 (+ 54646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.325760  6596 rpcz_store.cc:276] 0709 13:18:05.192336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.302105 (+1109769us) service_pool.cc:224] Handling call
0709 13:18:06.317745 (+ 15640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.250955 10266 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:18:06.287353  9845 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:06.254796 10350 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:18:06.295086 10244 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.349666  6593 rpcz_store.cc:276] 0709 13:18:05.177101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.286813 (+1109712us) service_pool.cc:224] Handling call
0709 13:18:06.349251 (+ 62438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.267601  9925 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:06.257376 10397 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 13:18:06.408279  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 124937us
I20260709 13:18:06.353533  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.388707  9925 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.319236 10406 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:06.290581 10393 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:18:06.416843  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 867) took 1223 ms. Trace:
I20260709 13:18:06.417382  6597 rpcz_store.cc:276] 0709 13:18:05.193579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.324981 (+1131402us) service_pool.cc:224] Handling call
0709 13:18:06.416746 (+ 91765us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:06.360214  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 94490.8us
I20260709 13:18:06.423993  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.323980 10276 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.365836  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:06.379092  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 172365us
I20260709 13:18:06.382207  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.461241  6589 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:06.461319  6596 raft_consensus.cc:2376] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 9. Re-sending same reply.
I20260709 13:18:06.383474 10368 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:06.384462 10266 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:06.386343  9845 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.408399 10244 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.393543 10129 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:06.397492  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.358554  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 848) took 1483 ms. Trace:
W20260709 13:18:06.409970  7200 leader_election.cc:336] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items
I20260709 13:18:06.304286 10268 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.425089  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2429) took 1063 ms. Trace:
I20260709 13:18:06.480355  6593 rpcz_store.cc:276] 0709 13:18:05.361952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.370524 (+1008572us) service_pool.cc:224] Handling call
0709 13:18:06.425018 (+ 54494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.350513 10287 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:06.355611 10398 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:06.461990  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:06.472990  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:06.490080  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.472705  6588 rpcz_store.cc:276] 0709 13:18:04.875199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.292465 (+1417266us) service_pool.cc:224] Handling call
0709 13:18:06.358486 (+ 66021us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.473932  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.490782  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:06.491461  6664 raft_consensus.cc:2410] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:06.477392  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.472820  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:06.484175  6671 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:06.492372  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 82441.6us
I20260709 13:18:06.489279  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:06.493011  7200 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34536 dropped due to backpressure: service queue is full with 50 items
I20260709 13:18:06.493176  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2409) took 3594 ms. Trace:
I20260709 13:18:06.493391  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:06.493526  6596 rpcz_store.cc:276] 0709 13:18:02.899275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.467584 (+3568309us) service_pool.cc:224] Handling call
0709 13:18:06.493124 (+ 25540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.415198 10350 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:06.487625  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.495574  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.501849  7235 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:06.502187  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.502791  6664 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.463274 10276 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.509181  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.509917  7231 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:06.506762  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:06.507407  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.507519  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.502931 10414 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:18:06.519524  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.519770  7235 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:06.520758  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.521749  6588 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:06.522881  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.523563  6588 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.467314 10266 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:06.526769  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.542865  6668 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.529641  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.549575  7231 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:06.553119  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:06.577925  6669 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:06.503216 10368 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:06.555052  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.570107  7236 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:06.538800  6664 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:06.508234 10405 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:06.485572 10295 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:18:06.597784  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.599179  7235 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:06.603895  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.604496  6591 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:06.605547  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.607308  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.607833  6593 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:06.610219  7236 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:06.494529 10129 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:06.613415  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.612174  6521 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:06.595173 10272 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:06.563494 10161 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:06.618564  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:06.621207  6588 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:06.622202  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.622828  6588 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:06.607141  9899 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:06.627036  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 136380us
I20260709 13:18:06.625998  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.627811  6669 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:06.516932 10350 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.630700  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.633610  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.635262  7235 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:06.640409  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.649219  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:06.650139  6516 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:18:06.650997  7200 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 2.019s (SENT)
I20260709 13:18:06.651593  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:06.625638  9873 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
I20260709 13:18:06.669515  6519 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:06.653064  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:06.663503  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.667973  6597 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:06.673462  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:06.666342  6593 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:06.681696  7229 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:06.681922  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:06.685909  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.692359 10270 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:06.694541 10270 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:06.684768 10405 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:06.696316 10405 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:06.697276 10405 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.699205 10268 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:06.703593 10365 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.534s	user 0.013s	sys 0.000s
I20260709 13:18:06.670898  9873 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:06.680112 10400 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.706977 10400 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.707423 10400 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.707942  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2436) took 1497 ms. Trace:
I20260709 13:18:06.708333  6521 rpcz_store.cc:276] 0709 13:18:05.210368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.938102 (+727734us) service_pool.cc:224] Handling call
0709 13:18:05.945185 (+  7083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:06.616848 (+671663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.707703 (+ 90855us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54002157d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:06.707886 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533330176}
I20260709 13:18:06.709231 10295 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:06.623911 10161 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.710028 10295 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:06.711063 10257 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.712323 10161 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.712849 10295 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:06.713814 10334 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:06.715895 10129 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:06.669808 10348 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:06.714012 10258 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:06.735734  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.736871  6588 raft_consensus.cc:2368] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:06.737761  6589 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:06.739566  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.745800  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.642429  9899 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.722209  7229 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:06.746780  6589 raft_consensus.cc:2368] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
W20260709 13:18:06.750125  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.750146  6669 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:06.616957 10266 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.771056  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.771782  7235 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:06.771833  6669 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:06.775935  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.741503 10280 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:06.718511  7200 leader_election.cc:400] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6.
I20260709 13:18:06.781196  7200 leader_election.cc:403] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:06.785853  6669 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:06.776037  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:06.781945  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 90863.3us
W20260709 13:18:06.787137  6595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2185) took 4034 ms (client timeout 2140 ms). Trace:
W20260709 13:18:06.787887  6595 rpcz_store.cc:269] 0709 13:18:02.752427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.752625 (+   198us) service_pool.cc:224] Handling call
0709 13:18:05.424669 (+2672044us) spinlock_profiling.cc:243] Waited 6.35 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:18:06.786972 (+1362303us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b540019d7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:06.787072 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183734016}
I20260709 13:18:06.792268  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.785192 10266 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:06.795495  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:06.787627 10348 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6.
I20260709 13:18:06.798914  6637 leader_election.cc:400] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:06.800369  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2076) took 1647 ms. Trace:
I20260709 13:18:06.804682  6637 leader_election.cc:400] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:06.800843  6520 rpcz_store.cc:276] 0709 13:18:05.152727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.155020 (+  2293us) service_pool.cc:224] Handling call
0709 13:18:05.155317 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.759295 (+603978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.800150 (+1040855us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b540007d5d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:06.800312 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387454208}
I20260709 13:18:06.812800  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:06.789669 10287 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:06.819199  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.820227 10049 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.802925  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:06.822381  6668 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:06.796427 10425 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.818328 10374 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.827201  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.797461 10334 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:06.839322  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:06.821386 10359 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.591s	user 0.005s	sys 0.005s
I20260709 13:18:06.837905  6486 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:06.840709  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:06.841992  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.843117  6595 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:06.845001  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.845134  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:06.841900  6486 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:06.819132 10247 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:06.845618  6595 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:06.847067  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:06.826766  9925 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:06.848605  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.842111 10359 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.850427  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:06.854560  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:06.896296  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49297.2us
I20260709 13:18:06.855050  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:06.898419  6591 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:06.844558 10268 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:06.904721  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54377.4us
I20260709 13:18:06.904676  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:06.907109  6591 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:18:06.909046  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:06.909840 10348 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.910109  6591 raft_consensus.cc:606] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:06.916162  6591 raft_consensus.cc:614] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:06.913441  6668 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:06.919030  6593 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 5: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:06.840673 10374 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:06.927346  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:06.919291  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:06.928471  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:06.920682  6668 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:06.929091  6521 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:06.844537 10262 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:06.921846 10430 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:06.934157  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:06.934417  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:06.935338  6668 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:18:06.922308 10419 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:18:06.828258 10049 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:06.931082  7235 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:06.942194  7235 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:06.921865 10421 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:18:06.901021 10169 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:06.921763 10420 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:18:06.935737  6668 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:06.903726 10411 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.933912 10334 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.916970 10348 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:06.940428 10410 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.475s	user 0.001s	sys 0.000s
I20260709 13:18:06.947538  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 22) took 3192 ms. Trace:
I20260709 13:18:06.840526 10257 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:06.851645 10247 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.949412  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:06.848425 10396 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:06.911314 10428 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:06.949131 10414 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:06.943127  7235 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.990969 10365 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.910470 10424 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:18:06.840014 10425 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.991855  6668 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:06.924551 10287 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.005357  6591 rpcz_store.cc:276] 0709 13:18:03.725377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.908899 (+3183522us) service_pool.cc:224] Handling call
0709 13:18:06.918103 (+  9204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:06.947887 10262 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.030748  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:06.993254 10129 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:18:07.010382  7235 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:06.971953  6636 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.516s (SENT)
I20260709 13:18:06.994325 10169 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.043360  6668 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:07.017803  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 115946us
W20260709 13:18:07.005152  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 102571us
I20260709 13:18:07.007100  7230 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:07.006562 10257 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:07.018222  9845 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.001s
I20260709 13:18:07.015484 10396 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:07.031973  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 856) took 1884 ms (client timeout 1506 ms). Trace:
I20260709 13:18:07.032648 10429 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:07.032531 10262 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.033636  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:18:07.034096 10374 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:07.035564 10247 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.032573 10416 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:18:07.017308 10365 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.041543  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:07.042582  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.045867  6594 rpcz_store.cc:269] 0709 13:18:05.143996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.028303 (+1884307us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:07.028501 (+   198us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:07.053946  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:07.005921 10411 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.018332 10414 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:07.008273 10334 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.077709  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:07.075500 10260 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:18:07.085793 10431 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:18:07.052697  9845 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.087560 10439 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:07.088097 10169 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:07.090080 10435 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:18:07.091938 10436 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:07.093057  6668 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:07.064498 10251 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:18:07.119045 10260 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:07.119758 10295 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:07.121050  9845 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.121328  6590 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:07.121872 10411 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.122241 10049 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:07.122884  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 150980us
I20260709 13:18:07.124500  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:07.049238 10257 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:07.126538  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 913342us
I20260709 13:18:07.129305  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:07.130474  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:07.103968  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.103488  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.132179  6594 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:07.102404 10268 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.113300  6595 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
W20260709 13:18:07.113673  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.115017  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.099788 10406 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.001s
I20260709 13:18:07.096275 10365 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:07.098539 10348 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:07.139297  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:07.106006  7235 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:07.146528  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.134843 10439 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:18:07.147181  6594 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:07.141633 10281 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:07.142127 10393 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:07.155694 10441 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:07.140035 10260 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.148691 10428 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:07.170185 10405 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:07.142906 10332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.232s	user 0.019s	sys 0.010s
I20260709 13:18:07.173372  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:07.180364  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.139701 10406 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.170161 10347 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:18:07.150918 10365 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:18:07.175354  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:07.100486 10414 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.144826 10429 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:07.177204  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.177969 10428 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.182665  7237 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:18:07.182855  7230 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:07.182983  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79193.7us
I20260709 13:18:07.183293  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.205695  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.181092  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.164487 10281 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:07.171113 10425 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.178764 10368 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:18:07.187750 10443 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.187829  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.146096 10396 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:07.190757 10395 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:18:07.205174 10334 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:07.216696  7232 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:07.229413  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:07.190948 10260 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:07.193138 10276 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.000s
I20260709 13:18:07.231485  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:07.231707 10276 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:07.179595 10332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.282s	user 0.020s	sys 0.011s
I20260709 13:18:07.232245  7234 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:07.233237  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.234045  7234 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:18:07.171875 10441 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.235071 10359 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.235702 10276 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:07.235314 10441 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.236716  7232 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:07.236769 10441 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:07.237164 10403 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.126s	user 0.004s	sys 0.014s
I20260709 13:18:07.173393  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:07.203660  6592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2424) took 4226 ms (client timeout 1881 ms). Trace:
I20260709 13:18:07.206323  6595 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:07.240034  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:07.240602  6595 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:07.185428  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56929.9us
I20260709 13:18:07.215335  7235 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:07.189127 10263 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.216399  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.241536  6589 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:07.238878  6592 rpcz_store.cc:269] 0709 13:18:02.977031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.883361 (+906330us) service_pool.cc:224] Handling call
0709 13:18:07.203426 (+3320065us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54002f6cd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.203571 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4730162560}
I20260709 13:18:07.221101  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.242082  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:07.230181 10400 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 13:18:07.222595  6522 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:18:07.191206 10244 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.702s	user 0.008s	sys 0.003s
I20260709 13:18:07.251528  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:07.217561 10262 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.239068  7228 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:07.217612  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:07.241806  7233 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:07.223102 10397 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:07.275262  7234 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:18:07.208989 10268 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:07.247406  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.232187  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:07.277787  6593 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:07.278307  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2410) took 4378 ms. Trace:
I20260709 13:18:07.278424  6588 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:07.278653  6593 rpcz_store.cc:276] 0709 13:18:02.899592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.277169 (+4377577us) service_pool.cc:224] Handling call
0709 13:18:07.277500 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.278122 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.278263 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.279054  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2411) took 4379 ms. Trace:
I20260709 13:18:07.279368  6588 rpcz_store.cc:276] 0709 13:18:02.899866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.277982 (+4378116us) service_pool.cc:224] Handling call
0709 13:18:07.278248 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.278852 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.279005 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.279970  6593 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:07.280100  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:07.280558  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2426) took 4303 ms. Trace:
I20260709 13:18:07.280795  6588 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:07.281297  6521 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:18:07.281358  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:07.281625  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 833) took 4303 ms. Trace:
I20260709 13:18:07.281764  6521 pending_rounds.cc:85] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 8
I20260709 13:18:07.282369  6521 pending_rounds.cc:107] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:18:07.282668  6521 raft_consensus.cc:2889] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:07.280927  6593 rpcz_store.cc:276] 0709 13:18:02.977565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.279574 (+4302009us) service_pool.cc:224] Handling call
0709 13:18:07.279827 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.280348 (+   521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.280505 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.284386 10350 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:07.284741  6593 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:07.285517  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 834) took 4306 ms. Trace:
I20260709 13:18:07.285807  6593 rpcz_store.cc:276] 0709 13:18:02.986269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.284339 (+4298070us) service_pool.cc:224] Handling call
0709 13:18:07.284579 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.285122 (+   543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.285476 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.282316  6588 rpcz_store.cc:276] 0709 13:18:02.978399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.280143 (+4301744us) service_pool.cc:224] Handling call
0709 13:18:07.280570 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.281314 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.281557 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.289992  6588 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.286914  6593 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:07.250576 10161 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.257532  6592 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:07.291067  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 835) took 4201 ms. Trace:
I20260709 13:18:07.291297  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:07.257249 10244 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.259141 10263 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:07.260296  7235 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:07.260762 10332 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:07.262089 10049 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:07.265440  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2439) took 1273 ms. Trace:
I20260709 13:18:07.301476  6522 rpcz_store.cc:276] 0709 13:18:05.991802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.992245 (+   443us) service_pool.cc:224] Handling call
0709 13:18:06.009826 (+ 17581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.243226 (+1233400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.265126 (+ 21900us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400355f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:07.265362 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2788208000}
I20260709 13:18:07.208865 10281 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:07.205420 10428 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:07.265468  6664 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:07.308166  6664 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:07.252664  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:07.309258  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:07.309774  7233 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:07.309736  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 57139.9us
I20260709 13:18:07.310534  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:07.312633  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:07.313151  6669 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:07.274648  7237 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
W20260709 13:18:07.254556  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.291425  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2404) took 4393 ms. Trace:
I20260709 13:18:07.291571  6588 rpcz_store.cc:276] 0709 13:18:03.089869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.289582 (+4199713us) service_pool.cc:224] Handling call
0709 13:18:07.289846 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.290833 (+   987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.291021 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.315531  6592 rpcz_store.cc:276] 0709 13:18:02.897523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.257054 (+4359531us) service_pool.cc:224] Handling call
0709 13:18:07.257361 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.291217 (+ 33856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.291380 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.257953  7229 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:18:07.292529  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2188) took 4229 ms. Trace:
I20260709 13:18:07.292701  6518 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:07.308931  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.316584  6593 rpcz_store.cc:276] 0709 13:18:03.062776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.286486 (+4223710us) service_pool.cc:224] Handling call
0709 13:18:07.286753 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.292231 (+  5478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.292465 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.293154  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.293741  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:07.279299  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:07.271087 10398 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.742s	user 0.006s	sys 0.005s
W20260709 13:18:07.262241 10382 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.872s	user 0.003s	sys 0.013s
I20260709 13:18:07.302366  6596 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.324656  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2406) took 4426 ms. Trace:
I20260709 13:18:07.215989 10448 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 13:18:07.325155  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:07.325341 10270 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:07.325775  6662 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:07.327328  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:07.328423  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:07.325009  6596 rpcz_store.cc:276] 0709 13:18:02.898011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.272936 (+4374925us) service_pool.cc:224] Handling call
0709 13:18:07.302134 (+ 29198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.324421 (+ 22287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.324600 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.329982  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.330964  6596 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:07.332470  7233 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:07.289294 10262 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.316154  6588 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 9. Ops: []
I20260709 13:18:07.317696  6513 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:07.318807  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:07.336544  6671 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:07.319795  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.338706  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.339326  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.275089 10397 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:07.319456  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.308929  6664 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.334607  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2190) took 3721 ms. Trace:
I20260709 13:18:07.341413  6664 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:07.341106  6596 rpcz_store.cc:276] 0709 13:18:03.612724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.333543 (+3720819us) service_pool.cc:224] Handling call
0709 13:18:07.333944 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.334185 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.334224 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:07.334282 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:07.334327 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.334398 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.334540 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.301160 10332 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:07.304885 10049 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.343210  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 836) took 4253 ms. Trace:
I20260709 13:18:07.346237  6588 rpcz_store.cc:276] 0709 13:18:03.090233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.315750 (+4225517us) service_pool.cc:224] Handling call
0709 13:18:07.316002 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.335767 (+ 19765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.343153 (+  7386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.347420  6588 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:07.347927  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2192) took 3731 ms. Trace:
I20260709 13:18:07.348206  6588 rpcz_store.cc:276] 0709 13:18:03.616559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.346976 (+3730417us) service_pool.cc:224] Handling call
0709 13:18:07.347244 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.347739 (+   495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.347883 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.349291  6588 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:07.349807  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2193) took 3733 ms. Trace:
I20260709 13:18:07.350095  6588 rpcz_store.cc:276] 0709 13:18:03.616828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.348894 (+3732066us) service_pool.cc:224] Handling call
0709 13:18:07.349140 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.349589 (+   449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.349756 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:07.304016  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 60403.7us
I20260709 13:18:07.362972  6669 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:07.353855  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2197) took 3566 ms. Trace:
I20260709 13:18:07.363860  6588 rpcz_store.cc:276] 0709 13:18:03.787322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.352777 (+3565455us) service_pool.cc:224] Handling call
0709 13:18:07.353226 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.353552 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.353577 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:07.353617 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:07.353647 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.353705 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.353804 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.365794  6588 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:07.366595  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.368772  6671 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 5.
I20260709 13:18:07.355845  7237 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:07.331156 10436 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:07.354668  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.361284  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:07.359081  6664 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:07.371918  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2199) took 3480 ms. Trace:
I20260709 13:18:07.330451 10276 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:07.270848 10400 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.377628  6588 rpcz_store.cc:276] 0709 13:18:03.891869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.365201 (+3473332us) service_pool.cc:224] Handling call
0709 13:18:07.365575 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.371585 (+  6010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.371845 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.381014  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:07.330209 10438 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:07.240461 10443 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.382478  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:07.390707  7233 raft_consensus.cc:2376] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10. Re-sending same reply.
I20260709 13:18:07.301247 10161 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.355051 10049 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:07.355095 10332 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:07.396733  6515 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:07.370908 10280 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.374336 10374 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.408630  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:07.375222 10436 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.414177  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.414990  6520 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:07.331837 10398 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.386660 10270 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:07.389968 10359 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:07.422180  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.423981 10359 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.424561 10359 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:07.425249  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.287585 10429 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.351218 10401 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:18:07.447276  6515 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:07.447944  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:07.453693  7233 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:07.372923 10257 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.455768  6671 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:18:07.456037  6669 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:07.456995  6671 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:07.457518  6664 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:07.464255  7236 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:07.413626 10261 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:07.466593  6664 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:07.447481  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:07.467661  6662 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:07.417973 10280 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.421643 10374 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:07.475411 10450 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:18:07.475381 10448 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:07.413343 10436 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:07.419514 10397 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.469192 10261 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:07.552837  9925 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.632608  9925 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.632980 10261 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.634676  9925 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.608584  6671 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.609658 10443 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:07.610317 10280 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:07.610949  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 163519us
I20260709 13:18:07.635696 10397 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:07.636814  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:07.634344 10261 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:07.638968  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:07.639757 10448 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:07.640074  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:07.640444 10448 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.641160 10440 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:07.641462  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:07.611917 10374 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.642278 10447 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:07.642854 10447 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.643339  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:07.643455 10268 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:07.643867  6590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2397) took 4748 ms (client timeout 3669 ms). Trace:
I20260709 13:18:07.644063 10268 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:07.644317  6590 rpcz_store.cc:269] 0709 13:18:02.895818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.127480 (+3231662us) service_pool.cc:224] Handling call
0709 13:18:07.643673 (+1516193us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54002fbcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.643808 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087334272}
I20260709 13:18:07.644893 10263 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:07.641755 10440 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:07.644942 10268 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.645985 10263 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:07.646368 10263 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:07.646304 10440 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:07.614596  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.410598 10169 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:18:07.652951  6593 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.620767 10398 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:07.625675  6514 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:18:07.653785  6514 pending_rounds.cc:85] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 7
I20260709 13:18:07.654093  6514 pending_rounds.cc:107] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:07.654399  6514 raft_consensus.cc:2889] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:07.625789 10244 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:07.628486  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.661528  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.631704 10453 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.157s	user 0.004s	sys 0.000s
I20260709 13:18:07.645598  6590 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:07.668823  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2202) took 3292 ms. Trace:
I20260709 13:18:07.669171  6590 rpcz_store.cc:276] 0709 13:18:04.376763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.645146 (+3268383us) service_pool.cc:224] Handling call
0709 13:18:07.645442 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.668571 (+ 23129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.668767 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.669615  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 740) took 1025 ms. Trace:
I20260709 13:18:07.670094  6517 rpcz_store.cc:276] 0709 13:18:06.644124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.644274 (+   150us) service_pool.cc:224] Handling call
0709 13:18:06.676129 (+ 31855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:06.717195 (+ 41066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.717242 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:06.717302 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:06.717348 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.669325 (+951977us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400354b54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:07.669386 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.669547 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93825920}
I20260709 13:18:07.670368  6590 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.646981 10263 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:07.666585 10272 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:07.687299 10272 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:07.688613  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2446) took 1052 ms. Trace:
I20260709 13:18:07.688433 10272 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.689014  6589 rpcz_store.cc:276] 0709 13:18:06.636349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.775615 (+139266us) service_pool.cc:224] Handling call
0709 13:18:07.688413 (+912798us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54002fd0d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.688557 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029252736}
I20260709 13:18:07.689803 10272 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:07.690284  6589 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:07.690716  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.643748 10447 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.691903  6595 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 8. Ops: [6.4-6.4]
I20260709 13:18:07.692788  6596 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.694197  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 846) took 2819 ms. Trace:
I20260709 13:18:07.694585  6589 rpcz_store.cc:276] 0709 13:18:04.874633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.689820 (+2815187us) service_pool.cc:224] Handling call
0709 13:18:07.690118 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.693965 (+  3847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.694150 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.696028  6589 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:07.696573  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 849) took 2821 ms. Trace:
I20260709 13:18:07.696916  6589 rpcz_store.cc:276] 0709 13:18:04.875523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.695478 (+2819955us) service_pool.cc:224] Handling call
0709 13:18:07.695739 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.696372 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.696529 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.697355 10406 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:07.698194  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.698465  6589 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:07.699097  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 859) took 2507 ms. Trace:
W20260709 13:18:07.699213  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2218) took 1585 ms (client timeout 1732 ms). Trace:
I20260709 13:18:07.699414  6589 rpcz_store.cc:276] 0709 13:18:05.191486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.697909 (+2506423us) service_pool.cc:224] Handling call
0709 13:18:07.698175 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.698885 (+   710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.699053 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:07.699857  6597 rpcz_store.cc:269] 0709 13:18:06.114034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.461676 (+347642us) service_pool.cc:224] Handling call
0709 13:18:07.699010 (+1237334us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540019e6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.699143 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303339264}
I20260709 13:18:07.700753  6589 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:07.701260  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2224) took 1472 ms. Trace:
I20260709 13:18:07.701592  6597 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:07.704344 10263 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:07.711171  6416 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 331) took 15829 ms (client timeout 15000 ms). Trace:
I20260709 13:18:07.642877  9899 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:07.641634 10448 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.662774 10374 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:07.857136 10374 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:07.857533 10374 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:07.665619 10251 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.859009 10251 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.861865 10251 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.673424  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2203) took 2964 ms. Trace:
I20260709 13:18:07.701550  6589 rpcz_store.cc:276] 0709 13:18:06.228407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.700257 (+1471850us) service_pool.cc:224] Handling call
0709 13:18:07.700494 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.701069 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.701220 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.862478  6590 rpcz_store.cc:276] 0709 13:18:04.708711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.669932 (+2961221us) service_pool.cc:224] Handling call
0709 13:18:07.670216 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.671818 (+  1602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.673353 (+  1535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.863029  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.863572  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.864164  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.864749  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.865497  6590 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:07.858433 10374 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.868094 10374 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:07.899555  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.702514  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2225) took 1462 ms. Trace:
I20260709 13:18:07.613608 10400 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.614732 10257 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:07.721565  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.723150  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.728621  9873 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:07.735343  6670 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:07.736510 10262 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:07.743474  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 842) took 4651 ms. Trace:
I20260709 13:18:07.906150  6593 rpcz_store.cc:276] 0709 13:18:03.091968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.317357 (+4225389us) service_pool.cc:224] Handling call
0709 13:18:07.652782 (+335425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.653347 (+   565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.743148 (+ 89801us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002fbcd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.743385 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770746240}
I20260709 13:18:07.751088  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.751847  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.752490  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.753381  6520 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.752522 10161 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:07.900410  6597 rpcz_store.cc:276] 0709 13:18:06.240300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.700920 (+1460620us) service_pool.cc:224] Handling call
0709 13:18:07.701284 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:07.702202 (+   918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.702437 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.758147  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2201) took 3429 ms. Trace:
I20260709 13:18:07.758837  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2399) took 4862 ms. Trace:
I20260709 13:18:07.910990  6595 rpcz_store.cc:276] 0709 13:18:02.896349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.250381 (+4354032us) service_pool.cc:224] Handling call
0709 13:18:07.691680 (+441299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:07.692308 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.758660 (+ 66352us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54002fd0d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.758780 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013840128}
I20260709 13:18:07.910888  6588 rpcz_store.cc:276] 0709 13:18:04.328962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.386746 (+3057784us) service_pool.cc:224] Handling call
0709 13:18:07.387043 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.430488 (+ 43445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.430537 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:07.430595 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:07.430671 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.757812 (+327141us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b54002f62d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:07.757877 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.758047 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99310848}
I20260709 13:18:07.912540  6595 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:07.758190  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1042) took 1140 ms. Trace:
I20260709 13:18:07.770013  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2191) took 4157 ms. Trace:
I20260709 13:18:07.911266  6597 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:18:07.913461  6597 pending_rounds.cc:85] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 6
I20260709 13:18:07.913064  6596 rpcz_store.cc:276] 0709 13:18:03.614866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.342823 (+3727957us) service_pool.cc:224] Handling call
0709 13:18:07.692614 (+349791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.693139 (+   525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.769777 (+ 76638us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54002fd0d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.769941 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803606400}
I20260709 13:18:07.913913  6597 pending_rounds.cc:107] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:18:07.914278  6597 raft_consensus.cc:2889] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:07.785192  6415 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 7, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 7 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:07.794991 10276 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:07.799425 10441 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.562s	user 0.013s	sys 0.000s
W20260709 13:18:07.802306  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.804271  6521 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:07.951275  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.951850  6514 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:18:07.952005  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.804764 10398 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.724982 10244 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:07.805142  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:07.954224  6513 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:07.954308  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:07.808200 10428 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.816849 10280 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:18:07.817639  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 97554.8us
I20260709 13:18:07.818905 10248 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:07.844419  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.963045  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.846791 10436 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.864816  6589 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:18:07.714016 10263 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:07.901543  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.866312  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:07.965315  6590 raft_consensus.cc:2410] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:07.912864  6588 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:07.914820  6596 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:07.916208  6593 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:07.774058 10281 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.967099  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2226) took 1703 ms. Trace:
I20260709 13:18:07.967528  6593 rpcz_store.cc:276] 0709 13:18:06.264210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.915667 (+1651457us) service_pool.cc:224] Handling call
0709 13:18:07.915913 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.966798 (+ 50885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.967035 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.968184 10281 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:07.969467 10281 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:07.918970  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:07.902292  9873 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:07.916709  7236 rpcz_store.cc:276] 0709 13:18:06.618145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.618316 (+   171us) service_pool.cc:224] Handling call
0709 13:18:07.757953 (+1139637us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b5400646c54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:07.758130 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942807936}
W20260709 13:18:07.955549 10396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.046s	sys 0.070s
I20260709 13:18:07.788229 10332 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:07.929205  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 876) took 1018 ms. Trace:
I20260709 13:18:07.974514  6595 rpcz_store.cc:276] 0709 13:18:06.906254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.912132 (+1005878us) service_pool.cc:224] Handling call
0709 13:18:07.912385 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.924259 (+ 11874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.924445 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:07.924800 10049 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 13:18:07.977324  7232 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 11. Ops: []
I20260709 13:18:07.949281  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2445) took 1313 ms. Trace:
I20260709 13:18:07.957015  6415 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:07.978102  6597 rpcz_store.cc:276] 0709 13:18:06.636052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.910637 (+1274585us) service_pool.cc:224] Handling call
0709 13:18:07.910939 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.948909 (+ 37970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.949202 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.973022  6593 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:07.973392  6590 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:07.660800  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:07.984125  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 323385us
I20260709 13:18:07.985080  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:07.934527  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:07.986292  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51866.5us
I20260709 13:18:07.956069 10244 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:07.992228  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.974274 10396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.046s	sys 0.071s
I20260709 13:18:07.990430  6415 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 7, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 7 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:07.993726  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:07.969851 10281 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.996986  6667 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:07.955377 10276 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:08.006810  6415 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:07.857851  6416 rpcz_store.cc:269] 0709 13:17:51.882332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.891355 (+  9023us) service_pool.cc:224] Handling call
0709 13:18:07.711051 (+15819696us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:02.379317 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:02.503772 (+124455us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:02.503825 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:18:02.503842 (+    17us) tablet.cc:662] Decoding operations
0709 13:18:02.554055 (+ 50213us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:02.560954 (+  6899us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:02.562647 (+  1693us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:18:02.583392 (+ 20745us) tablet.cc:701] Row locks acquired
0709 13:18:02.583450 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:18:02.583618 (+   168us) write_op.cc:270] Start()
0709 13:18:02.583762 (+   144us) write_op.cc:276] Timestamp: P: 1783603082583556 usec, L: 0
0709 13:18:02.583784 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:18:02.584593 (+   809us) log.cc:844] Serialized 6687 byte log entry
0709 13:18:03.260579 (+675986us) op_driver.cc:464] REPLICATION: finished
0709 13:18:03.262217 (+  1638us) write_op.cc:301] APPLY: starting
0709 13:18:03.262297 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:03.262704 (+   407us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:03.262824 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:03.265220 (+  2396us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:03.265508 (+   288us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:18:03.265559 (+    51us) write_op.cc:312] APPLY: finished
0709 13:18:03.265938 (+   379us) log.cc:844] Serialized 119 byte log entry
0709 13:18:03.266118 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:03.266265 (+   147us) write_op.cc:454] Released schema lock
0709 13:18:03.266314 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:03.266715 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:03.266758 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4163,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":1362,"apply.run_cpu_time_us":3913,"apply.run_wall_time_us":4461,"num_ops":17,"prepare.queue_time_us":366709,"prepare.run_cpu_time_us":19332,"prepare.run_wall_time_us":379107,"raft.queue_time_us":34976,"raft.run_cpu_time_us":1516,"raft.run_wall_time_us":1516,"replication_time_us":676605,"spinlock_wait_cycles":1792,"thread_start_us":412818,"threads_started":4,"wal-append.queue_time_us":174782,"wal-append.run_cpu_time_us":19550,"wal-append.run_wall_time_us":1501412}]]}
I20260709 13:18:07.978873  9899 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.018668  6517 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:07.958755 10398 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.999344 10272 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.016717  6415 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.992830 10400 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:08.023574  6665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.000s	sys 0.019s
W20260709 13:18:08.023968  6665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.000s	sys 0.019s
I20260709 13:18:07.994536 10247 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:07.972839 10248 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.989449 10406 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:18:08.124136  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 169963us
I20260709 13:18:08.004518 10281 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:08.080600  6637 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.011s	sys 0.006s
W20260709 13:18:08.137144  6637 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.012s	sys 0.006s
W20260709 13:18:08.137462  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 797056us
W20260709 13:18:08.073769  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.073123  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.088771  6415 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 7 to 8, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 8 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:08.090608  6518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.006s	sys 0.012s
W20260709 13:18:08.139668  6518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.006s	sys 0.012s
W20260709 13:18:08.100333  6522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.009s	sys 0.009s
W20260709 13:18:08.141114  6522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.010s	sys 0.009s
W20260709 13:18:08.105316 10258 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.005s	sys 0.015s
I20260709 13:18:08.112319 10480 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.998700 10161 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 13:18:08.149648  7236 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:18:08.151142  7232 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:08.115164  6486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.005s	sys 0.017s
I20260709 13:18:08.151937  7235 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:08.152472  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:08.151937  6486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.006s	sys 0.017s
W20260709 13:18:07.997407 10414 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.003s	sys 0.009s
I20260709 13:18:08.120338 10465 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:08.121783  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 477605us
I20260709 13:18:07.441839 10359 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:07.989717 10049 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:08.011130 10251 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:08.087175  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.180580  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.995090 10280 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
W20260709 13:18:08.159997 10414 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.003s	sys 0.013s
I20260709 13:18:08.023279 10443 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:08.153062  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 775626us
I20260709 13:18:08.186411  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:08.218010  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.156011  7236 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:08.214079  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:08.214470  7234 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:08.219650  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.220315  7230 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:18:08.167130  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:08.014767  9873 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.226351  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:08.226773  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.227547  7234 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:08.220496  7236 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:08.170423  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:08.171751  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:08.230523  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:08.173079  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:08.231630  6590 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.173348  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:08.232952  6593 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
W20260709 13:18:08.123837 10429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.007s	sys 0.010s
I20260709 13:18:08.176178  7235 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:08.235162  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.235901  7235 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:08.236754  7229 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:08.176563  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.237680  7228 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:08.180724  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.238762  7237 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:18:08.146354 10258 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.005s	sys 0.015s
W20260709 13:18:08.239538  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.183073  6415 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 6 to 9, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:08.184098  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.158969 10400 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 13:18:08.144655  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 122300us
I20260709 13:18:08.148474 10263 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:08.025276  9899 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.159754  7233 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:08.243871  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:08.122992 10247 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.167847  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.222998  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.185642  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:08.036653 10272 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:08.025287  9925 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:18:08.305094  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.307221  7230 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:08.174225  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:08.244794  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 91703.4us
I20260709 13:18:08.247836  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:08.221776  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:08.309855  7232 raft_consensus.cc:2376] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6. Re-sending same reply.
I20260709 13:18:08.185508 10270 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.298274  7231 raft_consensus.cc:2410] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:08.299853  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.300923  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:08.302101  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 155611us
I20260709 13:18:08.306366  7234 raft_consensus.cc:2376] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11. Re-sending same reply.
W20260709 13:18:08.244211 10429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.007s	sys 0.011s
I20260709 13:18:08.251979  6670 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:08.315189  6590 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:08.185710 10477 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:18:08.126011 10244 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:08.185704 10251 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:08.347569  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:08.330605  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.352198  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 178130us
I20260709 13:18:08.391360  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:08.356307  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:08.393875  7230 raft_consensus.cc:2410] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:18:08.254138 10456 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:18:08.395915  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:08.401805  7233 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:08.396612  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47261.8us
W20260709 13:18:08.396917  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 153661us
I20260709 13:18:08.403436  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.168990 10480 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.310298 10478 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.401120  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:08.401226  6589 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:08.402670  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:08.407164  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:08.399772  6415 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 5, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.407727  7230 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:08.402215  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:08.404613  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 219071us
I20260709 13:18:08.408504  7231 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:08.253861 10436 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:08.405292  6636 leader_election.cc:400] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:08.411804  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:08.513286  6636 leader_election.cc:403] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:18:08.430846  6670 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:08.432741  6415 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 2 to 9, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 9 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.433650  7230 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:08.515419  7230 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:08.224853 10448 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
W20260709 13:18:08.519549  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 114442us
I20260709 13:18:08.252069 10428 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:08.303056 10280 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.446328  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.443861  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 196071us
W20260709 13:18:08.486835  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.417615  7237 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:08.438508  7233 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:08.185442 10447 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.193s
W20260709 13:18:08.513733  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 102033us
I20260709 13:18:08.397358 10406 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:08.441967 10261 consensus_meta.cc:287] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.803s	user 0.006s	sys 0.006s
I20260709 13:18:08.505241 10244 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:08.396360 10270 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:08.400317 10397 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.000s
W20260709 13:18:08.482601 10463 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
W20260709 13:18:08.433837 10458 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 13:18:08.520648  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:08.561582  7236 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:18:08.561951  7236 pending_rounds.cc:85] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 6
W20260709 13:18:08.562101  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.562163  7236 pending_rounds.cc:107] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:18:08.562538  7236 raft_consensus.cc:2889] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:08.524792  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:08.404189 10481 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:08.418088 10260 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:08.523352  7230 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.504016 10429 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:08.522214 10400 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:08.522924  6415 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 5, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:08.506610 10332 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:08.483477 10396 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:18:08.565196  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:08.566043  6668 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:18:08.565953  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:08.635182  6669 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:08.566979  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:08.554847 10406 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.432610 10474 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 13:18:08.516386  9899 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:08.433179 10465 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:08.519260 10414 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:08.577315  7230 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:08.434713 10469 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.124s
I20260709 13:18:08.583406  6671 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:18:08.433292 10478 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.563057  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:08.607611  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.482949 10257 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:18:08.688791  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:08.689416 10270 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:08.690357  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:08.742430  6669 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:08.747539  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 265452us
I20260709 13:18:08.581529 10441 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 13:18:08.579807 10397 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.155s
I20260709 13:18:08.629752  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:08.718139  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 152983us
I20260709 13:18:08.719363  6415 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 4 to 5, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:08.573923 10488 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:08.554226 10476 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:18:08.434842 10262 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.126s
W20260709 13:18:08.798386  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45852.4us
I20260709 13:18:08.553692 10484 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.194s
I20260709 13:18:08.559355 10280 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.737908 10261 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.730921 10332 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:18:08.731146 10401 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.735337 10400 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:08.740190 10272 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:08.685683 10276 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:08.727192 10461 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:18:08.698863  9925 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:08.686838 10406 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:08.635655 10260 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:08.509949 10268 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:18:08.753423  6662 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:08.827476 10429 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:08.775955  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.777352  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.735100 10475 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.575s	user 0.000s	sys 0.001s
I20260709 13:18:08.728845 10489 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:08.740880 10398 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:08.823760  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 468134us
I20260709 13:18:08.743485 10049 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:08.828163  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 273378us
W20260709 13:18:08.829028  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 422414us
I20260709 13:18:08.830837  6415 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 7, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 7 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.844435 10276 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:08.696825 10251 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:08.807350 10441 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:08.806198 10488 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:08.768077  6669 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:08.809247 10247 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:08.894726  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:08.834906 10401 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:08.901055 10263 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:18:08.902314  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2449) took 1927 ms (client timeout 2191 ms). Trace:
W20260709 13:18:08.980041  6594 rpcz_store.cc:269] 0709 13:18:06.975284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.154019 (+178735us) service_pool.cc:224] Handling call
0709 13:18:08.902112 (+1748093us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002f5dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:08.902256 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2770566400}
I20260709 13:18:08.829986 10359 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:08.903004  6592 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:09.036286  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 116883us
W20260709 13:18:09.044992  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 150349us
I20260709 13:18:09.046059  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:08.907536  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 496764us
I20260709 13:18:08.851838 10272 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.882541  9925 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.881093 10258 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:08.916679  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:08.741508 10465 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:08.906975 10477 consensus_queue.cc:1243] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:08.980223  6415 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 4 to 6, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 6 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.829720 10261 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:08.999643  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.081897  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:09.082655  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:09.069142  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1964) took 1723 ms. Trace:
I20260709 13:18:09.083307  6662 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:09.068462  9925 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:08.944936 10441 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:09.031675 10263 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:09.080281  6415 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 4, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 4 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:09.083524  6665 rpcz_store.cc:276] 0709 13:18:07.318887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.319001 (+   114us) service_pool.cc:224] Handling call
0709 13:18:07.335854 (+ 16853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:07.337100 (+  1246us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:07.383979 (+ 46879us) log.cc:844] Serialized 14 byte log entry
0709 13:18:07.797484 (+413505us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b4800220158. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f 00007f04be098f37 00007f04be0ad55c 00007f04be0ad508 00007f04be0ad488 00007f04be0ad443 00007f04be0ac6ec 00007f04bcfd6a96
0709 13:18:08.512988 (+715504us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:08.533443 (+ 20455us) log.cc:844] Serialized 26 byte log entry
0709 13:18:08.533589 (+   146us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:08.533624 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.533686 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.033581 (+499895us) raft_consensus.cc:1685] finished
0709 13:18:09.042351 (+  8770us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.042554 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":877,"prepare.queue_time_us":394539,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":162777,"raft.run_cpu_time_us":7540,"raft.run_wall_time_us":488583,"spinlock_wait_cycles":21740800,"thread_start_us":1027102,"threads_started":2,"wal-append.queue_time_us":908596}
I20260709 13:18:09.065125 10484 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.103586  6594 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:09.096239  6415 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.065147 10414 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:09.098788 10483 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:09.098490 10490 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:09.098953 10492 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.200s
I20260709 13:18:09.069893 10474 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:18:09.098829 10482 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.098989 10495 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.200s
I20260709 13:18:09.100955 10489 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:09.099062 10487 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:18:09.099252 10486 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.157s
I20260709 13:18:09.068701 10398 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:09.070178 10332 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.090914 10477 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:09.068658 10258 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:08.945103 10276 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.043728 10049 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:18:09.099346 10496 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:18:09.072715 10272 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.086159 10397 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:18:09.143730  6636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.003s	sys 0.004s
I20260709 13:18:09.119889 10480 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:09.178907  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 141681us
I20260709 13:18:09.179490  6517 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:09.130918 10483 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:09.169332  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.176259  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:09.184880  6518 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:09.177793  6636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.004s	sys 0.004s
I20260709 13:18:09.164463  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:09.186192  6522 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:09.163079  6415 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:09.161573 10477 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:09.168905 10332 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.168147 10262 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.109174 10429 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.195000  7228 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:09.155951 10507 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:09.115958 10484 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.142743 10500 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:18:09.174518 10478 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:09.200258  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 283620us
I20260709 13:18:09.220726  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:09.174064 10398 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:09.202131  6415 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.198809 10477 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:09.091998 10260 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.199626 10257 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:09.210098  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:09.210883  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:09.233968  6663 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:09.234182  6671 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 5.
I20260709 13:18:09.234901 10503 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:18:09.235318  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:09.235993  6671 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:09.236774  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:09.208354 10262 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.213338  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:09.189057 10396 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.213990  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1170) took 1056 ms. Trace:
I20260709 13:18:09.210484  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 841) took 6118 ms. Trace:
I20260709 13:18:09.148470 10482 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.194756 10480 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:09.234826 10474 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.223222 10398 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.153935 10489 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.207719  6662 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:09.157275 10441 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.208840  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:09.230513  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.219260  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:09.241765  6592 rpcz_store.cc:276] 0709 13:18:03.091692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.316270 (+4224578us) service_pool.cc:224] Handling call
0709 13:18:08.902849 (+1586579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.033945 (+131096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.210310 (+176365us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54002f5dd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:09.210433 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3625999104}
I20260709 13:18:09.244346  6663 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:09.245589  6671 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:09.241654  7236 rpcz_store.cc:276] 0709 13:18:08.157880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.228314 (+ 70434us) service_pool.cc:224] Handling call
0709 13:18:08.246288 (+ 17974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.577899 (+331611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.213777 (+635878us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54006cc354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:09.213935 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724329216}
I20260709 13:18:09.265998  6663 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:09.258601  6415 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 5, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 5 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.264532  6669 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:09.269235  6662 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:18:09.247866 10374 consensus_meta.cc:287] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.379s	user 0.010s	sys 0.002s
I20260709 13:18:09.276723  6415 catalog_manager.cc:5697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 5, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 5 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
W20260709 13:18:09.238579 10244 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.678s	user 0.006s	sys 0.004s
I20260709 13:18:09.271809  6669 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:18:09.245956 10443 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.999s	user 0.008s	sys 0.005s
I20260709 13:18:09.280224 10443 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:09.229879 10266 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.275738 10510 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:09.257426 10169 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:09.250241 10272 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:09.235340 10281 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.074s	user 0.012s	sys 0.000s
I20260709 13:18:09.278723  6671 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:09.281497  6415 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 6, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 6 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:09.275101  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55899.7us
I20260709 13:18:09.262156 10401 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:09.286057  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:09.236701 10276 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:09.287427  6668 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:09.243547 10480 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.292579  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:09.317298  6513 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:09.293190  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:09.251389 10506 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.250996 10398 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:09.243352 10482 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.251963 10483 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.240716 10262 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.197454 10414 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.206321 10258 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.309629  6415 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 5 to 6, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 6 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:09.284511  6662 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:09.315876  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 94126.7us
I20260709 13:18:09.333227  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:09.289320 10441 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:09.326097  6407 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 921c423406da4f289884afca51ca198f (127.5.72.129:34121) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 921c423406da4f289884afca51ca198f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:09.241529 10396 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.257655 10257 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:09.334508  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 252698us
I20260709 13:18:09.296995 10261 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:09.244905 10474 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.342598  6415 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:09.214962 10507 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.290422 10486 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:18:09.345563  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:09.306231 10169 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.322093 10270 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:09.349697  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:09.321908 10480 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:09.368873  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2450) took 2393 ms (client timeout 1704 ms). Trace:
I20260709 13:18:09.336673 10262 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.297935 10266 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:09.369481  6591 rpcz_store.cc:269] 0709 13:18:06.975591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.030462 (+ 54871us) service_pool.cc:224] Handling call
0709 13:18:09.368662 (+2338200us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003a43d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:09.368805 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2789732608}
I20260709 13:18:09.326704 10506 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.371743 10517 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:09.372300 10508 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:09.372799 10406 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.338650 10488 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:09.310683  9925 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.315105 10281 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.367947 10270 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:09.392414 10514 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.392864 10519 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:09.393142 10514 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.347314 10258 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:09.392495 10406 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.387527  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:09.396848 10493 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:18:09.335088 10272 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:18:09.379297 10276 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:09.374248 10513 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.395083 10496 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:09.396226 10270 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:09.380383 10262 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.422827  6409 maintenance_manager.cc:419] P 10caaf50f65c441b844e3e6055117d18: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033371
W20260709 13:18:09.689925  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.134662  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.714329  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.894657  6520 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.009s
W20260709 13:18:10.137280  6520 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.005s	sys 0.009s
W20260709 13:18:09.700749  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.140817  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.141827  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.772106  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.143519  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:09.895509  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 71038.4us
W20260709 13:18:10.078805  6562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.007s	sys 0.029s
I20260709 13:18:10.143782  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:10.181567  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:10.163501  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:09.505851  6637 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.009s	sys 0.039s
W20260709 13:18:10.631719  6637 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.832s	user 0.011s	sys 0.039s
W20260709 13:18:10.632164  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 1.83263e+06us
I20260709 13:18:10.170912  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:10.163220  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:10.635790  6669 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
W20260709 13:18:10.229843  6487 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.173s (SENT) [suppressed 46 similar messages]
I20260709 13:18:10.373872  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2333) took 1161 ms. Trace:
I20260709 13:18:09.446336  7230 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:18:09.731534  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.644290  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.644673 10549 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:09.733736  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 388224us
I20260709 13:18:10.647496 10548 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.515544  9899 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:10.696918  9899 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:10.641456  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:09.413424 10169 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.617117 10248 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.446103 10257 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:09.700032 10258 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:09.456384 10262 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:09.458305 10263 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:10.652495 10551 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:10.691825 10554 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.546890  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 403742us
I20260709 13:18:11.046519 10262 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:11.046962 10258 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:10.674556  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.053557  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:10.666939  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:10.652529  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:11.057848  6408 maintenance_manager.cc:643] P 10caaf50f65c441b844e3e6055117d18: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.921s	user 0.142s	sys 0.066s Metrics: {"bytes_written":29123,"cfile_init":1,"dirs.queue_time_us":84359,"dirs.run_cpu_time_us":373,"dirs.run_wall_time_us":17519,"drs_written":1,"lbm_read_time_us":403,"lbm_reads_lt_1ms":4,"lbm_write_time_us":10773,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":29,"peak_mem_usage":0,"rows_written":63,"thread_start_us":93719,"threads_started":2,"wal-append.queue_time_us":7567}
I20260709 13:18:10.664621  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:10.645392  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:11.061538  6409 maintenance_manager.cc:419] P 10caaf50f65c441b844e3e6055117d18: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10351 bytes on disk
I20260709 13:18:11.064399  6419 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 6 to 7, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 7 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:11.065989 10257 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:10.660486  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:10.709599 10552 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:11.068305 10552 consensus_queue.cc:1243] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:11.067605  6419 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:10.724213 10559 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.069372 10559 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.069788 10257 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:11.072075 10559 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:10.734886  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:10.779865  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:10.621037  6562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.213s	user 0.011s	sys 0.029s
I20260709 13:18:11.077036  6408 maintenance_manager.cc:643] P 10caaf50f65c441b844e3e6055117d18: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":385,"lbm_reads_lt_1ms":4}
I20260709 13:18:11.077455  6520 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
W20260709 13:18:10.721514  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64595.4us
I20260709 13:18:10.626407  6662 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:11.081336 10553 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.082024 10553 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.084709 10553 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.085366 10553 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:18:10.721693  6637 connection.cc:256] RPC call timeout handler was delayed by -2.1742385430000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 13:18:11.091621  6637 connection.cc:256] RPC call timeout handler was delayed by -2.0177850149998449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:18:11.092068  6637 connection.cc:256] RPC call timeout handler was delayed by -1.7359385429999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 13:18:11.092453  6637 connection.cc:256] RPC call timeout handler was delayed by -1.4122136319999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2332s for a response
W20260709 13:18:11.092940  6637 connection.cc:256] RPC call timeout handler was delayed by -1.1323885670001346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:18:11.093328  6637 connection.cc:256] RPC call timeout handler was delayed by -1.1143885670001055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
W20260709 13:18:10.662881  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 519468us
W20260709 13:18:10.787520  6618 debug-util.cc:398] Leaking SignalData structure 0x7b0800343440 after lost signal to thread 5413
W20260709 13:18:11.098311  6618 debug-util.cc:398] Leaking SignalData structure 0x7b0800346300 after lost signal to thread 6454
I20260709 13:18:10.637645  6667 rpcz_store.cc:276] 0709 13:18:09.212280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.242540 (+ 30260us) service_pool.cc:224] Handling call
0709 13:18:09.242852 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.395222 (+152370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.395252 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.396792 (+  1540us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.467765 (+ 70973us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:09.467829 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.373070 (+905241us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540023ffd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:10.373146 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:10.373233 (+    87us) raft_consensus.cc:1685] finished
0709 13:18:10.373248 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.373690 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3575,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":5119,"spinlock_wait_cycles":349580160,"thread_start_us":70620,"threads_started":1,"wal-append.queue_time_us":90385,"wal-append.run_cpu_time_us":6896,"wal-append.run_wall_time_us":1052514}
W20260709 13:18:10.761314  6636 leader_election.cc:336] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.191s (SENT)
W20260709 13:18:11.103502  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 342347us
I20260709 13:18:10.731108  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:10.719018  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 56298us
I20260709 13:18:10.641201  7230 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:11.110023  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:10.719221  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:10.708432  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.112221  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.112486  7233 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:11.112798  6520 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:10.650765  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:10.749599  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:11.116407  7237 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:11.117714  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:10.828217  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 598392us
I20260709 13:18:11.118717  6595 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:11.119189  6514 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:11.119848  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.121125  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:10.698544  9899 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.125609  9899 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:11.126331  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.126987  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:09.440253  9925 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:10.546177 10184 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:11.137354 10184 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.137777 10184 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:11.137812  6517 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:11.138795 10551 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.139374 10551 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.141521 10551 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:10.732429  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.143900  7231 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:10.730765  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:11.144862  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.431808 10266 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.438738 10281 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:10.640395 10049 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:11.149387 10049 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.413810 10332 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:10.545624 10359 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.449476 10396 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:09.444231 10400 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.000s
I20260709 13:18:11.156193 10400 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:09.679657  9873 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:11.160604  6415 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 329) took 18491 ms (client timeout 15000 ms). Trace:
I20260709 13:18:11.161734  7237 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
W20260709 13:18:11.161023  6415 rpcz_store.cc:269] 0709 13:17:52.668615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.668861 (+   246us) service_pool.cc:224] Handling call
0709 13:18:11.160338 (+18491477us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:09.516990 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:09.517710 (+   720us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:09.517760 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:18:09.517789 (+    29us) tablet.cc:662] Decoding operations
0709 13:18:09.519832 (+  2043us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:09.519951 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:09.520117 (+   166us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:18:09.523197 (+  3080us) tablet.cc:701] Row locks acquired
0709 13:18:09.523230 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:18:09.523426 (+   196us) write_op.cc:270] Start()
0709 13:18:09.523547 (+   121us) write_op.cc:276] Timestamp: P: 1783603089523369 usec, L: 0
0709 13:18:09.523571 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:18:09.524325 (+   754us) log.cc:844] Serialized 8555 byte log entry
0709 13:18:09.573285 (+ 48960us) op_driver.cc:464] REPLICATION: finished
0709 13:18:09.575986 (+  2701us) write_op.cc:301] APPLY: starting
0709 13:18:09.576068 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:09.576476 (+   408us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:09.576596 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:09.578840 (+  2244us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:09.580576 (+  1736us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:18:09.580656 (+    80us) write_op.cc:312] APPLY: finished
0709 13:18:09.581060 (+   404us) log.cc:844] Serialized 152 byte log entry
0709 13:18:09.581781 (+   721us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:09.581955 (+   174us) write_op.cc:454] Released schema lock
0709 13:18:09.582005 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:09.585440 (+  3435us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:09.585473 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":47818,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":2457,"apply.run_cpu_time_us":4445,"apply.run_wall_time_us":9215,"num_ops":22,"prepare.queue_time_us":7569,"prepare.run_cpu_time_us":11228,"prepare.run_wall_time_us":50735,"raft.queue_time_us":871,"raft.run_cpu_time_us":2202,"raft.run_wall_time_us":2726,"replication_time_us":49552,"thread_start_us":48206,"threads_started":4,"wal-append.queue_time_us":43238,"wal-append.run_cpu_time_us":4386,"wal-append.run_wall_time_us":1016116}]]}
I20260709 13:18:09.634086 10428 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:18:11.167198  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 82404.6us
I20260709 13:18:11.168507 10428 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:11.169791 10428 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.463585 10480 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.186220 10480 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.623199 10280 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.443008 10398 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.191172 10398 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:09.482050 10436 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:09.664927 10439 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:09.633196 10440 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:11.195930 10440 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:11.195200 10184 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.197268 10440 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:10.401007 10448 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.199545 10398 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:11.199754 10448 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.202356  9899 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.752496 10457 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.435674 10484 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:10.224864 10486 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.206593 10457 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.208060 10486 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.208796 10557 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.209630 10557 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.694741 10488 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.211741 10488 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.212153  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.212611 10557 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.212987  7236 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:11.206954 10439 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:11.214035 10488 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.425767 10489 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.731978 10492 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.214592  7233 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:11.216487  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.216629 10492 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.217602  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:11.218354  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.003872 10495 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:09.410058 10500 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:11.220758 10500 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:10.403625 10501 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.222088 10500 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.222476 10501 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.701355 10465 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:09.427466 10474 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.259958 10474 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:10.144747 10481 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:09.433077 10482 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.265120 10482 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.563902 10491 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:11.267866 10491 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:11.268348 10448 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.421083 10496 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.270947  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:11.271652 10496 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.272537  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.273837  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.274206 10496 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:09.422729 10516 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:18:09.435627 10506 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:09.387643 10508 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:09.454285 10511 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:18:09.454948 10512 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:18:09.414960 10513 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.289186  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1050) took 2935 ms. Trace:
I20260709 13:18:11.291217  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 744) took 2725 ms. Trace:
I20260709 13:18:11.289722  7232 rpcz_store.cc:276] 0709 13:18:08.353274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.353723 (+   449us) service_pool.cc:224] Handling call
0709 13:18:08.354061 (+   338us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:08.558519 (+204458us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:08.806960 (+248441us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.270879 (+463919us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.587753 (+316874us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b180035dbe0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:09.729945 (+142192us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b180035dbe0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:09.730147 (+   202us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:09.730957 (+   810us) log.cc:844] Serialized 78 byte log entry
0709 13:18:09.731107 (+   150us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:09.731340 (+   233us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.731504 (+   164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.731593 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.288602 (+1557009us) raft_consensus.cc:1685] finished
0709 13:18:11.288642 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.288931 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":453142,"prepare.queue_time_us":376992,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":108,"raft.queue_time_us":616737,"raft.run_cpu_time_us":503,"raft.run_wall_time_us":42481,"spinlock_wait_cycles":20475136,"thread_start_us":797515,"threads_started":4,"wal-append.queue_time_us":490817}
I20260709 13:18:11.291632 10513 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:09.424989 10514 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.292627  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:09.386165 10517 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:09.734339 10518 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:10.553839 10546 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:10.503085 10545 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:11.299088 10546 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.299731 10545 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.300590 10517 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.301265 10546 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.302095 10545 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:10.659130 10547 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.303891 10547 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.306067 10457 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.306236 10547 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:10.663532 10548 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:10.645180 10549 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.409258 10522 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.580006 10532 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:11.309157 10548 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.309227 10522 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:10.866607 10554 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.311538  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:11.312350  7232 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:10.869587 10263 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:10.979553  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:11.005268  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:11.012766  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 240940us
I20260709 13:18:10.675479  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.315563 10547 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:11.046448 10169 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:18:11.315915  7228 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:11.316293 10547 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.316814 10169 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:11.316944 10522 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.317209 10169 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:11.048563 10262 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:11.074647  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:11.080339 10559 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.088094 10571 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:11.097957  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.320948 10571 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:11.321208  6594 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:11.321331 10169 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.322813  6520 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:11.322885 10169 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:11.323272  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 248787us
I20260709 13:18:11.323944 10262 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:11.324366 10262 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.106069  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:11.324952 10262 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:11.322165 10571 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:11.325690  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 219820us
I20260709 13:18:11.326516 10571 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:11.100092  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.107463  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:10.779848  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.312568 10554 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.322932 10559 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.330842  7234 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:11.113081  6590 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:11.332283 10547 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.332499 10559 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.333051  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.333740  6590 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:11.110687  7230 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:11.115098  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080034cde0 after lost signal to thread 6546
I20260709 13:18:11.114012  6521 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:11.122741 10569 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:11.114936  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.335371 10569 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.131059  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.135272  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.135653  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:11.337671 10569 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.337772  6522 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:11.136752  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.069784  6419 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:11.146098  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.150264  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.166662  6665 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:09.424821 10459 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.529s	user 0.004s	sys 0.031s
I20260709 13:18:11.143910  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.171622 10576 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:11.152451 10359 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.353547  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.354043  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2320) took 2149 ms. Trace:
I20260709 13:18:11.354913 10465 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:11.175161 10049 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.355818 10489 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:11.176703 10553 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.512341 10270 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.187232 10428 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:11.193471 10519 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.360697 10519 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.357875 10553 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.364370 10519 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.193512 10436 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:11.198732  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 93597.3us
I20260709 13:18:11.365489 10553 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.366302 10436 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.367403  6671 raft_consensus.cc:2410] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:11.200207 10398 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:09.486249 10477 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.375157  6669 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:11.203025  9899 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.376116  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:11.376731  6662 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:11.377280  9899 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.204130 10281 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.377982 10281 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.379520  7236 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:11.380024 10281 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.205819 10184 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.206362 10551 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.207140 10552 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:11.383190 10477 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:10.168659 10490 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.383229 10551 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.385143 10490 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.355836 10359 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.387172 10490 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.211059  7231 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:11.214771  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.387926  7236 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:11.214558 10439 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.219344 10495 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:11.393216 10551 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.393723 10495 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.395406 10495 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:11.395448  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:11.396428 10490 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:11.397449  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:11.219079 10492 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.225450 10501 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.398583 10490 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:11.217263  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.399158 10490 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.399335  7233 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
W20260709 13:18:11.227859  6562 leader_election.cc:336] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.502s (SENT)
I20260709 13:18:11.399675 10490 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:11.400605  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 172791us
I20260709 13:18:11.401234 10270 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.401903  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2094) took 4124 ms. Trace:
W20260709 13:18:11.228749  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 132907us
I20260709 13:18:11.402431  6516 rpcz_store.cc:276] 0709 13:18:07.276930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.299304 (+ 22374us) service_pool.cc:224] Handling call
0709 13:18:07.299703 (+   399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:07.323708 (+ 24005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.323740 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:07.337724 (+ 13984us) log.cc:844] Serialized 52 byte log entry
0709 13:18:07.885353 (+547629us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:07.885393 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.885460 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.410199 (+1524739us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:11.401040 (+1990841us) raft_consensus.cc:1685] finished
0709 13:18:11.401124 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.401631 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29023,"raft.queue_time_us":21508,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":12209,"spinlock_wait_cycles":137735936,"thread_start_us":493141,"threads_started":1,"wal-append.queue_time_us":518853}
I20260709 13:18:11.228762  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.245501  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:11.248409 10557 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:18:09.436690 10507 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.269313 10491 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.274863  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.411603  6513 raft_consensus.cc:2410] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:11.276857 10248 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:18:11.278173 10440 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:11.278811 10482 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.281935 10508 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.286420  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2328) took 2075 ms. Trace:
I20260709 13:18:11.291601  6666 rpcz_store.cc:276] 0709 13:18:08.565688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.566745 (+  1057us) service_pool.cc:224] Handling call
0709 13:18:08.628930 (+ 62185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:08.779517 (+150587us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:08.828576 (+ 49059us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.144138 (+315562us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.144814 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.144978 (+   164us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:09.145024 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.717539 (+572515us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b18004996c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419
0709 13:18:09.777086 (+ 59547us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54000e56d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:09.777150 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.290806 (+1513656us) raft_consensus.cc:1685] finished
0709 13:18:11.290848 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.291022 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14550,"prepare.queue_time_us":46785,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":262539,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":45243,"spinlock_wait_cycles":363211904,"thread_start_us":229774,"threads_started":1,"wal-append.queue_time_us":351437}
I20260709 13:18:11.295270 10514 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:11.308080 10549 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:11.310422  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2329) took 2099 ms. Trace:
W20260709 13:18:11.313742  7198 connection.cc:256] RPC call timeout handler was delayed by -1.3089781099999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
I20260709 13:18:11.310066 10532 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.324304  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:11.334887  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080034cd20 after lost signal to thread 6547
I20260709 13:18:11.127566  6595 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
W20260709 13:18:11.154850  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.347204  7232 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:11.348099  6596 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:11.348605  6419 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 2 to 7, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 7 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:11.350302  7230 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:11.343313  6592 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:11.344686  6519 raft_consensus.cc:2410] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:11.351037  6588 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:11.357899  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.371297  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.371824  7228 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:11.370780 10398 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.372999  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.374122  6663 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:11.352695  7234 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:11.383293  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.383551 10552 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.210250 10486 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.354385  6665 rpcz_store.cc:276] 0709 13:18:09.204470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.205396 (+   926us) service_pool.cc:224] Handling call
0709 13:18:09.223521 (+ 18125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.343559 (+2120038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.353781 (+ 10222us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54000cbf54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:11.353990 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4467921280}
W20260709 13:18:11.389623  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 69078.9us
I20260709 13:18:11.388441  7229 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:11.392683 10439 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:11.399734  6662 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:11.418569  6664 rpcz_store.cc:276] 0709 13:18:09.211069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.211289 (+   220us) service_pool.cc:224] Handling call
0709 13:18:09.211595 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.281486 (+2069891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.281540 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.281602 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:11.281636 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.286060 (+  4424us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540011c6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:11.286103 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.286273 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4759467392}
W20260709 13:18:11.450453  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:11.454749  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08000b92c0 after lost signal to thread 5413
W20260709 13:18:11.455911  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800129c60 after lost signal to thread 6454
W20260709 13:18:11.457113  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080028e9c0 after lost signal to thread 6546
W20260709 13:18:11.457350  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080029b820 after lost signal to thread 6547
W20260709 13:18:11.458352  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800170540 after lost signal to thread 6621
I20260709 13:18:11.333762 10262 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.459012 10557 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:11.463603 10552 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.413519  6522 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:11.465003  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:11.465166  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08001a1b40 after lost signal to thread 6637
W20260709 13:18:11.466418  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44915.8us
I20260709 13:18:11.466926  6594 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:11.458063 10490 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.418342 10491 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:11.471921  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080013a800 after lost signal to thread 6695
W20260709 13:18:11.472332  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080003b7a0 after lost signal to thread 7198
W20260709 13:18:11.472553  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003f90e0 after lost signal to thread 7200
I20260709 13:18:11.476145  6590 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:11.477137  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.477736  6590 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:11.478895  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.469856  6597 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:11.480456  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.480564 10532 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.481218  6597 raft_consensus.cc:2368] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9.
I20260709 13:18:11.482024  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.482892  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:11.473074  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003c7ce0 after lost signal to thread 7230
W20260709 13:18:11.483952  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080032a5a0 after lost signal to thread 7261
I20260709 13:18:11.483199  6591 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:11.484464  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080037d700 after lost signal to thread 7262
W20260709 13:18:11.470333  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 146086us
I20260709 13:18:11.464749  6670 rpcz_store.cc:276] 0709 13:18:09.211341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.211533 (+   192us) service_pool.cc:224] Handling call
0709 13:18:09.211764 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.302148 (+2090384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.302200 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.302255 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:11.302284 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.310058 (+  7774us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b540005b254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:11.310101 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.310262 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4806797184}
W20260709 13:18:11.484786  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002a9f80 after lost signal to thread 9873
W20260709 13:18:11.485321  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08000828a0 after lost signal to thread 9899
W20260709 13:18:11.485553  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080033e040 after lost signal to thread 9925
W20260709 13:18:11.470397  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080033b3a0 after lost signal to thread 6621
I20260709 13:18:11.482885  6597 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:11.419682 10514 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:11.489002  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800404c00 after lost signal to thread 10169
I20260709 13:18:11.489905  6593 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:11.487171  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.488678  6667 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
W20260709 13:18:11.516166  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080038fbc0 after lost signal to thread 10248
W20260709 13:18:11.517242  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080031f340 after lost signal to thread 10257
W20260709 13:18:11.499347  6618 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2e80 after lost signal to thread 6695
W20260709 13:18:11.517586  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003cad00 after lost signal to thread 10258
W20260709 13:18:11.518743  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08000ade40 after lost signal to thread 10262
I20260709 13:18:11.519281  6588 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:11.521524 10184 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.522121 10184 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.524227  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:11.524771  7235 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:11.525503  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:11.487728 10547 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.526889  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.527621  6597 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:11.532220  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.469147 10557 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:11.524704  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.527247 10547 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.537448  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:11.537612  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:11.519016  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003722a0 after lost signal to thread 10263
I20260709 13:18:11.533807  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.534943  7232 raft_consensus.cc:2368] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7.
W20260709 13:18:11.537900  6618 debug-util.cc:398] Leaking SignalData structure 0x7b08003bf120 after lost signal to thread 7261
I20260709 13:18:11.487527 10501 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:11.538340  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003af360 after lost signal to thread 10266
W20260709 13:18:11.538419  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080030cac0 after lost signal to thread 7262
I20260709 13:18:11.539547  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:11.483596 10551 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.541639  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.541627  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:11.542218  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.542318  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:11.545665  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080033a920 after lost signal to thread 10270
W20260709 13:18:11.546082  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4fe0 after lost signal to thread 10280
W20260709 13:18:11.546370  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080032b0c0 after lost signal to thread 10281
I20260709 13:18:11.547127  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.547425  6513 raft_consensus.cc:2410] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:18:11.547889  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.548283  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:11.548727  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.549083  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:11.549749  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.550172  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.524320 10184 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:11.550874  6669 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:11.551729  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:11.554720 10545 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:11.555295 10545 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:11.555384 10490 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.547158  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.556316 10545 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.557639  6590 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:11.560374  7237 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:11.565825  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.562707  6592 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:11.547329  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:11.567582  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6f60 after lost signal to thread 10332
I20260709 13:18:11.567744  6664 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:11.569130  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.572422  6591 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:11.526957 10262 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.574647  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.575402  6589 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:11.573289  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800378020 after lost signal to thread 10396
I20260709 13:18:11.573444  6595 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:11.533201 10552 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:11.576609  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab660 after lost signal to thread 10398
I20260709 13:18:11.578315  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:11.579216  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003be440 after lost signal to thread 10400
I20260709 13:18:11.560858  6419 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 4 to 6, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 6 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:11.579612  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 60432us
W20260709 13:18:11.579707  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08001a4700 after lost signal to thread 10428
W20260709 13:18:11.580700  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6e60 after lost signal to thread 10436
W20260709 13:18:11.580914  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002d45a0 after lost signal to thread 10439
W20260709 13:18:11.581264  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080027c360 after lost signal to thread 10440
W20260709 13:18:11.581540  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002b0900 after lost signal to thread 10457
W20260709 13:18:11.581744  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080038a600 after lost signal to thread 10459
W20260709 13:18:11.581943  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08000fe960 after lost signal to thread 10465
W20260709 13:18:11.582145  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08001ef8a0 after lost signal to thread 10474
I20260709 13:18:11.583269  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:11.584034  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003bafe0 after lost signal to thread 10477
W20260709 13:18:11.584313  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080020bcc0 after lost signal to thread 10480
I20260709 13:18:11.578930  6514 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:18:11.584542  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08000ab2a0 after lost signal to thread 10482
W20260709 13:18:11.584769  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080030f900 after lost signal to thread 10484
I20260709 13:18:11.539336 10501 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:11.585811  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003ba9e0 after lost signal to thread 10488
W20260709 13:18:11.589653  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002c67a0 after lost signal to thread 10489
I20260709 13:18:11.585903  6597 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
W20260709 13:18:11.590000  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003b9e20 after lost signal to thread 10491
I20260709 13:18:11.590337  6670 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:11.590644  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.593142  6663 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:11.594507  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.595643  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.588596  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:11.596359  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6880 after lost signal to thread 10492
I20260709 13:18:11.596472  7231 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:11.598420  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08001679e0 after lost signal to thread 10496
W20260709 13:18:11.601138  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080039d380 after lost signal to thread 10500
W20260709 13:18:11.601449  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080034e280 after lost signal to thread 10506
W20260709 13:18:11.601702  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800408500 after lost signal to thread 10507
W20260709 13:18:11.601907  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003e91c0 after lost signal to thread 10508
W20260709 13:18:11.602108  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080032bdc0 after lost signal to thread 10511
I20260709 13:18:11.602357  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:11.602396  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8320 after lost signal to thread 10512
W20260709 13:18:11.603066  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd7e0 after lost signal to thread 10513
W20260709 13:18:11.603273  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800281360 after lost signal to thread 10514
W20260709 13:18:11.603451  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800406be0 after lost signal to thread 10516
W20260709 13:18:11.603684  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003589c0 after lost signal to thread 10517
I20260709 13:18:11.602357  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:11.604007  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08003e6560 after lost signal to thread 10518
W20260709 13:18:11.604283  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800391140 after lost signal to thread 10522
I20260709 13:18:11.604236  6597 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:11.609851  6594 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:11.610819  7228 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:11.583540 10495 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:11.625532  6597 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:11.598004 10501 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:11.634083  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:11.609066 10428 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:11.575896 10551 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.547953 10557 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.642773  7200 leader_election.cc:400] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9.
I20260709 13:18:11.644325  7200 leader_election.cc:403] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:11.645183  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:11.646418  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.649031  6593 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:11.649930  6592 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:18:11.650305  6592 pending_rounds.cc:85] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 7
I20260709 13:18:11.650517  6592 pending_rounds.cc:107] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:11.650808  6592 raft_consensus.cc:2889] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:11.652156  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.653312  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.624660 10495 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.653401  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.587044 10552 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.680756  6590 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:11.686221  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:11.689185  6517 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:11.690975  6594 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:18:11.694764  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.695683  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.696802  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.699143  6588 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:11.700033  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.704234  6595 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:11.642304 10428 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:11.706239  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.655324 10492 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:11.710773  6593 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:11.676971 10495 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:11.677160 10569 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:11.681447 10491 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:11.714160  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:11.726430  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.727064  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:11.730584  6662 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:11.727732  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.732867  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:11.736644  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.733392  6588 raft_consensus.cc:2376] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 11. Re-sending same reply.
I20260709 13:18:11.711562 10428 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:11.761168  6593 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:18:11.769207 10589 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:11.687551 10554 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:18:11.636883 10514 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.775391 10514 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:11.775960 10552 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:11.730054 10519 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.777309 10519 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:11.778822  6591 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:11.781064  7228 raft_consensus.cc:2410] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:11.782636  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:11.783375  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:11.785066  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 69232us
W20260709 13:18:11.788233  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 74261.7us
I20260709 13:18:11.789218  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:11.792732  6589 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:11.794948  6593 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 13:18:11.796967  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.729612 10491 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.756742 10569 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:11.796322  6588 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:11.803509  6419 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 5, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 5 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:11.803627  6591 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:18:11.814895  6595 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:11.753367 10448 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:11.780529 10557 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:11.820111  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.821046  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.826809  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.829754  6588 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:11.833622  6596 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:11.826027  6662 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:11.809677 10491 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:11.857050  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.826519  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.859819  6419 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 3 to 4, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 4 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:11.852749  6588 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:11.853065  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:11.866508  7228 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:11.854571  6589 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:11.855446  6596 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:11.890094  6665 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:11.891409  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.901124  6589 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:18:11.905730  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:11.906661  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53683.4us
I20260709 13:18:11.939930  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:11.915175  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 126046us
I20260709 13:18:11.827687 10554 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:11.906107  6591 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:11.892261  6670 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:11.938529  6596 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:11.941767  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.943446  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:11.947546  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.822896 10439 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 13:18:11.961750  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:11.862349 10547 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:11.967509  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:11.971415  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:11.971432  6665 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:11.943928 10552 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:11.970636  6419 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 1 to 4, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 4 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:11.997730  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:11.956063 10281 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:12.008107  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:12.016762  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800249d40 after lost signal to thread 5413
W20260709 13:18:12.019368  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003293e0 after lost signal to thread 6454
W20260709 13:18:12.020519  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800288cc0 after lost signal to thread 6474
W20260709 13:18:12.020649 10594 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:18:12.023334  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:12.009827  6670 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:18:12.032186  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800210c60 after lost signal to thread 6546
W20260709 13:18:12.032902  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800210e60 after lost signal to thread 6547
I20260709 13:18:11.999439  6671 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:12.034227  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf4c0 after lost signal to thread 6621
I20260709 13:18:12.036188  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:11.983250 10486 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:11.994200 10552 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:12.038398 10440 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.002s
I20260709 13:18:12.037951 10569 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:11.957686 10549 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:12.082126  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.051740  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003292a0 after lost signal to thread 6695
I20260709 13:18:12.059369  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:12.102298  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.041066 10551 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:12.114001  6670 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:12.061394 10552 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:12.122380  6562 leader_election.cc:336] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.553s (SENT)
I20260709 13:18:12.039193 10554 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:18:12.141896  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:12.143440  6670 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:12.129295  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.127367  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:12.077286 10486 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:12.134814  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800314ec0 after lost signal to thread 7261
I20260709 13:18:12.072917 10477 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.146680  6671 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:12.141193  6592 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:12.158744  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.163134  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:12.058780 10457 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:18:12.169169  6665 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:12.166807  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080036bb60 after lost signal to thread 7262
I20260709 13:18:12.172580 10486 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.173899 10519 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:12.171635  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800273c00 after lost signal to thread 9873
W20260709 13:18:12.219378  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800372d80 after lost signal to thread 9925
W20260709 13:18:12.220225  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003299e0 after lost signal to thread 10266
W20260709 13:18:12.220539  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd600 after lost signal to thread 10270
W20260709 13:18:12.220840  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003cba40 after lost signal to thread 10280
W20260709 13:18:12.221151  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080028a9c0 after lost signal to thread 10281
W20260709 13:18:12.221475  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08001ff340 after lost signal to thread 10332
W20260709 13:18:12.221807  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08002eb720 after lost signal to thread 10396
W20260709 13:18:12.222131  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080029d320 after lost signal to thread 10398
W20260709 13:18:12.302877  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08001afd40 after lost signal to thread 10400
I20260709 13:18:12.158417 10169 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 13:18:12.255151  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.256138  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.329092  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.217447 10281 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:18:12.260707 10495 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 13:18:12.333869 10495 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.335196  7229 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:12.335470  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:12.336206 10495 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:12.263025  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 155873us
I20260709 13:18:12.336675  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:12.337337  7236 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:12.338135  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:12.269457  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:12.338819  7236 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:12.339375  7237 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:12.340320  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:12.341518  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:12.273619 10563 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:12.274439  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.343276  6663 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:12.344199  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.345021  6663 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:12.345878  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.277437  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:12.346990  7234 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:12.347954  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:12.348876  7234 raft_consensus.cc:2410] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:12.350092  7234 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:18:12.276717 10601 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:18:12.270429 10492 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.542s	user 0.000s	sys 0.007s
I20260709 13:18:12.352052 10492 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9.
I20260709 13:18:12.285604  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 895) took 1030 ms. Trace:
I20260709 13:18:12.352912  6597 rpcz_store.cc:276] 0709 13:18:11.255139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.643209 (+388070us) service_pool.cc:224] Handling call
0709 13:18:11.681902 (+ 38693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.711119 (+ 29217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.711150 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:11.750054 (+ 38904us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.057192 (+307138us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:12.057251 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.057341 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.285371 (+228030us) raft_consensus.cc:1685] finished
0709 13:18:12.285391 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.285528 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34957,"raft.queue_time_us":14104,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":32537,"thread_start_us":234026,"threads_started":1,"wal-append.queue_time_us":478501}
I20260709 13:18:12.355912  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.357292  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:12.281311  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 63709.5us
W20260709 13:18:12.361022  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.239485 10262 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:18:12.361832 10501 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.719s	user 0.006s	sys 0.005s
I20260709 13:18:12.363080  6663 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:12.364107  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:12.365996  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.269510 10549 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:12.312148  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:12.372699  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 82633.9us
I20260709 13:18:12.326117 10477 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.375067 10477 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.375928 10551 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:12.377054 10477 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:12.377619 10477 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:12.377839  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:12.379300 10495 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.330643  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:12.345957  7236 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:18:12.222878  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 261166us
I20260709 13:18:12.369346  7231 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:12.383772  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:12.369719 10428 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.564s	user 0.006s	sys 0.004s
I20260709 13:18:12.358469  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:12.387058  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.300916  6670 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:12.388432  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.373647 10492 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:12.316623  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4480 after lost signal to thread 10428
W20260709 13:18:12.390775  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003ea5a0 after lost signal to thread 10436
W20260709 13:18:12.391139  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080040ba20 after lost signal to thread 10439
W20260709 13:18:12.391877  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080035b520 after lost signal to thread 10440
W20260709 13:18:12.392549  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800351340 after lost signal to thread 10448
I20260709 13:18:12.373499  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:12.329869  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:12.393880  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800382bc0 after lost signal to thread 10457
W20260709 13:18:12.394887  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 59186.1us
W20260709 13:18:12.395617  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003ea760 after lost signal to thread 10459
I20260709 13:18:12.398231  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:12.396106  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080018c400 after lost signal to thread 10465
I20260709 13:18:12.404973  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.362278 10501 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:12.407335  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800272140 after lost signal to thread 10474
W20260709 13:18:12.418634  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080038f3a0 after lost signal to thread 10477
W20260709 13:18:12.418958  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7fe0 after lost signal to thread 10480
W20260709 13:18:12.419224  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800368400 after lost signal to thread 10481
W20260709 13:18:12.419478  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08002e68c0 after lost signal to thread 10482
W20260709 13:18:12.419736  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08002c9a00 after lost signal to thread 10484
W20260709 13:18:12.419991  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080030c480 after lost signal to thread 10486
W20260709 13:18:12.402254  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 323998us
W20260709 13:18:12.417312  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 90174.2us
I20260709 13:18:12.416344  6664 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:12.426378  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003fb600 after lost signal to thread 10488
W20260709 13:18:12.426795  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080035ae00 after lost signal to thread 10489
W20260709 13:18:12.427062  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800405960 after lost signal to thread 10490
W20260709 13:18:12.427342  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080039e460 after lost signal to thread 10491
W20260709 13:18:12.427595  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003f9a60 after lost signal to thread 10492
I20260709 13:18:12.431315  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:12.433599  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003c45e0 after lost signal to thread 10495
I20260709 13:18:12.377964 10477 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:12.434599  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.435209  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.433019  7237 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:18:12.433929  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2940 after lost signal to thread 10496
W20260709 13:18:12.439179  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003ed740 after lost signal to thread 10500
W20260709 13:18:12.440775  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800382600 after lost signal to thread 10501
W20260709 13:18:12.442205  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003d23e0 after lost signal to thread 10506
W20260709 13:18:12.444165  6543 debug-util.cc:398] Leaking SignalData structure 0x7b0800311660 after lost signal to thread 10507
I20260709 13:18:12.444576  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:12.402513 10492 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:12.451398  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:12.447427  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080035b8c0 after lost signal to thread 10508
I20260709 13:18:12.452193  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:12.452521  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08002c6780 after lost signal to thread 10511
W20260709 13:18:12.452981  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4b60 after lost signal to thread 10512
W20260709 13:18:12.453336  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080016f200 after lost signal to thread 10513
I20260709 13:18:12.402809 10551 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:12.446882  6670 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:12.457626  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:12.453238  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 75542.2us
I20260709 13:18:12.405416 10049 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:12.394745 10281 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.455598  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:12.456512  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003af560 after lost signal to thread 10514
W20260709 13:18:12.454869  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 81417.8us
I20260709 13:18:12.461938  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:12.416023 10439 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.459055  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:12.464710  6662 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:12.455504  6669 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:12.449055  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 387461us
I20260709 13:18:12.460945  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:12.451609 10491 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.545s	user 0.006s	sys 0.006s
I20260709 13:18:12.399858 10428 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:12.463331  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8360 after lost signal to thread 10516
I20260709 13:18:12.463394  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:12.467952  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08001be1a0 after lost signal to thread 10517
W20260709 13:18:12.468261  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080038bfe0 after lost signal to thread 10518
I20260709 13:18:12.465619  6590 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:12.460923  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:12.468575  6543 debug-util.cc:398] Leaking SignalData structure 0x7b08003aea60 after lost signal to thread 10522
I20260709 13:18:12.454206 10492 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.462989  6664 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
W20260709 13:18:12.445791 10514 consensus_meta.cc:287] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.001s
I20260709 13:18:12.468361  6663 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:12.468887  6543 debug-util.cc:398] Leaking SignalData structure 0x7b080039e420 after lost signal to thread 10532
I20260709 13:18:12.433120 10495 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:12.459779 10554 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:12.458952 10049 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.405017 10623 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:18:12.473804  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:12.462801 10616 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:18:12.513044  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:12.510668  6664 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:12.480445  6669 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:12.469496 10488 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.509624  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:12.471909 10492 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:12.492476  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.465217 10184 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:12.517104  6666 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:12.480113 10049 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:12.471264 10491 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.509796 10169 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:12.531864  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 62279.8us
W20260709 13:18:12.534296 10622 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:18:12.509400 10554 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:12.547084  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:12.536072 10439 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:12.555049  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.535524 10514 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:12.547947 10508 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:12.565559 10595 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.302s	user 0.001s	sys 0.001s
I20260709 13:18:12.508800 10621 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:18:12.534315 10488 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.576570  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2249) took 1023 ms. Trace:
I20260709 13:18:12.578431  7228 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:12.576223 10602 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 13:18:12.557381  6419 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.014s	sys 0.009s
W20260709 13:18:12.579312  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.548177 10169 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:12.574940  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 98895.4us
W20260709 13:18:12.570547 10557 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.743s	user 0.010s	sys 0.001s
W20260709 13:18:12.577675 10608 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 13:18:12.547513 10184 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:12.544831 10628 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:12.590255  6419 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.014s	sys 0.009s
I20260709 13:18:12.586934  6595 rpcz_store.cc:276] 0709 13:18:11.553635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.815822 (+262187us) service_pool.cc:224] Handling call
0709 13:18:11.830046 (+ 14224us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:11.889783 (+ 59737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.889822 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:11.956030 (+ 66208us) log.cc:844] Serialized 52 byte log entry
0709 13:18:12.327469 (+371439us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:12.327713 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.328131 (+   418us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.576339 (+248208us) raft_consensus.cc:1685] finished
0709 13:18:12.576360 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.576510 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6432,"raft.queue_time_us":45387,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":14713,"thread_start_us":298073,"threads_started":1,"wal-append.queue_time_us":486041}
I20260709 13:18:12.579054 10439 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:12.572844 10281 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:12.592980 10169 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:12.587247 10495 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:12.610862 10549 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.595125 10508 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:12.619174  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:12.595498 10184 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:12.595801 10616 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:12.621922 10281 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.622296  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:12.627331 10617 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:18:12.628131  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:12.627996 10628 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:12.621948  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1185) took 1046 ms. Trace:
I20260709 13:18:12.634501  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:12.609458 10439 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:12.637905  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2350) took 1064 ms. Trace:
I20260709 13:18:12.618300 10501 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:12.637183  7235 rpcz_store.cc:276] 0709 13:18:11.575663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.575787 (+   124us) service_pool.cc:224] Handling call
0709 13:18:11.583544 (+  7757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.596318 (+ 12774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.596343 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:11.596816 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.688854 (+ 92038us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:11.688895 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.690114 (+  1219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.564945 (+874831us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:12.620977 (+ 56032us) raft_consensus.cc:1685] finished
0709 13:18:12.621021 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.621883 (+   862us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34524544,"thread_start_us":91686,"threads_started":1,"wal-append.queue_time_us":661845}
W20260709 13:18:12.649597  6522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.005s	sys 0.019s
I20260709 13:18:12.609709 10552 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.644686  6667 rpcz_store.cc:276] 0709 13:18:11.574029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.574169 (+   140us) service_pool.cc:224] Handling call
0709 13:18:11.587686 (+ 13517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:11.613842 (+ 26156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.613888 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:11.639676 (+ 25788us) log.cc:844] Serialized 52 byte log entry
0709 13:18:12.069922 (+430246us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:12.113946 (+ 44024us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.114207 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.114319 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.629037 (+514718us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b1800468220. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:12.637643 (+  8606us) raft_consensus.cc:1685] finished
0709 13:18:12.637725 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.637852 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11068,"prepare.queue_time_us":43705,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4688,"raft.queue_time_us":25164,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":24328,"spinlock_wait_cycles":6302592,"thread_start_us":352475,"threads_started":1,"wal-append.queue_time_us":482491}
I20260709 13:18:12.621189 10549 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.651101 10495 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:12.680789  7230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.021s	sys 0.081s
W20260709 13:18:12.686015  7230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.022s	sys 0.082s
I20260709 13:18:12.659775 10486 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:12.644039 10281 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.664490 10465 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.667317 10428 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:12.726722 10448 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.685616 10563 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.698783  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2351) took 1077 ms. Trace:
I20260709 13:18:12.704073 10184 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.712812 10501 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:12.655140 10554 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:12.695871  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101036us
I20260709 13:18:12.733176 10501 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:12.733661 10501 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:12.733464 10495 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.734670 10501 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.735903 10501 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:12.737823 10488 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:12.738387 10488 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:12.759909  6561 leader_election.cc:336] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.143s (SENT)
I20260709 13:18:12.729883  6668 rpcz_store.cc:276] 0709 13:18:11.576105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.576234 (+   129us) service_pool.cc:224] Handling call
0709 13:18:11.589390 (+ 13156us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.601288 (+ 11898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.601317 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:11.638902 (+ 37585us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.858078 (+219176us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:11.858121 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.858200 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.575644 (+717444us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:12.648407 (+ 72763us) raft_consensus.cc:1685] finished
0709 13:18:12.648430 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.653244 (+  4814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":753,"raft.queue_time_us":11669,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":12782,"spinlock_wait_cycles":26767616,"thread_start_us":168566,"threads_started":1,"wal-append.queue_time_us":600835}
I20260709 13:18:12.732043 10448 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:12.764205 10448 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:12.765158 10448 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.737478 10439 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:12.768589 10439 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:12.771826 10049 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.774380 10049 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:12.766422 10448 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:12.759106 10169 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:12.779399 10508 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.781761 10169 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:12.782763  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:12.795634 10184 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:12.800760 10549 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:12.801510 10549 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:12.755661 10563 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.804545 10563 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:12.806821 10552 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:12.808084  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50681.6us
I20260709 13:18:12.777308 10492 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.810173  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:12.811744  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 396642us
I20260709 13:18:12.812598  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:12.818073  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 195844us
I20260709 13:18:12.820245  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:12.826292  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 364421us
I20260709 13:18:12.813427 10281 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:12.839445  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:12.838964  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 210887us
W20260709 13:18:12.871320  6636 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.502s (SENT)
I20260709 13:18:12.871433  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:12.872133  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:12.806355 10477 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:12.884001  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 275764us
I20260709 13:18:12.880913 10616 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:12.885804  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46468.4us
I20260709 13:18:12.887820  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:12.897259  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:12.897789  6662 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:12.884650 10184 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:12.901364  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:12.902021  6669 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:12.890666 10049 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:12.904068  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:12.906008  6666 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:12.906221  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:12.895605 10488 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:12.919384 10436 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:12.919900 10636 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:12.920452 10636 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:12.919909 10436 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:12.920941 10636 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:12.921371 10436 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:12.921725  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1191) took 1014 ms. Trace:
I20260709 13:18:12.922086  7228 rpcz_store.cc:276] 0709 13:18:11.906967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.944441 (+ 37474us) service_pool.cc:224] Handling call
0709 13:18:12.921501 (+977060us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54006cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:12.921652 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393684352}
W20260709 13:18:12.929904 10504 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:18:12.929965  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1175) took 3740 ms. Trace:
I20260709 13:18:12.930490 10263 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:12.931016 10263 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:12.930567  7230 rpcz_store.cc:276] 0709 13:18:09.189351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.198114 (+  8763us) service_pool.cc:224] Handling call
0709 13:18:09.198408 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.446228 (+247820us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b54007a4520. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:18:11.349856 (+1903628us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b58008e88d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fdd0aa 00007f04b905f447 00007f04b9077fe7 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4
0709 13:18:11.533507 (+183651us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:11.715464 (+181957us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b180042a1e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:18:12.074449 (+358985us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.929578 (+855129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.929632 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:12.929673 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.929753 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.929909 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":249935,"prepare.queue_time_us":65640,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":8009,"raft.queue_time_us":250485,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5106,"spinlock_wait_cycles":62962176,"thread_start_us":897057,"threads_started":2,"wal-append.queue_time_us":702102}
I20260709 13:18:12.931900 10263 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:12.933183  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 124105us
I20260709 13:18:12.934518  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:12.900581 10184 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.933285 10263 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:12.939055  6662 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:12.901710 10623 consensus_queue.cc:1243] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:12.940387  6666 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:12.945621 10436 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:12.950085  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 190189us
W20260709 13:18:12.955390  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.901198 10647 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:12.956001  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46643us
W20260709 13:18:12.957269  6561 leader_election.cc:336] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.623s (SENT)
I20260709 13:18:12.910130 10477 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:12.960268  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:12.964468  6561 leader_election.cc:336] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.597s (SENT)
I20260709 13:18:12.968058 10563 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.971576 10640 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.972150 10640 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.972803  6669 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:12.950946 10551 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:12.975353  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.975474 10639 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:18:12.976198 10486 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:12.977062 10428 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:12.977404  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 164886us
I20260709 13:18:12.977213 10486 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.977617 10428 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.978214  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:12.978826 10557 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:12.979312 10642 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:18:12.980059  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45623.3us
I20260709 13:18:12.980687 10514 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:12.981119 10281 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.964598 10645 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.979419  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:12.948205 10184 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:12.992643  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 45950us
W20260709 13:18:12.993672  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 87493.7us
I20260709 13:18:12.993901  6669 raft_consensus.cc:1217] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[5.4-9.6]   Dedup: 5.4->[7.5-9.6]
I20260709 13:18:12.994655  6669 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: [5.4-9.6]
I20260709 13:18:12.994693  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:12.981603 10552 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:12.974494 10640 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:12.907454 10641 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:18:12.959632 10647 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:12.954083 10616 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.999946 10049 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:12.996543 10514 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:13.036417 10049 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:13.037011 10049 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:13.005460 10504 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:13.037382 10514 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:12.990734 10448 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:18:13.039295 10643 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:12.988771 10648 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:18:12.979429 10557 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:13.008790  6562 leader_election.cc:336] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.597s (SENT)
I20260709 13:18:13.066658  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:13.059556  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:13.072638  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 65013.7us
I20260709 13:18:13.084764  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.084792  6517 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:12.998301 10440 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:18:13.103603  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.101557  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.108985  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.008107 10647 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:13.111999  6522 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:13.114539  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 105781us
W20260709 13:18:13.115767  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.078495 10661 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.119902 10661 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.121625 10436 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:13.059242 10504 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:13.124441  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.067340 10554 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:18:13.072156 10169 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.112478 10623 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:18:13.157747  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 142918us
I20260709 13:18:13.158203  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:13.158824  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 156974us
I20260709 13:18:13.123324 10504 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.158989  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.157480  6514 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:13.165884  6515 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:13.169013  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:13.169540  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.182511  6517 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:18:12.998449 10645 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:13.124903 10508 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:13.156809 10428 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:13.190268  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:13.172180  6561 leader_election.cc:336] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.822s (SENT)
I20260709 13:18:13.187510  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.189354  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:13.192350  7230 raft_consensus.cc:2410] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:13.159654 10563 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:13.191471  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.185669  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.162006 10477 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.197183  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:13.142457 10492 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:13.197331  7235 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:13.193252  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:13.204066  6597 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:13.207140  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:13.207664  7232 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:13.201623  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.209735  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.198355  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:13.200408  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:13.197887  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 193185us
I20260709 13:18:13.210729  7228 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:13.211369  6514 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:13.199576  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.212821  6520 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:13.201881  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:13.215065  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:13.215296  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:13.215492  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:13.215842  6591 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:13.216012  6590 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:13.216974  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:13.217965  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:13.198879 10641 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:18:13.218850  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:13.219028 10491 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:13.219396  6596 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:18:13.220163  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:13.220794  6592 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:13.221227  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:13.222405 10629 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.339s	user 0.010s	sys 0.001s
W20260709 13:18:13.223233 10650 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:13.224231  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:13.226476 10638 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 13:18:13.227542 10557 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:13.228361 10266 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:13.228782 10266 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:13.229681 10645 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:13.229597 10266 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:13.232733 10651 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:13.235088  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 1986) took 1072 ms. Trace:
I20260709 13:18:13.223507  7228 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:13.223661  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.237664  6517 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:13.187389 10281 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:13.240485  7234 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:13.166507 10661 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:13.241348 10436 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:13.245803  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:13.235466  6665 rpcz_store.cc:276] 0709 13:18:12.163173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.178822 (+ 15649us) service_pool.cc:224] Handling call
0709 13:18:12.223540 (+ 44718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.229003 (+  5463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.229090 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.258222 (+ 29132us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.660168 (+401946us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:12.660250 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.660451 (+   201us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.225926 (+565475us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:13.234794 (+  8868us) raft_consensus.cc:1685] finished
0709 13:18:13.234820 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.235033 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17989120,"thread_start_us":380426,"threads_started":1,"wal-append.queue_time_us":537700}
I20260709 13:18:13.250460  6513 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:13.250860 10514 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:13.252267  6518 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:13.199265 10508 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:13.168185 10169 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:13.194240 10552 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:13.282591 10552 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.283021  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.284113  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:13.284613  6516 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:13.284683 10552 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:13.188977 10660 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:13.285813 10660 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:13.251307 10514 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.286986 10514 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:13.286743 10660 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:13.254938  6591 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:13.256500  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:13.282311  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:13.290815  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:13.261507  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.291899  6520 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:13.263700  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:13.266041 10508 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:13.254148 10266 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:13.267282 10618 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:13.267995  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:13.271356 10428 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:13.272812 10647 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.311870  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.244071  6596 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:13.276196 10049 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:13.260561 10436 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:13.302330 10491 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:13.254930 10281 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.213878 10477 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:13.279842 10657 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:18:13.340209 10169 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:13.308387 10618 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:13.286461  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.317214 10440 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:13.288424  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:13.300585  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.301057 10551 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:13.301129 10457 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:18:13.304131  6562 leader_election.cc:336] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.822s (SENT)
I20260709 13:18:13.340560 10477 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:13.368178  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:13.310163  7237 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:18:13.310992  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47364.2us
I20260709 13:18:13.311067  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:13.312637  6519 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:13.278389 10184 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.330545  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.336647 10436 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.301927 10486 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:18:13.280387  6592 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:13.346720  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:13.383711  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72726.6us
I20260709 13:18:13.280458  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.302491 10630 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:13.364567  7232 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:13.365022  6520 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
W20260709 13:18:13.366515  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 78146.1us
I20260709 13:18:13.375800  6522 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:13.310395 10636 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.376842  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:13.328567 10491 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:13.379673  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.327103 10655 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:13.302304 10563 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:18:13.367852 10621 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:13.389469  7233 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
W20260709 13:18:13.390324  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:13.402506  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 175094us
I20260709 13:18:13.318116 10645 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:18:13.406106  6514 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:13.364734 10440 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:13.409945  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.438241  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.367708 10457 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:13.385515  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:13.383687 10654 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:13.411118  6561 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.939s (SENT)
W20260709 13:18:13.447376  6561 leader_election.cc:336] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.956s (SENT)
I20260709 13:18:13.401556 10049 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:13.401893 10623 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:13.445467  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 60068.4us
I20260709 13:18:13.446368  6517 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:13.375821 10557 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:13.447821  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:13.451011  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.417436 10491 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:13.456889  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:13.515969  6662 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:13.458894  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:13.516559  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:13.517076  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58276.7us
I20260709 13:18:13.433720 10465 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:13.462316  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.462960  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.519526  6664 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 5.
I20260709 13:18:13.463682  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:13.520562  6667 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:13.518281  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:13.464372  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.522646  6671 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:13.464938 10552 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:13.529481 10552 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:13.530784 10552 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.532616 10552 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:13.533433 10049 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:13.534013 10049 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:13.534811 10667 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
I20260709 13:18:13.537184  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.537456 10488 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.538116 10488 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.541042 10488 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:13.465495 10663 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:18:13.465955  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.544262  6513 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:13.466463 10669 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:13.467681  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.468225 10636 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:13.547109  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.474972  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.547852  6668 raft_consensus.cc:2376] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9. Re-sending same reply.
I20260709 13:18:13.547887  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.548576  7233 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:13.549505  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:13.549832 10281 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:13.550065  7233 raft_consensus.cc:2376] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8. Re-sending same reply.
I20260709 13:18:13.475713  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.551122  6669 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:18:13.476445  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:13.477820 10666 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
I20260709 13:18:13.468729 10665 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:13.553395  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.446147 10440 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:13.549513  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.555480  7237 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 7: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:13.560020  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:13.560956  7236 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:13.561478  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 257369us
I20260709 13:18:13.447033 10654 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:13.586829  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 68617.6us
W20260709 13:18:13.520700 10677 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:18:13.416507 10616 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:18:13.599709  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.601336  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:13.605082  6666 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:13.473901 10618 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.559144  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.601359  6665 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:13.604146  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:13.553339  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 160243us
I20260709 13:18:13.610777  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.611559  7237 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:13.561414 10551 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:13.452414 10621 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.650765 10621 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:13.651443  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.652221  6668 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:13.618728  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.654693  6669 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:13.658036  6667 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:13.627619  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:13.628786 10670 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:13.629244  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.632875 10618 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:13.519088 10557 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:13.675359  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.676195  7236 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:13.613885 10486 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:13.682789  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.567083 10465 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:13.598196 10654 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:13.620874 10551 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:13.601063 10665 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:13.663502  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.699327  6664 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:13.669479  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.598706 10049 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:13.672950  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.684015  6513 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:13.686555  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:13.690315  6662 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:13.691058  6665 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:13.692339  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.708056  6666 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:13.711328  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.712898  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.745541  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.747156  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.748121  6514 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:13.610512 10623 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:18:13.748859  6670 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:13.757504 10682 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:13.759692  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.777475  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.780401 10465 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:13.782032  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:13.777737  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.699162 10551 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:13.784175  7236 raft_consensus.cc:2410] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:13.696973 10486 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:13.787345  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.792999  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:13.762959  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 316078us
I20260709 13:18:13.785495  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.712364 10184 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:13.801715  6516 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.517s	user 0.013s	sys 0.000s
I20260709 13:18:13.802214  6516 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:13.804240  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 356858us
I20260709 13:18:13.758996 10665 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:13.788197 10504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.014s	sys 0.017s
I20260709 13:18:13.808759  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.760692 10685 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:13.829924  6519 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:13.839624  6519 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:18:13.840065  6519 pending_rounds.cc:85] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 6
I20260709 13:18:13.840329  6519 pending_rounds.cc:107] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:18:13.840588  6519 raft_consensus.cc:2889] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:13.802851 10648 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:13.829936 10563 consensus_queue.cc:1243] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:13.850255  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:13.794469  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:13.834483 10504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.015s	sys 0.018s
I20260709 13:18:13.844964  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:13.857865  7236 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 5.
I20260709 13:18:13.782256 10492 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:18:13.829098  6516 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:13.832367 10647 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:13.872956 10684 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:13.886397  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 92029.9us
I20260709 13:18:13.887392  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:13.888106  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:13.889365  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:13.889693  6516 raft_consensus.cc:2368] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:13.890800  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:13.893126  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 284374us
W20260709 13:18:13.892050  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 238988us
I20260709 13:18:13.899293 10263 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:13.891619 10645 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:13.871956 10563 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:13.870931 10621 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:13.873843 10477 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:18:13.899045 10686 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:13.943848  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:13.907122  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:13.911994 10563 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:13.970356  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.038726  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 203959us
W20260709 13:18:14.035580  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 94125.3us
I20260709 13:18:13.920895 10645 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:13.854652 10665 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:14.067529  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 160473us
I20260709 13:18:14.081617  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:14.082330  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:14.080468  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 136989us
I20260709 13:18:13.957664 10557 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:14.116398  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 51479.9us
I20260709 13:18:14.107690  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:14.112147  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:13.944396 10263 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:14.127838 10488 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:14.134732 10699 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:14.217276 10699 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:14.135655  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.135310 10701 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:14.138248 10691 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
W20260709 13:18:14.147260 10674 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
W20260709 13:18:14.148233 10689 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
W20260709 13:18:14.149163 10681 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 13:18:14.149441 10658 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:14.150067 10697 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:18:13.903949 10486 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:14.167685 10693 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:18:14.042838 10686 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:14.194128  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 82051.5us
I20260709 13:18:13.957702 10621 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:14.217814 10671 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.417s	user 0.001s	sys 0.001s
W20260709 13:18:14.222007  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 114384us
I20260709 13:18:14.252483 10682 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:14.236568  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:13.961515 10692 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.246s
I20260709 13:18:14.278245  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:14.343247  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:14.136891 10688 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 13:18:14.233582 10701 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:14.376261  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 97807.2us
I20260709 13:18:14.383426 10557 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.386610  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1090) took 1193 ms. Trace:
W20260709 13:18:14.387701  6563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.024s	sys 0.058s
I20260709 13:18:14.394739  7233 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
W20260709 13:18:14.403606  6563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.024s	sys 0.059s
W20260709 13:18:14.395581  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.008s	sys 0.012s
W20260709 13:18:14.403956  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 786574us
W20260709 13:18:14.404548  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.008s	sys 0.012s
I20260709 13:18:14.398897  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:14.408929  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 951053us
W20260709 13:18:14.412990  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 70031.4us
I20260709 13:18:14.279407 10694 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:14.413252  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.387588 10477 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.403507  7229 rpcz_store.cc:276] 0709 13:18:13.193054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.222101 (+ 29047us) service_pool.cc:224] Handling call
0709 13:18:13.222407 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:13.249238 (+ 26831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.249268 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:13.249681 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.516491 (+266810us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:13.516527 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.516593 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.386368 (+869775us) raft_consensus.cc:1685] finished
0709 13:18:14.386388 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.386523 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26449,"raft.queue_time_us":26656,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":266501,"threads_started":1,"wal-append.queue_time_us":528028}
I20260709 13:18:14.388543 10682 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:14.432771  6514 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.014s	sys 0.012s
W20260709 13:18:14.435685  6514 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.015s	sys 0.012s
W20260709 13:18:14.433259  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55775.1us
I20260709 13:18:14.244426 10563 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:14.438823  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:14.402817 10701 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:14.441816  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:14.479872  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:14.480700  6669 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:14.482025  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1120) took 1056 ms. Trace:
W20260709 13:18:14.485369  6407 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 921c423406da4f289884afca51ca198f (127.5.72.129:34121) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 921c423406da4f289884afca51ca198f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:14.485576  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46866.7us
W20260709 13:18:14.486101  6407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 60577.1us
I20260709 13:18:14.482901  7236 rpcz_store.cc:276] 0709 13:18:13.425958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.423812 (+997854us) service_pool.cc:224] Handling call
0709 13:18:14.481954 (+ 58142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.435359 10682 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:14.491503  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:14.475247 10492 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.498142  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:14.499035  7236 raft_consensus.cc:2410] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:14.498394  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1089) took 1305 ms. Trace:
I20260709 13:18:14.495962  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1086) took 1304 ms. Trace:
I20260709 13:18:14.495663  6669 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:18:14.495642  7229 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:14.504840  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1105) took 1098 ms. Trace:
I20260709 13:18:14.501654  7230 rpcz_store.cc:276] 0709 13:18:13.192814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.218725 (+ 25911us) service_pool.cc:224] Handling call
0709 13:18:13.218988 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:13.219232 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.219255 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:13.219664 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.407219 (+187555us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:13.407252 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.407313 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.498208 (+1090895us) raft_consensus.cc:1685] finished
0709 13:18:14.498227 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.498340 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":187226,"threads_started":1,"wal-append.queue_time_us":379139}
I20260709 13:18:14.505172  7229 rpcz_store.cc:276] 0709 13:18:13.406813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.479281 (+1072468us) service_pool.cc:224] Handling call
0709 13:18:14.504764 (+ 25483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.505954  6515 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:14.477942 10699 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:14.507330  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1109) took 1084 ms. Trace:
I20260709 13:18:14.507726  7236 rpcz_store.cc:276] 0709 13:18:13.423082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.488114 (+1065032us) service_pool.cc:224] Handling call
0709 13:18:14.507272 (+ 19158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.508736  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2139) took 1212 ms. Trace:
I20260709 13:18:14.509101  6517 rpcz_store.cc:276] 0709 13:18:13.284944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.318822 (+ 33878us) service_pool.cc:224] Handling call
0709 13:18:14.497241 (+1178419us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b5400044f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:14.497390 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124488064}
W20260709 13:18:14.511966  6521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.012s	sys 0.024s
W20260709 13:18:14.512320  6521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.012s	sys 0.024s
I20260709 13:18:14.441368 10477 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.502827  7231 rpcz_store.cc:276] 0709 13:18:13.192003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.192788 (+   785us) service_pool.cc:224] Handling call
0709 13:18:13.193050 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:13.203262 (+ 10212us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:13.203309 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:13.255616 (+ 52307us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.704621 (+449005us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:13.704679 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.704764 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.495697 (+790933us) raft_consensus.cc:1685] finished
0709 13:18:14.495739 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.495893 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1326,"raft.queue_time_us":12432,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":31749,"thread_start_us":402401,"threads_started":1,"wal-append.queue_time_us":597190}
W20260709 13:18:14.498021 10491 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.931s	user 0.005s	sys 0.014s
I20260709 13:18:14.420642 10645 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.425935 10557 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.519595  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:14.520009  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:14.514492  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:14.513494 10552 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.980s	user 0.007s	sys 0.008s
I20260709 13:18:14.523077 10491 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.531694 10709 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:14.532161 10709 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:14.532562  6663 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:14.533001 10491 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:14.521502 10707 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:18:14.533162 10709 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.531116 10690 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.509483 10492 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.492524 10686 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:14.559106  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 154145us
I20260709 13:18:14.518373 10563 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:14.564301  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:14.577524  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 179012us
W20260709 13:18:14.540159 10266 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.230s	user 0.011s	sys 0.003s
I20260709 13:18:14.487619 10263 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:14.535879 10695 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:18:14.728821  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:14.539525 10440 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.941s	user 0.010s	sys 0.000s
W20260709 13:18:14.542274 10678 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:18:14.534606 10690 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:14.636587  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 72357.3us
I20260709 13:18:14.758865 10495 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.759493  6563 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:14.759473 10495 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.761749  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:14.762152 10495 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:14.639232 10713 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:18:14.644001 10477 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:14.534562 10709 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:14.526881 10694 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:14.648057  7230 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:18:14.535429 10707 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:14.655287 10684 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:14.656292 10514 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.369s	user 0.010s	sys 0.002s
I20260709 13:18:14.657526 10621 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:14.659384 10704 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:14.534864 10703 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:14.672129  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:14.722308  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:14.724341  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:14.745172 10551 consensus_meta.cc:287] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.941s	user 0.012s	sys 0.000s
I20260709 13:18:14.746083 10557 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:14.728684 10436 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:14.755474  6670 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:14.758132 10440 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:14.637676 10492 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:14.754274  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:14.838271 10720 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:14.839517  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 352875us
W20260709 13:18:14.843366  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 171327us
I20260709 13:18:14.799783 10684 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.808133 10477 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.810405 10703 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.865666  6522 raft_consensus.cc:2368] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
W20260709 13:18:14.875787  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 121598us
I20260709 13:18:14.876526  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:14.897665 10720 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:14.898152 10655 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.898466 10720 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:14.898751 10655 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:14.899510 10720 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:14.851188 10684 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:14.844513 10716 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:14.854496 10694 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:14.811200 10514 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:14.863999 10618 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.182s	user 0.013s	sys 0.000s
I20260709 13:18:14.863662 10703 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:14.839251 10551 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.875240 10266 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.919227  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 157564us
I20260709 13:18:14.927759  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2473) took 1137 ms. Trace:
I20260709 13:18:14.864210 10508 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:14.929493  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1111) took 1505 ms. Trace:
I20260709 13:18:14.845489 10436 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.936846 10266 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.936942 10508 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.937817 10440 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:14.938094 10551 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:14.939226 10266 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:14.939363 10508 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:14.902835 10716 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.861678 10477 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:14.944798  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68321us
I20260709 13:18:14.896203 10719 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:18:14.946513  7235 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:14.948494 10722 raft_consensus.cc:2764] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:14.932813  6514 rpcz_store.cc:276] 0709 13:18:13.779789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.803255 (+ 23466us) service_pool.cc:224] Handling call
0709 13:18:13.832910 (+ 29655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.833178 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:14.439053 (+605875us) log.cc:844] Serialized 14 byte log entry
0709 13:18:14.916735 (+477682us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.916783 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:14.916823 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.916886 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.917010 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2193,"prepare.queue_time_us":13383,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":5465,"thread_start_us":1069093,"threads_started":2}
I20260709 13:18:14.934324  7236 rpcz_store.cc:276] 0709 13:18:13.423617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.508382 (+1084765us) service_pool.cc:224] Handling call
0709 13:18:14.518855 (+ 10473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.519101 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:14.808905 (+289804us) log.cc:844] Serialized 14 byte log entry
0709 13:18:14.927305 (+118400us) log.cc:844] Serialized 14 byte log entry
0709 13:18:14.928464 (+  1159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.928602 (+   138us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:14.928739 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.928942 (+   203us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.929304 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":397921,"raft.queue_time_us":289,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":4222,"thread_start_us":399822,"threads_started":2}
I20260709 13:18:14.971060 10719 consensus_queue.cc:1243] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:14.971740 10719 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:14.971868 10436 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:14.972378 10719 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.972496 10655 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:14.972863 10719 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:14.974125 10477 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:14.977037 10514 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:14.957811 10440 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.957183 10716 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:14.983345 10440 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:14.986805 10440 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:14.988131 10704 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:14.990694  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 116676us
I20260709 13:18:14.991439  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:14.978607 10724 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.998862  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:14.959821 10701 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:15.051256  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 52500.6us
I20260709 13:18:15.052994  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:15.054397 10266 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:15.055827 10266 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:15.056200 10266 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:15.062319  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:15.065654  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 88008.1us
I20260709 13:18:15.061105  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:14.983110 10682 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.080086  6562 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.077783  6668 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:15.083920  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:15.084024  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:15.084816  6589 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:15.090257 10694 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.092036  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:15.092150 10457 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.093456 10725 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:15.093787  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.093462 10736 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:15.107896  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.086272  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:15.011981 10724 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.110472  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:15.114030  6663 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:15.111326  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.115001  6665 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:15.059403 10719 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:15.116653  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.057044 10266 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.117551  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.085261 10563 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:15.082309 10436 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:15.065924 10733 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:18:15.121965  6590 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:15.117750  6595 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:15.119012  6594 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:15.121435  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.061045 10641 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:18:15.133777  6589 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:15.135126 10738 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:15.135344  6563 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 2.018s (SENT)
I20260709 13:18:15.135594 10266 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.125559 10720 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:15.135859  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.136443 10720 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.137054 10720 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:15.142083 10739 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:15.152281 10701 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.153828  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1207) took 1699 ms. Trace:
I20260709 13:18:15.156016 10703 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.161514 10716 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:15.162256 10679 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.475s	user 0.004s	sys 0.031s
I20260709 13:18:15.154389  7230 rpcz_store.cc:276] 0709 13:18:13.454378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.505563 (+1051185us) service_pool.cc:224] Handling call
0709 13:18:15.153535 (+647972us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400576a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:15.153744 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278080384}
I20260709 13:18:15.169040  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 792) took 1476 ms. Trace:
I20260709 13:18:15.132985  6593 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
W20260709 13:18:15.180387  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 221621us
W20260709 13:18:15.136381 10732 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:18:15.193853  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76370.9us
I20260709 13:18:15.173799  6515 rpcz_store.cc:276] 0709 13:18:13.692492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.781701 (+ 89209us) service_pool.cc:224] Handling call
0709 13:18:14.499704 (+718003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.506382 (+  6678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.168537 (+662155us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b5400044f50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:15.168858 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603761024}
I20260709 13:18:15.149164  6592 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:15.125962 10736 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:15.142527 10739 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:15.219895  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 84590.3us
I20260709 13:18:15.133239 10436 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.222302 10563 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:15.225539 10741 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.229346 10719 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:15.242945 10742 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.269040 10719 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.271577 10719 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.273342 10719 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.247998 10739 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:15.255148  6522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2465) took 1563 ms (client timeout 2073 ms). Trace:
W20260709 13:18:15.275312  6522 rpcz_store.cc:269] 0709 13:18:13.692006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.692111 (+   105us) service_pool.cc:224] Handling call
0709 13:18:15.254928 (+1562817us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400111054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:15.255084 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2676173184}
I20260709 13:18:15.275678 10739 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:15.273859 10719 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.231209  6590 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:15.269117 10742 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.247481 10724 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:15.257774 10436 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:15.286435  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.289587  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64023.9us
I20260709 13:18:15.237166 10703 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.298797 10563 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.302417  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.302815 10685 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:15.303156  6592 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:15.304217  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.305184 10725 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:15.305334  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:15.306135  6592 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:15.311981 10742 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.312266  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:15.312704 10621 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.313170  6590 raft_consensus.cc:2410] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:18:15.313202 10621 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.303206 10685 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:15.315560  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1093) took 2116 ms. Trace:
W20260709 13:18:15.315971 10705 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.933s	user 0.005s	sys 0.021s
I20260709 13:18:15.315656 10685 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.316025  7235 rpcz_store.cc:276] 0709 13:18:13.199578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.272085 (+ 72507us) service_pool.cc:224] Handling call
0709 13:18:13.272397 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.963065 (+1690668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.315286 (+352221us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54006a3bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:15.315481 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3849437056}
I20260709 13:18:15.316884 10685 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:15.265894 10720 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.319069 10640 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:15.319608 10640 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:15.320971  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.321791 10621 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:15.321966  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2469) took 1628 ms. Trace:
I20260709 13:18:15.322577  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:15.328274  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.329676  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.330329  6662 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:15.330724  7230 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:18:15.331998  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.332249  7230 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:15.331756  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:15.333280  6665 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:15.333571 10436 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:15.333952 10436 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.334306 10436 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:15.334244  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.334990  6662 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 6.
I20260709 13:18:15.336694 10640 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:15.337388  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:15.337575 10716 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:15.338253 10680 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.876s	user 0.017s	sys 0.006s
I20260709 13:18:15.339751  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1095) took 2123 ms. Trace:
I20260709 13:18:15.341033 10281 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.340819 10684 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.340086  7234 rpcz_store.cc:276] 0709 13:18:13.216783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.293259 (+ 76476us) service_pool.cc:224] Handling call
0709 13:18:13.293657 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:13.304611 (+ 10954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.304656 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:13.311526 (+  6870us) log.cc:844] Serialized 52 byte log entry
0709 13:18:13.407797 (+ 96271us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:13.407852 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.407912 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.337692 (+1929780us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:15.339456 (+  1764us) raft_consensus.cc:1685] finished
0709 13:18:15.339475 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.339680 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3624,"raft.queue_time_us":9128,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7114,"spinlock_wait_cycles":51529088,"thread_start_us":69935,"threads_started":1,"wal-append.queue_time_us":134062}
I20260709 13:18:15.341537 10281 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.285211 10736 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.343578 10281 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:15.347663  6596 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:15.350106  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:15.290692 10654 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.579s	user 0.002s	sys 0.012s
I20260709 13:18:15.355281 10654 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.355739  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:15.356096  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1098) took 2064 ms. Trace:
I20260709 13:18:15.356865  7236 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:15.357337  7235 pending_rounds.cc:85] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 7
I20260709 13:18:15.357795  7235 pending_rounds.cc:107] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:18:15.358055  7235 raft_consensus.cc:2889] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.356468  7229 rpcz_store.cc:276] 0709 13:18:13.292114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.508719 (+1216605us) service_pool.cc:224] Handling call
0709 13:18:14.509087 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.851280 (+342193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.851314 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.853392 (+  2078us) log.cc:844] Serialized 26 byte log entry
0709 13:18:14.989777 (+136385us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:14.996732 (+  6955us) log.cc:844] Serialized 14 byte log entry
0709 13:18:14.998395 (+  1663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.998472 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.334740 (+336268us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b18003fbda0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:15.355825 (+ 21085us) raft_consensus.cc:1685] finished
0709 13:18:15.355914 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.356038 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6689,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":7941,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":58327,"spinlock_wait_cycles":15537408,"thread_start_us":428890,"threads_started":2,"wal-append.queue_time_us":186353}
I20260709 13:18:15.322345  6513 rpcz_store.cc:276] 0709 13:18:13.694020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.779433 (+ 85413us) service_pool.cc:224] Handling call
0709 13:18:13.779789 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:13.780159 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.780190 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:13.780762 (+   572us) log.cc:844] Serialized 52 byte log entry
0709 13:18:13.907652 (+126890us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:13.907709 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.907784 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.315409 (+1407625us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:15.321693 (+  6284us) raft_consensus.cc:1685] finished
0709 13:18:15.321713 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.321907 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5940,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":12897,"spinlock_wait_cycles":35845888,"thread_start_us":126475,"threads_started":1,"wal-append.queue_time_us":601878}
I20260709 13:18:15.359683  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.360999  6592 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:15.368932  6663 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:15.344475 10436 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.375340  6662 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:15.378695  6665 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:15.378947  7229 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:15.379681  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:15.380447  6662 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:15.394594  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:15.423817 10676 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.809s	user 0.017s	sys 0.022s
I20260709 13:18:15.426041  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1092) took 2231 ms. Trace:
I20260709 13:18:15.426410  7228 rpcz_store.cc:276] 0709 13:18:13.194567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.246439 (+ 51872us) service_pool.cc:224] Handling call
0709 13:18:13.261107 (+ 14668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:13.261522 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.261563 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:13.262047 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.357888 (+ 95841us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:13.358018 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.358474 (+   456us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.423284 (+2064810us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:15.425754 (+  2470us) raft_consensus.cc:1685] finished
0709 13:18:15.425772 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.425980 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1109,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":17578880,"thread_start_us":95123,"threads_started":1,"wal-append.queue_time_us":175741}
W20260709 13:18:15.459080  6488 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.619s (SENT) [suppressed 43 similar messages]
I20260709 13:18:15.459244 10716 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:15.459373 10687 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:15.459877 10716 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.460366 10645 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:15.460119 10687 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.460899 10645 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.462394 10716 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.462814 10687 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.463011 10645 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:15.465621  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:15.466856 10645 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:15.467536 10645 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.467880 10645 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:15.469576 10751 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:15.476322 10645 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:15.479117  6562 leader_election.cc:336] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.765s (SENT)
I20260709 13:18:15.479674  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:15.509145  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.503474  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:15.533730  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.536142  6662 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:15.578748  6671 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:18:15.581315  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77935.7us
W20260709 13:18:15.582278  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 48689.6us
I20260709 13:18:15.583593  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.603082  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.600422  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.605136  6667 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:15.600091  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:15.600935  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 228014us
I20260709 13:18:15.607965  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:15.609321  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.610589  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.448083 10692 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:15.612282  6591 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:18:15.613495  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.612946  6664 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:15.613086  6665 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:15.618726  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.615517  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:15.624399  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 145308us
I20260709 13:18:15.627044  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.577742 10703 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.633769  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.577816 10660 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:15.642585  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.599802 10049 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:15.591895 10621 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.599818 10716 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:15.612686 10281 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:15.659407  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:15.632688 10707 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.008s	sys 0.018s
I20260709 13:18:15.612653 10640 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:15.663069  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:15.747386  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.749440  6597 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:15.638110 10694 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:15.612524 10655 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:18:15.658706 10708 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.630901 10477 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:15.630417 10514 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.620393 10628 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.128s
W20260709 13:18:15.759577  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.650079 10738 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:15.765311 10514 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:15.751979 10707 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.010s	sys 0.020s
I20260709 13:18:15.657820 10660 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.764161 10266 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:15.662918 10621 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.629878 10692 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.761392  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.772143  6597 raft_consensus.cc:2368] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:15.752584 10716 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:15.758226 10281 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.654881 10748 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:15.754390 10049 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:15.770056  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 310989us
I20260709 13:18:15.656960 10557 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:15.651361 10703 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.764161 10722 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.766585 10707 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:15.770327 10266 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:15.766641 10514 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.818217 10621 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:15.818912 10748 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:15.822247 10660 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.828864 10703 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:15.824065 10707 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.828270  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:15.885581  6663 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:15.847975 10049 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.816367 10692 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:15.779734 10436 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:15.827594  6670 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:15.836109 10736 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:15.845629  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.841113  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:15.905830  6591 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:15.906795  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.907799  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.831812 10716 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:15.834687 10684 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:15.762306 10708 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:15.855844  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 272336us
I20260709 13:18:15.925742  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:15.927451  6637 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:15.932668  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:15.932934  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:15.936069  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:15.938010  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.938768  6595 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:15.930421  6637 leader_election.cc:400] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:15.940838  6637 leader_election.cc:403] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:15.816009  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:15.820032  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:15.811401  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.821499 10722 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:15.819931 10633 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:15.892896  6663 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
W20260709 13:18:15.880627 10739 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.000s
I20260709 13:18:15.939612 10648 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.984966  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:15.903687 10645 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:15.999281  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.999693  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 183740us
I20260709 13:18:15.999742  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:15.998431  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.934926 10436 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:15.934739 10703 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:16.005314  6515 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:16.005487  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:15.937487 10698 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:18:16.002156  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:15.983376  6591 raft_consensus.cc:2410] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:18:15.963675 10741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.027s	sys 0.038s
I20260709 13:18:15.903318 10049 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.012917  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:16.013273  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 386381us
W20260709 13:18:16.017808  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.018203  6514 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:16.014142  7200 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.019s	sys 0.028s
I20260709 13:18:16.018733  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:16.033272  7200 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.020s	sys 0.028s
W20260709 13:18:16.033751  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 526865us
I20260709 13:18:16.033919  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.006609 10633 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:16.035142  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.020143  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:15.980973 10661 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:18:15.904461 10736 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:15.936599 10660 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:18:16.004846 10501 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:15.917168 10266 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.034775  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:16.042912  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.044899  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:16.046967  6595 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:16.025455 10436 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.060906  7234 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:16.062817  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:16.064733  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.032186 10448 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:16.065510  7231 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:16.040263 10623 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:16.073202  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:16.075950  7231 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:16.043946 10687 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:16.076150  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.001391 10748 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.079021  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2140) took 2793 ms. Trace:
I20260709 13:18:16.081763  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.082729  7231 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:16.053125 10641 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:16.021616 10648 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.059207 10436 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.071655 10621 raft_consensus.cc:2764] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:16.057542 10741 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:16.088205  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.080507  6520 rpcz_store.cc:276] 0709 13:18:13.285217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.406708 (+121491us) service_pool.cc:224] Handling call
0709 13:18:13.439281 (+ 32573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.449788 (+ 10507us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:13.523761 (+ 73973us) log.cc:844] Serialized 14 byte log entry
0709 13:18:15.147155 (+1623394us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b1800018d60. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:15.777262 (+630107us) spinlock_profiling.cc:243] Waited 881 us on lock 0x7b1800018d60. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:18:16.078280 (+301018us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.078574 (+   294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.078646 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:16.078732 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.078842 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.078971 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33604,"prepare.queue_time_us":786267,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":1756,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":3468,"spinlock_wait_cycles":17334912,"thread_start_us":431527,"threads_started":2,"wal-append.queue_time_us":1867768}
I20260709 13:18:16.088469  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.068073 10281 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.118947 10281 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.120002 10741 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.121145 10281 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.121773 10281 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:16.122206 10741 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.074666 10724 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:16.122831 10724 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:16.091198 10514 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:16.123541 10514 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:16.123950 10514 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:16.123725 10724 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.124769  6591 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:16.124872 10724 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.126013  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.126356  7234 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:16.075524 10640 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.091612  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:16.127220  7231 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.126539  6589 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.073323 10501 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:16.126025  6591 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:16.096186 10736 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:16.099992 10768 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:18:16.101910  6561 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.721s (SENT)
W20260709 13:18:16.133456  6561 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.779s (SENT)
I20260709 13:18:16.133944  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:16.103701 10773 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:18:16.104269 10766 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:18:16.104972  6590 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:16.087553 10641 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.107055 10777 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:16.061169 10636 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:18:16.115216 10708 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.084043 10440 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:16.130417  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.083803 10690 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.083509 10623 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:16.092140 10703 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:16.137559 10694 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:16.147030  6514 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:16.147302  6591 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:16.077590 10448 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.168020 10457 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.181382  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:16.182036  6591 raft_consensus.cc:606] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:16.182351  6591 raft_consensus.cc:614] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:16.184273  6591 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:16.185290 10655 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.185878 10655 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.188624 10655 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.122542 10281 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:18:16.153196 10771 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:16.190066  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 796) took 1914 ms. Trace:
I20260709 13:18:16.191497 10648 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.190598  6519 rpcz_store.cc:276] 0709 13:18:14.275926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.276063 (+   137us) service_pool.cc:224] Handling call
0709 13:18:14.276376 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.425190 (+1148814us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b180028d860. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b9079bfb 00007f04b908464b
0709 13:18:15.425314 (+   124us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:15.656236 (+230922us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.031391 (+375155us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.031882 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.032020 (+   138us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:16.032061 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.032143 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.189722 (+157579us) raft_consensus.cc:1685] finished
0709 13:18:16.189843 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.189997 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2532,"prepare.queue_time_us":606083,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":215640,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":17970,"spinlock_wait_cycles":25983232,"thread_start_us":716821,"threads_started":3,"wal-append.queue_time_us":435391}
I20260709 13:18:16.087208 10687 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.141352 10641 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.192731  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2378) took 1674 ms. Trace:
I20260709 13:18:16.193080  6668 rpcz_store.cc:276] 0709 13:18:14.518098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.518273 (+   175us) service_pool.cc:224] Handling call
0709 13:18:14.525700 (+  7427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.081121 (+555421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.192507 (+1111386us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54000c58d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:16.192666 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250346240}
I20260709 13:18:16.170356 10703 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:16.172616 10623 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.173623 10440 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.124805 10514 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.179370 10719 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.179515 10049 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.167335 10501 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.169749 10736 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.203753  6669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:16.216236  6669 raft_consensus.cc:606] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:16.208936 10687 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.209129 10682 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.209620 10703 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.212705  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.213917 10557 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:16.215024 10514 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.166697 10777 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:16.200686  7231 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
W20260709 13:18:16.222906  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 227670us
I20260709 13:18:16.241050  6637 leader_election.cc:400] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:16.242297  6637 leader_election.cc:403] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:16.145226  6589 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
W20260709 13:18:16.230273  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.221707  6593 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:16.196125 10448 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.237967  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:16.238225  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.246927  6519 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.245327  6591 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:16.238792 10557 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:16.329421  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.332437  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:16.332552  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:16.337199  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2277) took 1248 ms. Trace:
I20260709 13:18:16.340724  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.341589  6516 raft_consensus.cc:2410] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:18:16.337828  6594 rpcz_store.cc:276] 0709 13:18:15.082655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.082788 (+   133us) service_pool.cc:224] Handling call
0709 13:18:15.083068 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.132426 (+ 49358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.330978 (+1198552us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b54002fc1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:16.331178 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81950464}
I20260709 13:18:16.344175  6522 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:16.297286 10703 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:16.350610  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 309740us
W20260709 13:18:16.346933  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 301556us
I20260709 13:18:16.353328  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.355422  6516 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 7: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:16.355916  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.353084  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:16.365573  6520 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:16.376405  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.383033  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.400296  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.401804  6514 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:16.403882  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 325876us
W20260709 13:18:16.405870  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.412853  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 277657us
I20260709 13:18:16.421576  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:16.389026 10723 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:18:16.438144  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:16.439520  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 86468.4us
I20260709 13:18:16.402401 10712 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.451102  6594 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:18:16.451915  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.452824  6589 raft_consensus.cc:2368] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:16.408856 10687 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:16.443893  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:16.455476  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:16.447844 10514 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:16.450380 10648 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.498600 10648 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.498895 10633 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.500087 10633 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.501086 10648 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.461438  6594 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:16.463706 10436 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.457082 10687 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.504076  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.464838 10781 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:16.449609 10785 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:18:16.467743  6589 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:16.512692  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.513970  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.469956  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.471138 10504 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:18:16.471763  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 60368.1us
I20260709 13:18:16.472735  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:16.466534  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:16.517084  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50619.1us
I20260709 13:18:16.518409  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:16.481945  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.519377  6592 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:16.482224 10630 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:16.483003  6597 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:16.419296 10777 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.485821 10720 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.486013 10557 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.453827 10712 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.454851 10698 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:16.427569 10457 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:18:16.511425  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.525363 10714 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:16.520521  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65161.6us
I20260709 13:18:16.522365  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:16.538293 10792 raft_consensus.cc:2764] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:18:16.539098 10792 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:16.533638 10661 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:16.523429 10514 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:16.528915 10621 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:16.540223 10266 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.526244 10682 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:16.530066 10557 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.530290 10648 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:16.541538 10648 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:16.539433 10792 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:16.542405 10648 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.542990 10792 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.543916 10648 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.544013 10792 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.540027 10789 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:18:16.525141  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52509.2us
I20260709 13:18:16.549652  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 789) took 2939 ms. Trace:
I20260709 13:18:16.530887 10698 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:16.555608  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.567636  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.585161  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:16.556407  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 302542us
I20260709 13:18:16.589365  6671 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
W20260709 13:18:16.571188  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.559119  6519 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:16.532979 10436 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.582827  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:16.582515  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:16.591120  6668 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 6.
I20260709 13:18:16.591370  6663 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:16.573874  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:16.564040 10722 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.570429  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:16.595956  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:16.593381  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:16.597280  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.598672 10641 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.600493 10720 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.601296 10641 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.602023 10641 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:16.603106  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:16.594434 10746 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:16.605094 10746 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.566694 10740 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:18:16.607584  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:16.607661 10746 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.608721 10707 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:16.608964 10746 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:16.609206 10707 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:16.609272  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:16.610139 10549 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:16.610235 10707 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.568902 10658 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:16.611650 10658 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.537180 10714 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.614925 10714 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.616365 10720 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:16.594900 10698 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.550211  6518 rpcz_store.cc:276] 0709 13:18:13.609933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.610070 (+   137us) service_pool.cc:224] Handling call
0709 13:18:13.629914 (+ 19844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:13.658995 (+ 29081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.659021 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.427306 (+768285us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b180034cf00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a
0709 13:18:16.107872 (+1680566us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b2c000014c8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:18:16.108330 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.407559 (+299229us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:16.407651 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.407834 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.549371 (+141537us) raft_consensus.cc:1685] finished
0709 13:18:16.549400 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.549578 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28712,"raft.queue_time_us":34891,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":8474,"spinlock_wait_cycles":22546048,"thread_start_us":298822,"threads_started":1,"wal-append.queue_time_us":434450}
I20260709 13:18:16.622196  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.622421  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.622845  7231 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:16.623178  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:16.628893  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.637959  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:16.574227 10763 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.550s	user 0.010s	sys 0.000s
I20260709 13:18:16.640358  7234 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:16.641326  6671 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:16.645054  6665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:16.645761  6665 raft_consensus.cc:606] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:16.646190  6665 raft_consensus.cc:614] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:16.540179 10514 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:16.622862 10694 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:18:16.568279 10654 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.577718 10440 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.640285 10557 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:16.566834 10049 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:16.576077 10682 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:18:16.651381  6521 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:16.652967  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.642678 10641 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:18:16.575700 10687 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.659277  6664 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:16.660305 10740 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.660688  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.661062  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:16.661998  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.676594 10714 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:16.664675  6636 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.854s (SENT)
I20260709 13:18:16.671866  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:16.708724  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44090.8us
I20260709 13:18:16.579473 10701 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:16.709756 10701 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.674494  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.713130  6589 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:16.714237  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.714942  6589 raft_consensus.cc:2376] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 9. Re-sending same reply.
I20260709 13:18:16.716005  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.717340  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.718000  6589 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 7.
I20260709 13:18:16.675215  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.719156  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.675333  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:16.719071  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 58084.7us
I20260709 13:18:16.720304  6516 raft_consensus.cc:2410] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:18:16.595186 10719 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:16.678090  6518 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:18:16.723214  6518 pending_rounds.cc:85] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 8
I20260709 13:18:16.723165  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.723651  6518 pending_rounds.cc:107] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:18:16.723086  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.724124  6518 raft_consensus.cc:2889] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:16.681897  7233 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:16.727560  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.695353  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:16.566257 10640 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:16.644768 10741 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:16.670603 10514 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.703926 10557 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:16.728327  6592 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:16.728946  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:16.729427  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:16.728394  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 210038us
I20260709 13:18:16.733152  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.734225 10658 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.730248 10740 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.735035  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.742882  7235 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:16.727238 10654 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:16.679033 10720 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:16.728123 10687 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:16.728310 10694 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:16.792387  7229 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:18:16.763147  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.792832  7229 pending_rounds.cc:85] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 9
I20260709 13:18:16.793154  7233 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:16.794150  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.794835  7233 raft_consensus.cc:2368] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:16.703780 10440 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:18:16.767271  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.775420 10720 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:16.764372 10724 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:16.817171 10724 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.817553 10724 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:16.782637  6636 leader_election.cc:336] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.036s (SENT)
I20260709 13:18:16.818202  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:16.819902 10049 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:16.759284 10623 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:16.825132  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2474) took 1522 ms. Trace:
I20260709 13:18:16.825866  6595 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:16.793181  7229 pending_rounds.cc:107] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:18:16.797101  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.826507  7229 raft_consensus.cc:2889] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:16.790977 10457 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:18:16.790503 10654 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.790529 10687 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.832544 10687 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:16.802915 10794 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.060s	user 0.003s	sys 0.000s
I20260709 13:18:16.803531  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.845822  7236 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:16.846936  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.803785 10630 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:16.805192  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2272) took 1742 ms (client timeout 1876 ms). Trace:
I20260709 13:18:16.809937  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:16.776244 10736 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:16.815595  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 916) took 1522 ms. Trace:
I20260709 13:18:16.817327  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:16.849926  7232 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:16.848112  6596 rpcz_store.cc:269] 0709 13:18:15.062260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.083632 (+ 21372us) service_pool.cc:224] Handling call
0709 13:18:16.804970 (+1721338us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400210a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:16.805104 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418514176}
W20260709 13:18:16.778496  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 49656.7us
I20260709 13:18:16.823022  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:16.825567  6597 rpcz_store.cc:276] 0709 13:18:15.303119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.779923 (+476804us) service_pool.cc:224] Handling call
0709 13:18:16.824946 (+1045023us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540034fb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:16.825072 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118241024}
I20260709 13:18:16.796023 10557 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.814215 10712 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:16.821063 10646 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.858444  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:16.829561  6664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:16.858842  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:16.859234  6664 raft_consensus.cc:606] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:16.859644  6664 raft_consensus.cc:614] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:16.829356 10698 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:16.830502  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.838850  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2022) took 1306 ms. Trace:
W20260709 13:18:16.844087  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.865975  6589 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.730716 10719 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.855561  7236 raft_consensus.cc:2368] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:16.821357 10694 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:16.849742  6588 rpcz_store.cc:276] 0709 13:18:15.293434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.068582 (+775148us) service_pool.cc:224] Handling call
0709 13:18:16.068900 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.127775 (+ 58875us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:16.130870 (+  3095us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.456324 (+325454us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.456377 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:16.456423 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.815348 (+358925us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b54001b6f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:16.815398 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.815537 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44636,"prepare.queue_time_us":9531,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":1052,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":48969,"spinlock_wait_cycles":218752000,"thread_start_us":325060,"threads_started":1,"wal-append.queue_time_us":550474}
I20260709 13:18:16.874017  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.875201  7233 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:16.876562  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.791136 10789 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:18:16.865939  6662 rpcz_store.cc:276] 0709 13:18:15.532454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.539927 (+  7473us) service_pool.cc:224] Handling call
0709 13:18:15.540313 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.540572 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:16.047676 (+507104us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.838510 (+790834us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.838560 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:16.838601 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.838684 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.838791 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238790,"prepare.queue_time_us":533430,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":2758,"raft.queue_time_us":242955,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":145,"spinlock_wait_cycles":26548480,"thread_start_us":982780,"threads_started":3,"wal-append.queue_time_us":405476}
I20260709 13:18:16.877206  7236 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:16.878641  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:16.887367  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.850229 10630 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:16.849476 10741 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.855995 10714 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:16.899401  6594 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.901561  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.902190  7236 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:16.902266  7232 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:16.903103  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.903802  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:16.903759  7236 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:16.859288 10808 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:16.907126  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.907426  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.912870  7236 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:16.857224 10746 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:18:16.910511  6588 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:16.916644  6589 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:16.920882  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:16.921754  6589 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:16.903153  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.917671  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:16.858942 10682 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:18:16.856350 10736 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.919327  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.933087  6597 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:16.857143 10648 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:18:16.929734  7236 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:16.865417 10633 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:18:16.939534  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:16.940200  6669 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:16.940375  6519 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:16.909364 10808 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.944235  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:16.946519  6662 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:16.946719  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.944743  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:16.950100 10791 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.479s	user 0.000s	sys 0.003s
I20260709 13:18:16.957652  6664 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:16.959079  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.960052  6588 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:16.961742  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2484) took 1656 ms. Trace:
I20260709 13:18:16.975623  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:16.905239 10741 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:16.979301  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:16.979450  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:16.983408  6662 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 6: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:16.986272  6588 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:16.988143  6662 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:16.952232 10808 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:16.995195  6597 raft_consensus.cc:2410] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:16.997465  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:16.998086  6664 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:17.001515  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.002556  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.000739  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:17.003895  6636 leader_election.cc:400] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:17.004681 10704 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.005184 10704 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.005650 10719 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.007356 10704 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.007402 10698 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:17.005080 10809 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:16.969069 10630 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.008216 10809 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.008596 10809 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.017823  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.019522  6592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:17.020126  6592 raft_consensus.cc:606] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:17.020462  6592 raft_consensus.cc:614] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:16.951411 10694 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.024266 10809 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.027510  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.028965  6588 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.031209  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 919) took 1737 ms. Trace:
I20260709 13:18:17.032342  6595 rpcz_store.cc:276] 0709 13:18:15.294204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.088537 (+794333us) service_pool.cc:224] Handling call
0709 13:18:16.825547 (+737010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.826206 (+   659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.031009 (+204803us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540034fb50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:17.031149 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1631794560}
I20260709 13:18:17.000530  6594 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:17.021725  6592 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:18:17.034025  6592 pending_rounds.cc:85] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 2
I20260709 13:18:17.034291  6592 pending_rounds.cc:107] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:18:17.034554  6592 raft_consensus.cc:2889] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:17.004165  6636 leader_election.cc:403] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:17.038406  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:17.040992  6595 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:17.041047  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:17.037832  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:17.048431  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:17.051553  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 293032us
I20260709 13:18:17.074417  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.077093  6588 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:17.081707  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:17.032187 10049 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:18:17.055240 10703 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:17.055045 10661 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:17.088299  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:17.089368  6593 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:17.049996 10775 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:17.071650 10682 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.075812 10658 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:17.095536  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.080868 10809 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.097467  6593 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:17.089730 10787 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:17.100113  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.099061  6588 raft_consensus.cc:2410] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:18:17.115072 10787 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:17.115533 10787 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:17.116293  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.107479  7237 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:17.117574  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.118752  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:17.108280  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.119801  6595 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:17.120410  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.107127 10775 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:17.132876  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.111413  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1107) took 1052 ms. Trace:
I20260709 13:18:17.111578  7236 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.133422  7235 rpcz_store.cc:276] 0709 13:18:16.059375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.073561 (+ 14186us) service_pool.cc:224] Handling call
0709 13:18:17.111194 (+1037633us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b5400575dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:17.111327 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483502848}
I20260709 13:18:17.112288 10623 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.094102 10703 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:17.109473 10701 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.123872  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.126267  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:17.129914  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.110705 10645 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:17.135095  7237 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.148931  6593 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:17.135437  7234 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.136404  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.092923 10661 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:17.141426  7231 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.141227  6522 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:17.144624  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.145416  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:17.159317  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.159422  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.161525  6520 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.166040  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.175666  6518 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.176112  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.179299  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.179814  7234 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:17.179934  6522 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:17.124949 10712 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.125321 10658 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:17.182272  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.182657  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:17.139868 10709 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.140846 10623 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.190131  7235 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:17.190793  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.144194 10703 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:17.199090  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 116016us
I20260709 13:18:17.187239  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.199765  6593 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:17.195942  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:17.146351 10645 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:17.166975 10661 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.103828 10682 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.190745  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.190336 10789 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:17.198642  6563 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:17.207592  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.201362  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:17.202167  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:17.106389 10808 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.166195 10736 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.197976 10709 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.213615  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:17.214731  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.210736  6516 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:17.224126  7235 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.216710  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:17.225697  6666 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:17.218446  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.227387  6667 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:17.227437  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:17.229672  7229 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.148007 10724 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:17.233495  7237 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:17.220293 10633 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:17.235127  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:17.236282  6518 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.203903 10266 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:17.210000 10816 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:18:17.237743  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:17.243727  6670 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:17.245025  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.238400  7231 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:17.238888  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:17.239032  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.240248  6588 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:17.247448  6593 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:17.250018  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.251921  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.252605  6668 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:17.255479  6667 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:17.220112 10719 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:17.259610  6664 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:17.262018  6517 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:17.262018  7235 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:17.223254 10701 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:17.214195 10789 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:17.265936  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2288) took 1886 ms. Trace:
I20260709 13:18:17.267035  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:17.271100  6515 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:18:17.271924  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.272635  6670 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:17.266364  6591 rpcz_store.cc:276] 0709 13:18:15.379364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.184979 (+805615us) service_pool.cc:224] Handling call
0709 13:18:16.245110 (+ 60131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.247409 (+  2299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.265756 (+1018347us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b540019d2d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:17.265877 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137659136}
I20260709 13:18:17.275296  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.275786  6591 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:17.273636  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:17.279392  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:17.199026 10712 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.284372  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.240044 10741 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.242978 10716 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:17.286670  7233 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:17.286903  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:17.288682  6666 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:17.269793 10719 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.289361  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.284821 10704 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:17.270774 10701 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.296631  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.310024  7231 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.270933 10789 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:17.271046 10722 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.279310 10648 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:17.280728 10549 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:17.302029  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:17.303217  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.314630  6667 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:17.284451 10266 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:17.305922  6670 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:17.312031  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.321388  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.314932  6591 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:17.316983  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.322746  7231 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:18:17.319696  6668 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:17.297462 10741 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:17.350507  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.351300  6591 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.263408 10645 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.290401 10746 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:18:17.302053 10714 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:18:17.337441  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:17.315963 10648 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:17.288534 10830 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:18:17.348413 10716 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:18:17.317687 10823 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:18:17.309563 10719 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:17.304392 10633 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:18:17.313138 10701 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:17.309139 10049 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:18:17.353813 10266 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:17.386368  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62895.2us
I20260709 13:18:17.373302 10648 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:17.395818  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 90923.4us
I20260709 13:18:17.331538 10722 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.371219 10440 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:18:17.389927 10824 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:18:17.375849 10645 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:17.404615  6637 leader_election.cc:400] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:17.414880  6637 leader_election.cc:403] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:17.301819 10658 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:17.409263  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.410336  6663 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:17.388017 10716 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:17.403450 10808 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:17.390928 10830 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:18:17.429206  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:17.417727 10827 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:18:17.476838  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57867.7us
I20260709 13:18:17.475255  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.487366 10266 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:17.489735 10687 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.002s
I20260709 13:18:17.491457 10722 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:17.492458 10833 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:18:17.494210 10712 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.494808 10724 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:17.427719 10803 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.446s	user 0.004s	sys 0.000s
I20260709 13:18:16.962119  6590 rpcz_store.cc:276] 0709 13:18:15.305695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.153102 (+847407us) service_pool.cc:224] Handling call
0709 13:18:16.170651 (+ 17549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.170903 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:16.661253 (+490350us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.961273 (+300020us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.961439 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.961482 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:16.961534 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.961594 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.961691 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51486,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":85,"thread_start_us":750001,"threads_started":2}
W20260709 13:18:17.423672 10825 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:18:17.435071 10826 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 13:18:17.463702 10654 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:17.454144 10720 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.499398 10720 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:17.499636 10716 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:17.504287 10822 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:18:17.504603  6666 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:17.504437 10723 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:17.505139 10723 consensus_queue.cc:1243] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:17.505590  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:17.506181  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.506687  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.507598  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.505626 10808 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:17.508512 10808 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.508695  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.508980  6670 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:17.510046  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.511229  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.511770  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:17.470425  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.430596  6521 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:17.513403  6593 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:17.513624  6588 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.419929 10440 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:17.504338  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.514071 10741 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.514590  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.525811  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:17.560585  6519 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:17.508903 10808 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:17.537000  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 132575us
I20260709 13:18:17.534159  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:17.562948  6671 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
W20260709 13:18:17.532465  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45393.3us
I20260709 13:18:17.564134 10781 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:17.565168  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:17.565737 10781 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:17.566776 10281 raft_consensus.cc:2764] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:17.566854 10640 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:17.568779  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.569437  7236 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:17.570207 10448 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:17.540378  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:17.571362  6521 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.571708 10648 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.572538 10623 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:17.573856 10834 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 13:18:17.575052  6513 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:18:17.490190 10687 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:17.556492  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:17.575076  6515 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:17.577270  7200 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:17.511821  6591 raft_consensus.cc:2376] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 10. Re-sending same reply.
I20260709 13:18:17.486037 10682 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.565816 10839 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:17.578554  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.579803  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.580396  6591 raft_consensus.cc:2368] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:17.580999  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.581493  6591 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:17.582170  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.589600  6521 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:17.518373  6670 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:17.528398 10457 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:17.592695  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:17.559806  6590 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:17.593297  6671 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.575284  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:17.594228  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.594751  7234 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:17.595207  6591 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:17.549446 10646 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.685s	user 0.005s	sys 0.009s
I20260709 13:18:17.597793  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:17.598526  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:17.525269 10738 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:17.602515  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.534184 10830 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 8.
I20260709 13:18:17.605016  6514 raft_consensus.cc:2368] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 7. Current term is 9.
I20260709 13:18:17.608781  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.609469  6593 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:17.512934 10840 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:18:17.612641  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:17.622429  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:17.613665 10501 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:18:17.597679 10457 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:17.631428 10838 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 13:18:17.633340 10832 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
W20260709 13:18:17.634092 10836 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:18:17.634910 10746 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:17.636231 10630 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:17.574344 10714 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:18:17.597951 10640 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:17.603958 10514 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.840s	user 0.009s	sys 0.003s
W20260709 13:18:17.608307 10829 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
I20260709 13:18:17.653383  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.564147 10504 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:18:17.611120 10646 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:17.641547 10821 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.400s	user 0.000s	sys 0.001s
I20260709 13:18:17.611874 10844 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:17.593079 10741 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:17.683107 10844 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:17.683673 10844 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:17.684993 10741 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:17.686733  6590 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:17.668696  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:17.645737 10785 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.027s	sys 0.030s
I20260709 13:18:17.670686 10723 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:17.674918 10704 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:17.676846 10740 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.645893 10457 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.652631 10746 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:17.678891 10787 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.002s
I20260709 13:18:17.616812 10738 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.711774  6520 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:17.652808 10049 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:18:17.712832  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 150638us
I20260709 13:18:17.678267 10623 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.714816 10687 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:17.725054  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 95804.3us
W20260709 13:18:17.700881 10785 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.027s	sys 0.030s
I20260709 13:18:17.712966 10746 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.710529 10648 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:17.731793  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.681830 10514 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.724071 10722 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:18:17.735751  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.750910  7232 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:17.756844  6591 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:17.740172  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1125) took 1046 ms. Trace:
I20260709 13:18:17.718793 10787 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.738051 10722 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:17.766115  7228 rpcz_store.cc:276] 0709 13:18:16.693624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.693771 (+   147us) service_pool.cc:224] Handling call
0709 13:18:16.722710 (+ 28939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.731947 (+  9237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.731981 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.817617 (+ 85636us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b18002b19e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a
0709 13:18:16.818040 (+   423us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b2c00047568. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:18:16.818547 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.091121 (+272574us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:17.091173 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.091266 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.640884 (+549618us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:17.739959 (+ 99075us) raft_consensus.cc:1685] finished
0709 13:18:17.739979 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.740113 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1391,"raft.queue_time_us":1477,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":41982848,"thread_start_us":235925,"threads_started":1,"wal-append.queue_time_us":368150}
W20260709 13:18:17.730978 10799 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.008s	user 0.008s	sys 0.000s
I20260709 13:18:17.771699  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2294) took 1072 ms. Trace:
I20260709 13:18:17.786012  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.790089  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.723335 10440 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:17.800103  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:17.795564 10738 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.783491  6596 rpcz_store.cc:276] 0709 13:18:16.698922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.025759 (+326837us) service_pool.cc:224] Handling call
0709 13:18:17.026042 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.026221 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:17.143959 (+117738us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.466029 (+322070us) spinlock_profiling.cc:243] Waited 9.64 ms on lock 0x7b1800432a00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a
0709 13:18:17.761171 (+295142us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.767705 (+  6534us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.771300 (+  3595us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:17.771376 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.771465 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.771624 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":168399,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":7644,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":37011,"spinlock_wait_cycles":22176000,"thread_start_us":484976,"threads_started":3,"wal-append.queue_time_us":158847}
I20260709 13:18:17.799404  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2301) took 1042 ms. Trace:
W20260709 13:18:17.791596  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 214388us
I20260709 13:18:17.808316  6665 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:17.803988  6591 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:18:17.808646  6671 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:17.811064  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.813162  6591 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:17.815003 10722 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:17.815279  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:17.816121 10722 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.817293 10514 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:17.799912 10853 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:17.818085 10853 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:17.819684 10719 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.820166 10853 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:17.821638 10719 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:17.823167 10848 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:18:17.825662  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.859310  6591 raft_consensus.cc:2410] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:17.799790 10684 raft_consensus.cc:2764] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:18:17.828835 10703 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.000s
I20260709 13:18:17.916692 10703 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.919041 10703 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.806883  6592 rpcz_store.cc:276] 0709 13:18:16.756849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.047328 (+290479us) service_pool.cc:224] Handling call
0709 13:18:17.073925 (+ 26597us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:17.079235 (+  5310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.079268 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:17.090935 (+ 11667us) log.cc:844] Serialized 52 byte log entry
0709 13:18:17.238905 (+147970us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:17.265803 (+ 26898us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.266043 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.266143 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.799171 (+533028us) raft_consensus.cc:1685] finished
0709 13:18:17.799192 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.799350 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27226,"raft.queue_time_us":6051,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":11618,"spinlock_wait_cycles":16336768,"thread_start_us":106683,"threads_started":1,"wal-append.queue_time_us":214788}
I20260709 13:18:17.920658  6592 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:17.922138  6592 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:17.925417  6592 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:17.927907  6592 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:18:17.831131 10845 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:18:17.832537 10661 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.000s
I20260709 13:18:17.932524 10661 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.934487 10661 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.935123  6664 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:17.836879  6593 raft_consensus.cc:2410] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:17.840992 10808 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:17.937196 10701 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.002s
I20260709 13:18:17.937479 10808 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:17.843434 10787 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.847429 10844 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.937733 10701 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.939688 10844 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:17.904481  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.940621  6671 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:17.941253 10630 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.941713 10630 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:17.943478 10789 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:18:17.944407 10630 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.915542  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.972831  6591 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:17.945938 10709 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.947149  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 927) took 1221 ms. Trace:
I20260709 13:18:17.952173 10746 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.960757  6667 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:17.973745  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.973686  6594 rpcz_store.cc:276] 0709 13:18:16.725864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.732847 (+  6983us) service_pool.cc:224] Handling call
0709 13:18:17.946956 (+1214109us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540023cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:17.947092 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321287040}
I20260709 13:18:17.947746  6595 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:17.966754  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.962731 10701 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.976830 10746 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:17.972361 10789 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.978734 10789 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:17.979394  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 256462us
I20260709 13:18:17.980049  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:17.980485 10841 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:17.981739  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:17.916611 10457 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:17.811822  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:17.982555  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:17.995846  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 185262us
I20260709 13:18:17.985410  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.997262  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:17.986116  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1127) took 1126 ms. Trace:
I20260709 13:18:17.997782  6594 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.997967  7229 rpcz_store.cc:276] 0709 13:18:16.859672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.888569 (+ 28897us) service_pool.cc:224] Handling call
0709 13:18:17.985895 (+1097326us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54004e0554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:17.986046 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718173952}
I20260709 13:18:17.982364 10841 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:17.997488  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:18.001555  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:17.999037  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:18.003609  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.005285  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.007159  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:18.008723  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.011636  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.013645  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:18.017725  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:18.027153  6488 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:18.027194  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.031463 10723 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:18.032600  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.031780 10640 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:17.999491 10839 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:17.982964 10850 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:17.996500 10785 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:18.106105  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79012.1us
W20260709 13:18:18.106385  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65837.9us
I20260709 13:18:18.106971  6637 leader_election.cc:400] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:18.024905 10477 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:18.089360  6594 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:18.055718  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.107761  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:18.028010 10844 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:18.113508  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:18.114204  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:18.115654  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:18.024739 10789 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:18.033636 10440 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:18.032501 10787 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:18.025364 10808 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:18.025068 10514 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:18.046512 10687 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:18.131338  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:18.024636 10858 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:18.047070 10504 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:18.127303  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.127501  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.104590 10850 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:18.114967 10841 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:18.027007 10781 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:18:18.029157 10777 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:18:18.105610 10501 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:18:18.142040  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:18.123675 10712 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:18.144706  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:18.143271  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:18.148454  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:18.110960 10740 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:18:18.158721 10851 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:18.168444 10457 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:18.149387 10658 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:18.152513 10630 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:18.174880 10630 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:18.175719 10630 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.187253  6590 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:18.141830 10850 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:18.128544 10687 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.109045 10645 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:18:18.137917 10701 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:18.115159 10648 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:18:18.129300 10785 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:18:18.149986 10844 raft_consensus.cc:2764] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:18:18.188957  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44321.4us
I20260709 13:18:18.194428 10862 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.207151 10862 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.208209 10712 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:18.209265 10712 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:18.211236  6595 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:18.197535 10687 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:18.214322 10861 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:18:18.190713 10839 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:18.220988  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:18.190892 10658 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:18.227479 10658 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:18.192471 10661 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.231063 10661 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:18.233001 10661 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:18.149299  6594 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:18.125968 10808 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.184449 10789 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:18.202984  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:18.244863  6663 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:18.236800  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:18.243685  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:18.235831 10701 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:18.247792 10701 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:18.251263 10714 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.251778 10714 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.253392  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.253712 10714 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:18.255533  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:18.256171 10809 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:18.256523 10809 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:18.256824  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:18.257508  6664 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:18.257311 10809 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.258221  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:18.265443  6664 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:18.258319 10809 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:18.261456  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1137) took 1488 ms. Trace:
I20260709 13:18:18.273100  7232 rpcz_store.cc:276] 0709 13:18:16.772796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.902909 (+130113us) service_pool.cc:224] Handling call
0709 13:18:18.261275 (+1358366us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b540005bc54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:18.261402 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904273792}
I20260709 13:18:18.244144  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2489) took 1106 ms. Trace:
I20260709 13:18:18.274610  6516 rpcz_store.cc:276] 0709 13:18:17.138081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.222430 (+ 84349us) service_pool.cc:224] Handling call
0709 13:18:17.256623 (+ 34193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.264100 (+  7477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.264147 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.264206 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:17.264252 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.243935 (+979683us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b540007e4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:18.243976 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.244091 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16594304}
I20260709 13:18:18.264449  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:18.269599  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.276573  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 169413us
I20260709 13:18:18.277343  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:18.278127 10777 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:18.278784  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.279539 10777 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:18.279877  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:18.280606 10858 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:18.280973 10858 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:18.281306 10858 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:18.288281 10858 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.289219 10661 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:18.289958 10661 consensus_queue.cc:1243] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:18.290334 10863 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:18.290777 10863 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:18.290740  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:18.291996 10661 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:18.291818 10863 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:18.292467 10661 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:18.292955 10661 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:18.293008 10863 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:18.290323 10858 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:18.301600  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:18.302637  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:18.303406 10699 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:18.303851 10699 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:18.304231 10699 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:18.304512 10858 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:18.304724  7229 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:18.305058 10858 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:18.305938 10858 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.310261  7231 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:18.311815  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:18.313768  7231 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:18:18.315930  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.233104 10687 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.317387  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:18.317497  7232 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:18.321524  6513 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:18.321926  7231 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:18.324843  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:18.326397  6518 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:18:18.541397 10858 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:18.541807  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.261379 10808 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:18.335918 10844 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
W20260709 13:18:18.394761  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.506951  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 397529us
W20260709 13:18:18.504603  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 263248us
W20260709 13:18:18.342207  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 51558.6us
I20260709 13:18:18.566048  6637 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.547274 10719 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.548628  6519 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:18.541368  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:18.549036 10514 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:18.544628 10856 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:18.542958  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 278606us
I20260709 13:18:18.540393 10699 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.545918 10661 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.551115  6663 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:18.254241 10714 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:18.337543 10787 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:18.338392 10704 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:18.552676  6636 leader_election.cc:336] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 3.249s (SENT)
I20260709 13:18:18.553755  6594 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:18.559578  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2480) took 1437 ms. Trace:
I20260709 13:18:18.560324  6517 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:18.587535  6514 rpcz_store.cc:276] 0709 13:18:17.121914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.122026 (+   112us) service_pool.cc:224] Handling call
0709 13:18:18.559402 (+1437376us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54003cd054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:18.559525 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025646336}
I20260709 13:18:18.566397  6664 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:18.567039  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.556346 10858 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.576406  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:18.593001  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:18.569612 10690 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.382714 10777 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:18.596942  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.569619 10557 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:18.600523  6663 pending_rounds.cc:85] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
I20260709 13:18:18.602452  6663 pending_rounds.cc:107] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:18:18.602850  6663 raft_consensus.cc:2889] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:18.575070 10640 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:18:18.601019  7228 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:18.572888 10851 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:18.603699  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:18.605458  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:18.614886  6668 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:18.577301 10787 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:18.569283 10719 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.639047 10787 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:18.640461 10719 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:18.643944 10850 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:18.649781 10281 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:18.650300 10281 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:18.651540 10831 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.300s	user 0.006s	sys 0.002s
I20260709 13:18:18.651330 10281 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.652508 10281 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:18.652661  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 796) took 1654 ms. Trace:
I20260709 13:18:18.653810  6597 raft_consensus.cc:2376] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9. Re-sending same reply.
I20260709 13:18:18.653092  6662 rpcz_store.cc:276] 0709 13:18:16.998566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.004441 (+  5875us) service_pool.cc:224] Handling call
0709 13:18:17.032912 (+ 28471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.036362 (+  3450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.036408 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.039660 (+  3252us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.202025 (+162365us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:17.202081 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.202171 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.651262 (+1449091us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b240021b5e8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5e1a9c4 00007f04b5e18acb 00007f04b5fd3dc9 00007f04b5fd3a55 00007f04b5fd309d 00007f04b8c0510e 00007f04b8c1c88c 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221
0709 13:18:18.652371 (+  1109us) raft_consensus.cc:1685] finished
0709 13:18:18.652391 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.652591 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17630,"raft.queue_time_us":1872,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":11123,"spinlock_wait_cycles":4934528,"thread_start_us":118321,"threads_started":1,"wal-append.queue_time_us":215716}
I20260709 13:18:18.655047  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:18.655189  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2502) took 1387 ms (client timeout 1661 ms). Trace:
W20260709 13:18:18.655539  6597 rpcz_store.cc:269] 0709 13:18:17.268407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.505931 (+237524us) service_pool.cc:224] Handling call
0709 13:18:18.655033 (+1149102us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540019d2d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:18.655138 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2637898496}
I20260709 13:18:18.656054  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:18.656456 10736 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:18.656946 10736 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:18.657768 10736 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:18.625509 10862 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:18.627707  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1130) took 1856 ms. Trace:
I20260709 13:18:18.661693 10863 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:18.661391  7233 rpcz_store.cc:276] 0709 13:18:16.770852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.917804 (+146952us) service_pool.cc:224] Handling call
0709 13:18:16.918173 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.287214 (+369041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.627468 (+1340254us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b5400567f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:18.627647 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846963456}
I20260709 13:18:18.662341 10863 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:18.663308 10863 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:18.664215 10864 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.097s	user 0.002s	sys 0.000s
I20260709 13:18:18.664537 10863 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:18.665390  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 799) took 1078 ms. Trace:
I20260709 13:18:18.665830  6669 rpcz_store.cc:276] 0709 13:18:17.587027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.762997 (+175970us) service_pool.cc:224] Handling call
0709 13:18:17.764726 (+  1729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.982828 (+218102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.982863 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.983313 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.147125 (+163812us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:18.147164 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.147235 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.663473 (+516238us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:18.665082 (+  1609us) raft_consensus.cc:1685] finished
0709 13:18:18.665103 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.665327 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217739,"raft.queue_time_us":218146,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":66,"spinlock_wait_cycles":9917824,"thread_start_us":163430,"threads_started":1,"wal-append.queue_time_us":256010}
I20260709 13:18:18.667938  6669 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:18:18.668982  6668 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:18:18.669939 10777 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:18.670526 10777 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.671439  6593 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:18:18.672494  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:18.672619 10777 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:18.673578  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:18.674098  6593 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:18.674010 10701 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.674939  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2420) took 1170 ms. Trace:
I20260709 13:18:18.675277  6666 rpcz_store.cc:276] 0709 13:18:17.504120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.507530 (+  3410us) service_pool.cc:224] Handling call
0709 13:18:17.507809 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.528026 (+ 20217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.528072 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.528585 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.248547 (+719962us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b180006dd20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:18.516355 (+267808us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:18.516411 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.516509 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.674677 (+158168us) raft_consensus.cc:1685] finished
0709 13:18:18.674700 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.674885 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19824,"raft.queue_time_us":50069,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":23013,"spinlock_wait_cycles":15499648,"thread_start_us":267564,"threads_started":1,"wal-append.queue_time_us":1046544}
I20260709 13:18:18.597944 10840 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:18:18.677414 10840 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:18.677731 10840 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:18.678442 10840 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.632071 10867 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:18.679555 10840 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:18.679888 10867 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.681918 10867 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:18.632251 10640 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:18:18.632630  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.682427 10640 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:18.683343 10640 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.683861 10858 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:18.684468 10640 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:18.686751 10633 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:18.688593 10740 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:18.689621  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:18.631007 10789 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.691293  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:18.692286 10830 raft_consensus.cc:2764] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:18:18.635771 10687 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:18.676307 10694 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:18.693408  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:18.605319 10781 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:18.695523  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2034) took 1425 ms. Trace:
I20260709 13:18:18.695518  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:18.734724  6666 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
I20260709 13:18:18.734174  6665 rpcz_store.cc:276] 0709 13:18:17.270026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.270843 (+   817us) service_pool.cc:224] Handling call
0709 13:18:17.289839 (+ 18996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.810868 (+521029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.694997 (+884129us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b540011dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:18.695464 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191935232}
W20260709 13:18:18.696635  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.736966  6596 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [8.9-11.11]
I20260709 13:18:18.738749  6597 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:18.695577  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:18.739985  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:18.740326  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44818.5us
I20260709 13:18:18.587015 10844 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:18.692665 10775 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:18.758477  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:18.733619  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:18.755738  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:18.595142 10839 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.143s
I20260709 13:18:18.767050  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:18.606848 10514 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.144s
I20260709 13:18:18.763469  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:18.672495 10699 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:18.676174 10777 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:18.606710 10690 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.736248 10740 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:18.762785  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:18.757442  6593 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
W20260709 13:18:18.768445  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 194640us
I20260709 13:18:18.768419  6662 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:18.770493  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:18.705756 10851 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:18.786947  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.773707  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:18.788013  6596 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:18.784874  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:18.621457 10557 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.758769 10714 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:18.832654  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:18.833802  6593 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:18:18.835750 10704 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:18.840855 10690 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:18.779028 10867 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:18.827641 10719 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:18.828951 10699 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:18.875914 10867 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.876595 10699 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.877944 10867 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:18.847524  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.852219  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 278220us
I20260709 13:18:18.854502  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:18.875032 10704 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:18.786531 10839 consensus_queue.cc:1243] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:18.880219  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:18.758641 10630 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 13:18:18.745947 10781 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.758422 10787 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:18.893323  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:18.786926 10775 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:18.917456  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:18.918426  7233 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:18.924038  7232 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:18.927928  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.939625  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:18.940442  7233 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:18.880795 10549 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:18.892452 10630 consensus_queue.cc:1243] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:18:18.951992  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.952137  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 164207us
W20260709 13:18:18.952785  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.958374  6596 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:18.934158 10873 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.944696 10549 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:18.969694  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.970757  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:18.971975  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:18.973229  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:18.985404  7230 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:18.973546  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:18.974846  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:18.897132 10719 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:18:18.986564  7236 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:18.986604  7234 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:18.985404  7232 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:18.987645  7233 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:18.959240 10858 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:18:18.934010 10808 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:18.953337 10704 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:19.004789  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:19.005033  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:19.015776  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:19.014992  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.016597  7229 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:19.016598  6514 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:18.959491 10699 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.018741  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:19.020764  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:19.021785  7236 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:19.022924  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.023911  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:19.024439  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:19.025195  7232 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:19.029325  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:19.031010  7232 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:18.976679 10870 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:19.000484 10549 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:19.040055  7229 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:19.015344 10557 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.045691  7228 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 8. Ops: []
I20260709 13:18:19.033918 10830 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:19.000223 10775 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:19.002590 10777 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.011013 10883 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:18:19.050859 10871 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:18:19.053119  6514 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:18.976841 10873 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.060966  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:19.029740 10714 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:19.066730 10876 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:18:19.069267 10781 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.070781  6666 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:18:19.073241  7230 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:19.055202 10830 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.060292 10630 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:19.080606 10630 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:19.039361 10870 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:19.083066  6521 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 13:18:19.083994 10883 consensus_queue.cc:1243] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:19.084375  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:19.057777 10850 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:19.085105  6521 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:19.034090  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:19.084548 10883 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:19.092626  7235 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:19.010886 10851 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.097419  7233 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:19.099084  7236 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:19.053658  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:19.090730  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:19.100239  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46650.8us
I20260709 13:18:19.100911  6596 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:18:19.102548  6596 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:18:19.104769  6596 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:18:19.104802  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.116444  6596 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 13:18:19.127928  7234 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:19.140949  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:19.142812  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:19.092684 10883 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:19.099092 10740 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:19.159498  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.103894 10557 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:19.146021 10281 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.165405  6589 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:18:19.155990  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:19.154202 10690 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.153597 10850 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:19.188081  6521 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:19.150985 10787 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:19.157907 10436 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:19.204785  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.204453  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 812) took 1695 ms. Trace:
I20260709 13:18:19.151393 10870 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.227149  6520 rpcz_store.cc:276] 0709 13:18:17.508887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.592947 (+ 84060us) service_pool.cc:224] Handling call
0709 13:18:17.618475 (+ 25528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.719416 (+100941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.203610 (+1484194us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b5400355054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:19.204189 (+   579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193266432}
I20260709 13:18:19.229058 10898 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.274075  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:19.222198 10869 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:18:19.166997 10281 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.271368 10808 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:19.327148  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.275774 10436 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:19.217942 10885 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:19.213989 10690 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.341157 10630 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:19.341693 10640 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.008s
I20260709 13:18:19.315161 10740 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.344169 10436 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:19.338358 10897 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:18:19.349712 10704 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:19.350391  6595 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:18:19.344965 10863 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.680s	user 0.007s	sys 0.002s
I20260709 13:18:19.274544 10830 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:19.279572 10787 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:19.350504  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.311225 10850 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:19.354972  6669 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:18:19.352856  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 459568us
I20260709 13:18:19.366406  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:19.234874 10714 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.371153  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.373026  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:19.371838  6669 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:19.340531 10885 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.378741  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.381912  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:19.375259 10885 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:19.384616  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.407390  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.409118  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.409394  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:19.409672  6669 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:19.311750 10898 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:19.386260 10777 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.405164 10646 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:19.403059  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 247115us
W20260709 13:18:19.412586 10872 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
W20260709 13:18:19.417829  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 130492us
W20260709 13:18:19.415409 10889 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
W20260709 13:18:19.416761 10892 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:18:19.416838 10709 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:19.415478 10902 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.285444 10883 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:19.428324  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 154341us
W20260709 13:18:19.491734  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.428526  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:19.479478  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:19.499473  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 71021us
I20260709 13:18:19.426470 10646 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.503695 10840 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:19.504107 10893 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:18:19.505136  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2331) took 1282 ms. Trace:
I20260709 13:18:19.505487  6594 rpcz_store.cc:276] 0709 13:18:18.222505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.587993 (+365488us) service_pool.cc:224] Handling call
0709 13:18:18.588433 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.589000 (+   567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.589067 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.589936 (+   869us) log.cc:844] Serialized 26 byte log entry
0709 13:18:19.054257 (+464321us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:19.054304 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.055872 (+  1568us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.504923 (+449051us) raft_consensus.cc:1685] finished
0709 13:18:19.504944 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.505077 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56853,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":464026,"threads_started":1,"wal-append.queue_time_us":644301}
W20260709 13:18:19.512959  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 103631us
I20260709 13:18:19.518812  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2435) took 1329 ms. Trace:
I20260709 13:18:19.519342  6667 rpcz_store.cc:276] 0709 13:18:18.189621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.191566 (+  1945us) service_pool.cc:224] Handling call
0709 13:18:18.191988 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.198027 (+  6039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.198074 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.198707 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.416965 (+218258us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:18.417015 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.417095 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.518540 (+1101445us) raft_consensus.cc:1685] finished
0709 13:18:19.518567 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.518738 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6933,"raft.queue_time_us":6584,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":33405,"thread_start_us":197816,"threads_started":1,"wal-append.queue_time_us":830855}
I20260709 13:18:19.521698  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.523202  7235 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:19.430540 10850 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:18:19.506575 10049 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:19.494254 10906 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:19.497864 10787 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:19.517874 10899 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:19.498369 10908 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
W20260709 13:18:19.533586 10882 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:18:19.539259 10049 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:19.494077 10902 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:19.538111 10840 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:19.561117  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:19.564366  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.565862  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:19.566457  6591 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:19.566812  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.544872 10646 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:19.524626 10907 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:18:19.573828  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2161) took 2351 ms. Trace:
I20260709 13:18:19.578909  6517 rpcz_store.cc:276] 0709 13:18:17.222826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.271091 (+ 48265us) service_pool.cc:224] Handling call
0709 13:18:18.560082 (+1288991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.587571 (+ 27489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.569838 (+982267us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54003cd050. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:19.573749 (+  3911us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2922043264}
I20260709 13:18:19.574543  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:19.575419  6522 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:19.545245 10912 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:19.544215 10830 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:18:19.543895 10787 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.586918  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.582794  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:19.728443  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 145803us
W20260709 13:18:19.730099  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.585403  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.624728  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:19.629046  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:19.565295 10840 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:18:19.627267 10916 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:18:19.584789 10906 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.588846 10902 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.585086 10781 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.155s
I20260709 13:18:19.774209  6667 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:19.739967 10905 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:18:19.583410 10049 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:19.761868 10549 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.713s	user 0.015s	sys 0.000s
W20260709 13:18:19.766793  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 142184us
W20260709 13:18:19.626133 10917 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:19.590950 10785 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:19.653688 10920 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:19.784032  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:19.735114 10907 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:19.738358 10640 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:19.706156  6591 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:19.585362 10925 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:19.769289 10910 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.786891 10899 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:19.846657  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:19.894953 10899 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.895861  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:19.774884 10850 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:19.898177 10850 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:19.898406 10910 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:19.899410 10850 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.901409 10910 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:19.849332 10925 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:19.774209 10906 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.852630 10808 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:19.902649 10925 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:19.902988  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.902981 10808 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.904191 10906 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:19.854875 10914 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
I20260709 13:18:19.905395 10808 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.907032 10640 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:19.908291 10640 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.856101 10787 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:19.857012  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:19.859380 10928 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.859884 10907 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:19.860870 10840 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.862752 10785 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:19.867625  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:19.870347  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:19.875690 10923 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:18:19.876221 10903 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:18:19.839253 10549 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:19.853767 10919 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:18:19.897411  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:19.938750  6596 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:19.939316  6591 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:19.916908  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:19.918774 10785 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:19.921900 10907 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:19.913517 10910 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.924898 10746 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:18:19.942122  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 381084us
I20260709 13:18:19.932987  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:19.945005  6596 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:18:19.955453  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.954001  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:19.957705  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:19.930563 10787 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.968194  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:19.883322 10920 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:19.986296  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:19.946521 10840 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:19.990141  6591 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:19.998749 10922 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:19.999307 10922 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:19.999776  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:19.954763 10746 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.000145 10922 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.000999  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.001474  7233 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:20.002142  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.002691  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:20.003182  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 955) took 1111 ms. Trace:
I20260709 13:18:20.003491  7237 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 6.
W20260709 13:18:20.004618  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.003608  6590 rpcz_store.cc:276] 0709 13:18:18.891470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.493001 (+601531us) service_pool.cc:224] Handling call
0709 13:18:19.493432 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:19.502916 (+  9484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:19.502964 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:19.512845 (+  9881us) log.cc:844] Serialized 26 byte log entry
0709 13:18:19.826041 (+313196us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:19.826083 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.826150 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.002933 (+176783us) raft_consensus.cc:1685] finished
0709 13:18:20.002956 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.003103 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8836,"raft.queue_time_us":90118,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":282966,"threads_started":1,"wal-append.queue_time_us":452267}
I20260709 13:18:20.006520 10549 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:20.007056  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.008119 10866 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:20.008546 10866 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:20.008996  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:20.009608  7233 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:20.010095  6667 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:20.011060  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.012143  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:20.012408 10549 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:20.012917  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:20.013459  6667 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:20.013603  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:20.009506 10866 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.018402  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2332) took 1795 ms. Trace:
I20260709 13:18:20.018795  6588 rpcz_store.cc:276] 0709 13:18:18.222862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.606759 (+383897us) service_pool.cc:224] Handling call
0709 13:18:18.607096 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.774757 (+167661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.774813 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.847485 (+ 72672us) log.cc:844] Serialized 26 byte log entry
0709 13:18:19.353743 (+506258us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:19.353791 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.353864 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.018211 (+664347us) raft_consensus.cc:1685] finished
0709 13:18:20.018243 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.018356 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38567,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"thread_start_us":559751,"threads_started":2,"wal-append.queue_time_us":479520}
I20260709 13:18:19.982535 10907 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:19.984956 10906 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:20.017251 10885 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.003s
I20260709 13:18:20.034293  7233 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:19.959520 10910 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.059417  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 952) took 1389 ms. Trace:
I20260709 13:18:20.059857  6597 rpcz_store.cc:276] 0709 13:18:18.670138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.804153 (+134015us) service_pool.cc:224] Handling call
0709 13:18:18.804563 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:19.403829 (+599266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:19.403873 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:19.409597 (+  5724us) log.cc:844] Serialized 26 byte log entry
0709 13:18:19.543329 (+133732us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:19.543375 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.543465 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.059127 (+515662us) raft_consensus.cc:1685] finished
0709 13:18:20.059171 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.059341 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":360703,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"thread_start_us":547229,"threads_started":2,"wal-append.queue_time_us":232216}
W20260709 13:18:20.023697  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.058785  6562 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:19.931295 10928 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.049090 10549 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.102897  6665 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:20.051693 10906 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.989264 10902 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:20.124519  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65889.7us
I20260709 13:18:20.013631 10911 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:20.055505 10781 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:19.984858 10931 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.172s
I20260709 13:18:20.137158 10902 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:19.988062 10787 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.144461 10936 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:20.052789 10907 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.133069 10781 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:20.148510 10781 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:20.124296 10906 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:20.148607 10901 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:20.130987 10911 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.057061 10885 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.168200  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 802) took 1969 ms. Trace:
I20260709 13:18:20.132067 10926 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:20.172375  6662 rpcz_store.cc:276] 0709 13:18:18.199245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.656854 (+457609us) service_pool.cc:224] Handling call
0709 13:18:18.657147 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.784672 (+127525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.167877 (+1383205us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54001e0ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:20.168128 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255277312}
I20260709 13:18:20.175392  7233 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:20.173275  6667 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:20.177315  6662 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:18:20.130163 10549 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.181380  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2048) took 1290 ms. Trace:
I20260709 13:18:20.181756  6662 rpcz_store.cc:276] 0709 13:18:18.890700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.176754 (+1286054us) service_pool.cc:224] Handling call
0709 13:18:20.177044 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.179544 (+  2500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.181326 (+  1782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:20.182145  6667 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:20.183095  6662 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:20.184798 10931 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:20.185667 10931 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:20.186312 10931 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.187312 10931 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.172708 10901 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.188568 10931 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:20.189168 10901 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.185690 10787 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.190567 10901 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.194093 10885 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.194396 10785 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.180164 10682 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:18:20.195663 10682 consensus_queue.cc:1243] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:20.180092 10926 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.196164 10682 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:20.196521 10682 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.197010 10926 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.197356 10682 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.199712  6594 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:20.167155 10911 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.200400 10902 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.200691 10906 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.202881 10906 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.206291  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:20.198364 10682 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.205932  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:20.184253  6667 pending_rounds.cc:85] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 4
I20260709 13:18:20.209142  7237 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:20.209142  6667 pending_rounds.cc:107] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:18:20.211570  6667 raft_consensus.cc:2889] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:20.205920 10927 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:18:20.212153 10266 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:20.222586 10266 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:20.223757 10266 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.225135 10266 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:20.237171 10910 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:20.244385 10911 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.244823 10928 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:20.245850 10549 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.246565 10911 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.252444 10925 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.254894 10709 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:20.255676 10925 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.260048 10781 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.255681 10709 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.262202 10781 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.305302 10709 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.312513 10844 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:20.313088 10844 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:20.314308 10844 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.317682 10844 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:20.320299 10915 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.501s	user 0.001s	sys 0.000s
W20260709 13:18:20.323151 10939 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:18:20.322965 10746 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.324358 10746 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.331612 10907 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.333952 10873 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.382086  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:20.390817  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.393961  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.399989  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.401398  6597 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:20.413380 10709 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:20.415159 10719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.388s	user 0.011s	sys 0.007s
W20260709 13:18:20.415778 10719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.401s	user 0.011s	sys 0.007s
I20260709 13:18:20.409083  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.429903 10910 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.430819 10910 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.433722 10910 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:20.426060 10830 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:20.437927  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.437623 10830 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.438827  6597 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 6.
I20260709 13:18:20.434909  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.441758 10699 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:20.442711 10699 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.445989 10699 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:20.448956 10840 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:18.875442  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:20.457033 10830 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:20.459648  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 936) took 3513 ms (client timeout 2004 ms). Trace:
W20260709 13:18:20.460232  6589 rpcz_store.cc:269] 0709 13:18:16.946157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.949318 (+  3161us) service_pool.cc:224] Handling call
0709 13:18:20.459057 (+3509739us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54002fbcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:20.459411 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5017699200}
I20260709 13:18:20.464282  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.465391  6589 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:20.465607  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2319) took 2949 ms. Trace:
I20260709 13:18:20.468118  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.466049  6592 rpcz_store.cc:276] 0709 13:18:17.516186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.929187 (+413001us) service_pool.cc:224] Handling call
0709 13:18:18.149709 (+220522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.179127 (+ 29418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.179157 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.184900 (+  5743us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.416437 (+231537us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:18.416496 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.416595 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.068124 (+651529us) raft_consensus.cc:1685] finished
0709 13:18:19.068152 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.465024 (+1396872us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540019dcd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:20.465343 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29472,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":3858,"spinlock_wait_cycles":506258816,"thread_start_us":213134,"threads_started":1,"wal-append.queue_time_us":739055,"wal-append.run_cpu_time_us":2620,"wal-append.run_wall_time_us":1154251}
I20260709 13:18:20.469286  6516 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:20.469367  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2047) took 1582 ms. Trace:
I20260709 13:18:20.469905  6665 rpcz_store.cc:276] 0709 13:18:18.886608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.032346 (+1145738us) service_pool.cc:224] Handling call
0709 13:18:20.056564 (+ 24218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:20.113480 (+ 56916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.468823 (+355343us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b54001cb1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:20.469186 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105515776}
I20260709 13:18:20.479629  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 800) took 2288 ms. Trace:
I20260709 13:18:20.480165  6663 rpcz_store.cc:276] 0709 13:18:18.191207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.599805 (+408598us) service_pool.cc:224] Handling call
0709 13:18:18.600245 (+   440us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:18.733832 (+133587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.733863 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:18.734506 (+   643us) log.cc:844] Serialized 78 byte log entry
0709 13:18:18.964608 (+230102us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:19.024918 (+ 60310us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.924139 (+899221us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b1800375a00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:20.473464 (+549325us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b1800375a00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:18:20.473853 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 13:18:20.478916 (+  5063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.479119 (+   203us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.479197 (+    78us) raft_consensus.cc:1685] finished
0709 13:18:20.479209 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.479448 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1440092,"prepare.queue_time_us":1514561,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":118,"raft.queue_time_us":29516,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":22283,"spinlock_wait_cycles":32050432,"thread_start_us":184222,"threads_started":1,"wal-append.queue_time_us":297344}
I20260709 13:18:20.483256 10839 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:20.484182 10839 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.489027  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2438) took 2298 ms. Trace:
W20260709 13:18:20.489290  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.489620  6670 rpcz_store.cc:276] 0709 13:18:18.190498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.193651 (+  3153us) service_pool.cc:224] Handling call
0709 13:18:18.194011 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.236486 (+ 42475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.236516 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.236924 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.517305 (+280381us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:18.517355 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.488328 (+1970973us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b540011dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:20.488405 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.488473 (+    68us) raft_consensus.cc:1685] finished
0709 13:18:20.488488 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.488834 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":980,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":2449,"spinlock_wait_cycles":96929152,"thread_start_us":258373,"threads_started":1,"wal-append.queue_time_us":584810,"wal-append.run_cpu_time_us":2675,"wal-append.run_wall_time_us":1243131}
I20260709 13:18:20.488914 10844 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.449942 10840 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.493237 10844 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.500782  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1154) took 1479 ms. Trace:
I20260709 13:18:20.501192  7228 rpcz_store.cc:276] 0709 13:18:19.021344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.021552 (+   208us) service_pool.cc:224] Handling call
0709 13:18:19.021823 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.048689 (+ 26866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.500141 (+1451452us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b540027d8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:20.500521 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54455040}
I20260709 13:18:20.505506  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.502282 10281 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.511091  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.509910  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.502607 10844 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:20.514900  6589 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
W20260709 13:18:20.516156  6563 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 2.368s (SENT) [suppressed 34 similar messages]
I20260709 13:18:20.518800 10924 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.519629 10924 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.521036  6670 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:20.515178  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.446565 10699 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.533880 10709 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.544592  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.554193  6663 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:20.545614 10840 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.551169 10699 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.553068  7237 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:20.555874  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.555727 10785 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.556236 10907 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.556692  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:20.561575 10924 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.562455  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.565346  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:20.565546  7235 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:20.566038 10709 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.566359  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:20.567312  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:20.569664 10701 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:20.571143  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.578483  7228 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.578397 10907 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.546159 10719 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:18:20.584766 10719 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:20.585165 10719 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.585449  6592 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:18:20.588729 10856 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:20.589514 10898 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.599910  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:20.601675 10856 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:20.602519 10856 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:20.603061  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.608691  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 804) took 1985 ms. Trace:
I20260709 13:18:20.609176  6666 rpcz_store.cc:276] 0709 13:18:18.636514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.745431 (+108917us) service_pool.cc:224] Handling call
0709 13:18:18.745755 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.071231 (+325476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.608205 (+1536974us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54002bcd54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:20.608497 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730143616}
I20260709 13:18:20.615501 10633 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.616075 10633 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.616745 10907 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.618542 10633 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.621749  6666 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:20.622399  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:20.623263  7234 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:20.626607 10633 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.586462 10719 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.640234 10623 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.640836 10623 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.641050 10908 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.641887 10908 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.642755 10661 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.643648  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2432) took 3011 ms. Trace:
I20260709 13:18:20.643998  6671 rpcz_store.cc:276] 0709 13:18:17.600135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.810040 (+209905us) service_pool.cc:224] Handling call
0709 13:18:17.810412 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.941010 (+130598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.610355 (+2669345us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540017d4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:20.611549 (+  1194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299020032}
I20260709 13:18:20.645118 10908 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:20.645555 10633 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.604127 10856 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.637292 10699 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.655225  7233 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:20.635614 10785 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:20.656353  6487 connection.cc:256] RPC call timeout handler was delayed by -2.22089422199997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:18:20.656831  6487 connection.cc:256] RPC call timeout handler was delayed by -1.9077612260000478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 13:18:20.657444  6487 connection.cc:256] RPC call timeout handler was delayed by -1.3758997239999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
I20260709 13:18:20.657933 10785 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.660540 10440 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.661242 10440 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.662976 10715 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:20.663748 10715 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.665856 10719 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.666270 10715 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.666922 10891 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.669591 10709 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.669723  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:20.669513  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.647334 10698 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.671804  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1133) took 2320 ms. Trace:
I20260709 13:18:20.673040  6591 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:20.645892 10908 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.674818  7228 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:20.686264  6593 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:20.687311  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:20.708276  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.695242  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.688603  6597 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:20.690074  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:20.706789  6593 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.710649  6671 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:20.679363  7231 rpcz_store.cc:276] 0709 13:18:18.314965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.315156 (+   191us) service_pool.cc:224] Handling call
0709 13:18:18.315382 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.554428 (+239046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.612512 (+2058084us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b5400684554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:20.634993 (+ 22481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14641024}
I20260709 13:18:20.691131 10709 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:20.715596  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.721701 10850 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:20.722340  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:20.690290  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 50735.9us
I20260709 13:18:20.725916  7237 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:20.697777 10440 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.682973 10908 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.733366  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:20.714470 10623 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.728044 10709 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.739269  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2451) took 1077 ms. Trace:
I20260709 13:18:20.742611  6596 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 13:18:20.743108 10645 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:20.743816  6597 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:20.689234 10891 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.750988  6665 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:20.688030 10698 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.746176 10709 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.739727 10856 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.760030 10633 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.768421  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2447) took 2098 ms. Trace:
W20260709 13:18:20.769038  6595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 957) took 1873 ms (client timeout 1595 ms). Trace:
I20260709 13:18:20.768460 10856 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.723212 10830 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:18:20.749294 10907 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:20.768536  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 78847.3us
I20260709 13:18:20.760520 10920 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:20.771648 10920 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.772322 10907 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:20.769609  6595 rpcz_store.cc:269] 0709 13:18:18.895428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.004383 (+108955us) service_pool.cc:224] Handling call
0709 13:18:20.768652 (+1764269us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54001b6f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:20.768859 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736899840}
I20260709 13:18:20.777675 10698 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.779960  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:20.781596  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:20.788028  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:20.788427  6593 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.789428 10715 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:20.790573 10891 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.791208 10891 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:20.777684 10907 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:20.794919 10715 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:20.746702  6662 rpcz_store.cc:276] 0709 13:18:19.559499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.183835 (+624336us) service_pool.cc:224] Handling call
0709 13:18:20.184107 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:20.184308 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:20.511070 (+326762us) log.cc:844] Serialized 14 byte log entry
0709 13:18:20.574039 (+ 62969us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:20.576081 (+  2042us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:20.596775 (+ 20694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.614840 (+ 18065us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.636352 (+ 21512us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":291382,"prepare.queue_time_us":293060,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":14535,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":7015,"thread_start_us":57672,"threads_started":2,"wal-append.queue_time_us":39937}
I20260709 13:18:20.804571  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.810341  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.807969  6593 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.807405  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:20.810665  7200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.013s	sys 0.057s
I20260709 13:18:20.816962  6596 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
W20260709 13:18:20.824438  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45211.5us
I20260709 13:18:20.769097  6669 rpcz_store.cc:276] 0709 13:18:18.638503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.411401 (+772898us) service_pool.cc:224] Handling call
0709 13:18:19.411708 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.411946 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:20.554494 (+1142548us) log.cc:844] Serialized 14 byte log entry
0709 13:18:20.648773 (+ 94279us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:20.675840 (+ 27067us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:20.715857 (+ 40017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.723394 (+  7537us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.736950 (+ 13556us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1064430,"prepare.queue_time_us":1065761,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":94,"raft.queue_time_us":4124,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":93381,"threads_started":2,"wal-append.queue_time_us":103319}
W20260709 13:18:20.823550  7200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.013s	sys 0.058s
I20260709 13:18:20.827963  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:20.833361  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 84721.9us
I20260709 13:18:20.855183  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:20.859478 10963 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:18:20.861936  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.815742 10891 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:18:20.866262  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.867265  6662 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:20.853479 10898 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:20.853878 10850 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:20.872478  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.856091 10830 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:20.876416  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.877105 10856 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.877420  6669 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:20.877905 10898 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:20.878329  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.879071 10830 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.879534  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.879624 10830 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:20.880721  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.881762  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:20.882236  6519 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:20.877990  6513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2496) took 2167 ms (client timeout 1897 ms). Trace:
W20260709 13:18:20.876521  7236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1158) took 1847 ms (client timeout 1897 ms). Trace:
I20260709 13:18:20.877290 10715 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:20.874469  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.876555  6515 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:20.884012  6596 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:20.885677  6588 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:20.884797  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.865413 10661 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:20.883478  6513 rpcz_store.cc:269] 0709 13:18:18.710449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.710580 (+   131us) service_pool.cc:224] Handling call
0709 13:18:20.854068 (+2143488us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003f75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:20.877752 (+ 23684us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2810394368}
I20260709 13:18:20.889860  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.890208  6521 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:20.891321  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:20.891981  6514 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:20.893420  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:20.895071  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.895706  6520 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:20.896477  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.897722  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.898211  6520 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:20.899318  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:20.900738 10690 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:20.901247  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.902233  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:20.902316  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:20.903237  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.903544  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.903861  6516 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:20.905153 10898 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.899459  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:20.909149  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.909639  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.910193  6517 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:18:20.911382  6407 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 921c423406da4f289884afca51ca198f (127.5.72.129:34121) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 921c423406da4f289884afca51ca198f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:20.912029  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.883554 10850 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.912889 10850 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:20.912930  6664 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:20.914234 10866 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:20.899995 10962 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:18:20.914921  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:20.916760  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:20.915627  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:20.915812  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:20.883574  7236 rpcz_store.cc:269] 0709 13:18:19.022319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.023659 (+  1340us) service_pool.cc:224] Handling call
0709 13:18:20.853996 (+1830337us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b5400575dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:20.869852 (+ 15856us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171347712}
I20260709 13:18:20.900161  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.903327  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:20.934852  6666 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.904824  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.936705  6513 raft_consensus.cc:2410] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:18:20.937991  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 980262us
I20260709 13:18:20.940132 10281 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:20.943624  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.944499  6520 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:20.949494  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.950961 10645 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:20.952457  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:20.889150 10661 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.885236 10715 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:20.951973 10623 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:18:20.962177  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:20.956889 10891 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:20.971501 10641 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.972229 10641 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.972808 10623 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:20.974015 10623 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:20.974373 10623 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:20.974653  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 94348.9us
I20260709 13:18:20.975425 10623 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:20.976276  6664 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.972822 10775 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.983093 10775 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.983665  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.985304 10775 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.986292 10623 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.986811 10641 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.954334 10440 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:18:20.990324 10902 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:20.990715  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:20.993007  6664 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:20.993124 10862 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:20.993702 10862 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:20.995900 10862 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:20.996778  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:20.996949  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:20.997879 10891 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.998348 10891 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.998668 10808 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:20.987452 10908 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.001804  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.002956 10808 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:21.003475 10808 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:21.003966 10808 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:21.009195  6519 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:21.010142  6516 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:20.973857  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.011255 10902 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:21.011408  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:21.012746 10440 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.013159 10440 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.013530 10440 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.014397 10902 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:21.024228 10902 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:20.961740 10661 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.996815  6668 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:21.016582  6666 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:20.935325 10709 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.036535  6669 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:21.038110  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.040695  7237 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:21.042192  7231 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:21.043111 10907 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.046096 10907 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.038934  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.051293  6593 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:20.952208 10633 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:18:21.052342  6595 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:21.030287 10902 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:21.058296  7234 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:21.063457  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.064311  7237 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:21.061836  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.065313  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2531) took 1140 ms. Trace:
I20260709 13:18:21.057941 10715 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.066349  7236 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:21.067941 10694 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:18:21.070475  6514 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:20.956969 10920 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.065853  6593 rpcz_store.cc:276] 0709 13:18:19.919059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.942992 (+ 23933us) service_pool.cc:224] Handling call
0709 13:18:19.943263 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:20.685645 (+742382us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b540011b320. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:18:21.051675 (+366030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.059153 (+  7478us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090999168}
I20260709 13:18:21.056701 10841 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:18:21.084856  6521 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:21.072672 10907 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.067054 10856 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:21.109663  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.114229  7236 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:21.117089  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.117864  7233 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.118809  7237 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:21.068542 10808 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.098605 10703 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.106076 10920 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:21.123951 10809 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.129176  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:21.130486 10777 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.131002 10777 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.131309 10715 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.131428 10777 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:21.136159  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 103882us
I20260709 13:18:21.137010  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:21.164914  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:21.174817 10682 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:21.175307 10682 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:21.175698 10682 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:21.167280 10703 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:21.181615  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44718.1us
I20260709 13:18:21.165199 10709 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:21.183393  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.186206  6671 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:21.187923  7233 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:21.170087 10701 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.164568 10856 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.113416 10902 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.215548  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:21.199715 10862 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:21.222265  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:21.251524  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.230885 10862 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:21.221185 10623 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:21.223964 10719 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.260864 10862 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.225138 10709 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:21.225499  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 113972us
I20260709 13:18:21.239420 10839 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.262769 10709 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.261086 10623 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:21.263654 10709 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.231256  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:21.264895  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 61214.4us
I20260709 13:18:21.267810  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.268959  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:21.273412  6669 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:21.182456 10809 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.274600  6667 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:21.247006 10746 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:21.280408  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.281777  6514 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.261516 10719 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.276062 10841 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.306597 10841 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.306737  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2524) took 2410 ms. Trace:
I20260709 13:18:21.307252 10841 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.241174 10703 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:21.231199 10883 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.280966 10775 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.290472 10902 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.261977 10661 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.307268  6594 rpcz_store.cc:276] 0709 13:18:18.896016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.510154 (+614138us) service_pool.cc:224] Handling call
0709 13:18:19.510580 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.200113 (+689533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.306146 (+1106033us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54002f6cd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:21.306502 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1584247552}
I20260709 13:18:21.306770 10719 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.252828 10701 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.290735 10839 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:18:21.285651 10777 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.333048  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.318082 10746 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.346133 10912 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
W20260709 13:18:21.347872 10976 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:18:21.348524 10746 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.321341 10775 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.326808 10808 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.354810 10908 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.356462 10701 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.356899 10682 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.359930 10934 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:21.340526 10891 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.358330 10858 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:21.362143 10891 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.362954 10281 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:21.363142 10682 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.363462 10281 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:21.364545 10281 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.365876 10281 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.361960 10690 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:21.367408 10440 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.369191 10777 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.369867 10777 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:21.371116 10440 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.370903 10808 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:21.327356 10703 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:21.389117  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.391983 10967 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.560s	user 0.005s	sys 0.000s
I20260709 13:18:21.394315  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.394843  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.398366  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.399381 10809 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:21.399998  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.401741 10661 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:21.401840  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.402907 10856 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.403983  6513 raft_consensus.cc:2410] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:18:21.405004  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.406688  6519 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:21.407882  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.408465  6515 raft_consensus.cc:2376] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8. Re-sending same reply.
I20260709 13:18:21.409405  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.410846  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.412984  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.414361  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.414983  6522 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:21.416497 10830 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.002s
I20260709 13:18:21.416889 10830 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:21.419893 10850 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 13:18:21.420409 10850 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:21.421415  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.421777  6662 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 7: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:21.421952  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.428299  6522 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:21.431134  6593 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:21.374874 10719 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.433025  6591 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.439898  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.381955 10703 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.385456 10907 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.445902  6597 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:21.385320 10682 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.443928  6514 raft_consensus.cc:2410] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:18:21.445781  6589 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:18:21.446183  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:21.445775  6596 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.447288  6520 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:21.447536  6590 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:18:21.460948  6589 pending_rounds.cc:85] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 3
I20260709 13:18:21.443794  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.448220  6518 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:21.452219 10880 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.460665  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:21.463004  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:21.461242  6590 pending_rounds.cc:85] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 8
I20260709 13:18:21.468760  6595 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:21.461427  6589 pending_rounds.cc:107] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:18:21.470748  6589 raft_consensus.cc:2889] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:21.470918  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:21.471130  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:21.471665  6592 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.467492  6590 pending_rounds.cc:107] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:18:21.472605  6590 raft_consensus.cc:2889] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:21.475912  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:21.477417  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.478592  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.479779  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:21.457871 10703 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.495234  6593 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:21.496443  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 227564us
I20260709 13:18:21.484462  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.498548  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:21.459607 10850 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:21.501174  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:21.471330 10898 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:21.508742  7234 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.511209  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.508373  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.512097  6522 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:21.514595  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 250839us
I20260709 13:18:21.517838  6597 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:18:21.498108 10645 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.498101 10830 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:21.457751 10623 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.522898  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.523708  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.525044  6669 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:21.525547 10907 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:21.525954 10907 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.525861 10440 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:21.526443 10907 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.526988  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.528117  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.509158 10694 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.532696  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.498126 10682 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.515980 10856 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.541013  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.495060 10880 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:21.544811  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 183593us
I20260709 13:18:21.520239 10645 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.546272  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.549039  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.549273  6669 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:21.549587  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.549834  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.550794  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.552372  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:21.553233  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 90678.4us
I20260709 13:18:21.506138 10661 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:18:21.514245 10698 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:18:21.557643  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.555099  7237 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.553432  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.534266 10440 consensus_queue.cc:1243] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:21.532531 10898 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:21.564703  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 55562.2us
I20260709 13:18:21.549924 10645 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.567025  6667 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:21.569202  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.569408  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.569819  6517 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:21.571527 10912 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:21.572119 10912 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.574203 10912 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.575485  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.576699 10989 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:21.578341 10989 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:21.578879 10989 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:21.580626 10719 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.582758  6592 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
W20260709 13:18:21.582999 10986 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:18:21.583666 10922 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.540355 10920 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:21.595796  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53073.9us
I20260709 13:18:21.587489  7231 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:21.596942  7235 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:21.601094  6522 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:21.578408 10281 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.609483  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 55660.5us
I20260709 13:18:21.552495 10858 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:21.611318  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.612581  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.613111  7235 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.619426  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.558698 10856 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.623735 10281 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.625680 10856 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.626091 10281 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.626745  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.626909 10988 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.627470 10988 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.628633  7231 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:21.629539 10988 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.629550 10862 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.630662  7235 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:21.626765  6590 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 13:18:21.637110  7234 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:21.605183 10690 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:21.641738  6596 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:21.647011  6662 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.649570  7228 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:21.650465  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.558817 10694 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.613700 10883 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.580125 10989 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.611377 10641 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
W20260709 13:18:21.675225  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 99792.6us
I20260709 13:18:21.623764 10504 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.679738  6668 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:21.630498 10862 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.657126  6415 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 9, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 9 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:21.639514 10661 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.688294  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:21.662424 10690 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.706377  7237 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:21.695698  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.687615 10281 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:21.697336  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.698927  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.702481 10862 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.702934  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.702493 10694 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.706807 10898 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:21.708456 10987 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:18:21.709205 10661 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:21.687252 10641 consensus_queue.cc:1243] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:18:21.711542 10880 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.683871 10709 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.761266 10775 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.762072 10709 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.713698 10858 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.714772  7229 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:21.715308 10682 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.717944 10841 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.718684 10993 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:21.731457  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.725003 10504 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.728219 10281 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.734167  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.735770 10990 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:21.738345  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:21.769074  7233 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:18:21.739377  6669 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:18:21.738436 10698 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:18:21.739822  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.771755  6519 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:21.740986  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.741488  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:21.751442  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 105784us
I20260709 13:18:21.751804  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.751996 10694 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:21.725216 10690 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:21.755019  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:21.755976 10623 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.757953  7231 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:21.758646 10907 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.759158  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.764882  6667 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.766420  7228 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
W20260709 13:18:21.770479  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 131569us
I20260709 13:18:21.683620 10883 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.774981  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:21.787889  6591 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.781055  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.781224  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.782325  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.789515  7232 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:21.784823  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:21.787266  7236 raft_consensus.cc:2368] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:21.791765  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:21.795035  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:21.792356  6596 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:21.798401  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.795812  6597 raft_consensus.cc:2376] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10. Re-sending same reply.
I20260709 13:18:21.771016 10841 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.804322  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:21.777554 10641 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:21.777271 10682 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.770501 10990 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:21.807550  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.808650  6589 raft_consensus.cc:2368] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 13.
I20260709 13:18:21.808449  6595 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:21.809900  6592 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:18:21.808264 10775 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:18:21.812244 10281 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:21.812584 10661 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.813094 10661 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.814220  7235 raft_consensus.cc:2376] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8. Re-sending same reply.
I20260709 13:18:21.814821 10907 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.815145 10661 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.815429 10907 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:21.816515  6596 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.817732  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.819425  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.820257  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.820418  6562 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.820843  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:21.821388  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.822227  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.822604  6636 leader_election.cc:400] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9.
I20260709 13:18:21.823640 10999 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:21.824608 10641 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:21.825289  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:21.845511  6591 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:21.845825  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.787489 10993 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.791684 10633 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:18:21.826962 10694 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:21.827675 10990 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.827029 10862 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.808483  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:21.852857  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:21.815090  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.831485  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.838327  7237 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:21.825778 10682 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.854413  7232 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:21.847450 10709 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.863111  6671 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:21.866228  6636 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:21.848528 10999 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:21.867261  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 65948.3us
I20260709 13:18:21.858036 10866 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:21.849597 10281 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:21.853855 10661 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:21.848834 10993 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.871994  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:21.868073 10880 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.889202  7231 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:21.862524 10934 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.015s
I20260709 13:18:21.876963  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:21.875531  7235 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:21.873911 10866 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:21.896621  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.899085  6522 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:21.932548  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.899173  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.932576  6522 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:21.900095 10809 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.940723  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.941388  6518 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
W20260709 13:18:21.943529  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.937732  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:21.901439 10777 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:21.904971  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.907377  6516 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.908159  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.908871  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:21.909767  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.910455 10988 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.913990 10912 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:21.919886 10682 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.920339 10995 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.920938 11001 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:18:21.887820 10862 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:21.922909 10694 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.923277 10880 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.923940 10978 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 13:18:21.888278 10641 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.964197  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.965998 10862 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:21.967195 10862 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.960696 10898 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.968607 10898 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.968922  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:21.969645  6595 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:21.971105 11000 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.112s	user 0.003s	sys 0.000s
I20260709 13:18:21.955351 10777 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:21.972373 10777 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.973590 10694 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:21.967231 10880 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:21.976161  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.976882  7232 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:21.948043  6597 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:21.874470 10281 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.940614  6522 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:21.940263 10934 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.002s	sys 0.016s
I20260709 13:18:21.952227 10887 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:18:21.954396  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 88231.1us
I20260709 13:18:21.954998  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:21.958413  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.988943  7235 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:21.963033  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:21.963793 10633 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:21.965215  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.988647  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.983174  7234 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:21.948532 10866 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:21.993391 10809 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:21.993810 10809 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:21.994246 10809 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:21.994650 10645 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:21.982332 10775 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:21.993413  6520 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:21.996833 10645 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:21.997505  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:21.997848  6521 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:21.988152 10887 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:21.999145 10887 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:22.000443 10887 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.960572 10990 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.985208 10281 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:22.004681  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49764.4us
I20260709 13:18:21.960882 10912 consensus_queue.cc:1243] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:22.005017  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.005468 10709 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:22.005797  7231 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:22.005867 10709 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.006444 10709 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:22.006786  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.007031 10633 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:22.006659 10999 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.007387 10633 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:21.985422  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.007889  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.008556  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.008311 10633 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.009135  7230 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:22.009920 10633 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:22.010035  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.991663  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:21.992554  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:21.993898  6522 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:21.891920 10661 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:22.028451  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.012521  7235 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:21.965400 10995 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.027714  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:22.032128  7229 raft_consensus.cc:2410] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:18:21.990108  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:22.028321 10841 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:22.034502  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.037022  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:22.037771  7234 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.983410 10898 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.042979  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:22.043200  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:22.010871 10883 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:22.047726  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.050684  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:22.052850  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 89883.6us
I20260709 13:18:22.049170  7231 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:22.056234 10907 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:22.056638 10907 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:22.057932  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.057520 10907 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:22.059420 10998 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.240s	user 0.000s	sys 0.004s
I20260709 13:18:22.061897  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:22.062813  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:22.077867  6593 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:22.069845 10504 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:22.087497  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:22.069639 10709 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.071412  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.071537  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.073210  6662 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.983174 10988 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.085328  6515 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:18:22.034248 10841 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.036664 10850 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:22.049454 10912 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:22.051680 10934 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:22.094787  6522 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:18:22.126682  6522 pending_rounds.cc:85] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 9
I20260709 13:18:22.116137  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:22.115900  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.127961  7229 raft_consensus.cc:2410] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:22.122273  6513 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:22.127178  6522 pending_rounds.cc:107] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:18:22.095176  6667 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:22.107654 10850 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:22.115855 10841 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.133344 10850 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:22.095516  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:22.083635 10775 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:22.136453 10775 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.122933  7234 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:22.132085  6522 raft_consensus.cc:2889] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:22.164193  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:22.176203  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 70663.3us
I20260709 13:18:22.062857 10990 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.138118 10898 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:22.124501 10934 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:22.177250 10979 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:18:22.162881 10912 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:22.190227  7230 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:18:22.189401  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81364.7us
I20260709 13:18:22.192445  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:22.175380 10856 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.198582  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 210221us
I20260709 13:18:22.215509 10866 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:22.217407  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.219404  7235 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:22.201049  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:22.260466  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.188124 10934 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:22.276669  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:22.277829 10996 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:18:22.189615 10907 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:22.213110  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:22.331650  6671 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:22.332638  6667 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:22.335883  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:22.336704  6662 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.336825  6514 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:22.338711  6522 pending_rounds.cc:85] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 5
I20260709 13:18:22.339133  6522 pending_rounds.cc:107] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:18:22.339430  6522 raft_consensus.cc:2889] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:22.340176  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.340477  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.342932  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:22.341154  6663 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:22.345569  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.349623  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:22.351629  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.353675  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.213630 10898 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:22.359833  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:22.360001 10880 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.360785  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 167975us
I20260709 13:18:22.360664 11015 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:18:22.361326 10440 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:22.361411 10719 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:18:22.355001 10623 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.002s
I20260709 13:18:22.186532 10990 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:22.356129  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:22.341442 10746 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:22.377641  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 185263us
I20260709 13:18:22.375531  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:22.351127 10866 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:22.362501  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.386588  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:22.398120  6667 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:22.378572 10682 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:22.187503 10661 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.342288 10856 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.405256 10944 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.380837 10880 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.410338 10709 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.420504 10777 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.401108 10775 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.421351  6517 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:18:22.424616  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.425403  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.431699  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75638.4us
W20260709 13:18:22.420969 11008 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.432s	user 0.001s	sys 0.000s
I20260709 13:18:22.421911 10682 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.445084  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:22.447789 10682 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.441218 10912 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:22.475081 10281 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.477268 10281 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:22.446908 10850 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.479812 10850 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.480396 10850 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:18:22.482417 11005 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.161s	user 0.002s	sys 0.002s
W20260709 13:18:22.490024 11009 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:22.491701 10645 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.492285 10645 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.495643 10645 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:22.498728  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.502556  7235 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:22.503178 11010 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:18:22.507124  7237 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:22.512476  7237 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:18:22.514671  7237 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:18:22.520033  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54445.4us
I20260709 13:18:22.523939  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:22.524683  6522 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:22.527531  6662 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
W20260709 13:18:22.521587 11012 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:18:22.558413  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 117756us
I20260709 13:18:22.468493 10775 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:22.570344  6522 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:22.438143 10995 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:22.554440 10281 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.529963 10988 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.554738 10902 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:22.590749  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 220710us
I20260709 13:18:22.518708 10709 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:18:22.526515 10719 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:22.531337 10998 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:18:22.636823  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46807.1us
W20260709 13:18:22.584268 10633 consensus_meta.cc:287] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.003s
I20260709 13:18:22.642482 10883 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:22.643532 10887 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.643s	user 0.003s	sys 0.014s
I20260709 13:18:22.645103 10880 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:22.645617 10809 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.651s	user 0.015s	sys 0.002s
I20260709 13:18:22.652210  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2358) took 1228 ms. Trace:
W20260709 13:18:22.655263 11011 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:18:22.654381 10703 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.145s	user 0.012s	sys 0.004s
I20260709 13:18:22.556411 10898 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.660898 10682 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.663990  7233 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:22.664681 10440 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:22.640161 10902 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:22.679390  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:22.680284  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:22.652487 10990 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:22.685262  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.559687 11015 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:22.682387 10988 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:22.719609  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:22.587208 10775 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:22.737496  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 350994us
I20260709 13:18:22.689538 10830 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:18:22.739521 10830 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:22.740558 10830 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:22.740873 10830 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:22.741714 10830 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.742952 10830 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:22.713377 10703 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.749151 10703 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.749887 10703 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:22.750204 10703 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:22.755360 10703 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:22.760417  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.748980 10694 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.767625  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:22.769709  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.685312  6594 rpcz_store.cc:276] 0709 13:18:21.423979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.432005 (+  8026us) service_pool.cc:224] Handling call
0709 13:18:21.448078 (+ 16073us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:18:21.566739 (+118661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:21.566768 (+    29us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:18:21.569832 (+  3064us) log.cc:844] Serialized 104 byte log entry
0709 13:18:21.670329 (+100497us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:21.684384 (+ 14055us) log.cc:844] Serialized 14 byte log entry
0709 13:18:21.696376 (+ 11992us) log.cc:844] Serialized 14 byte log entry
0709 13:18:21.726864 (+ 30488us) log.cc:844] Serialized 14 byte log entry
0709 13:18:21.750654 (+ 23790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.930564 (+179910us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540019d7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:21.930709 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:22.058693 (+127984us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:22.651661 (+592968us) raft_consensus.cc:1685] finished
0709 13:18:22.651704 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:22.652060 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101845,"prepare.queue_time_us":17299,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":2099,"raft.queue_time_us":119468,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":21000,"spinlock_wait_cycles":316400384,"thread_start_us":99921,"threads_started":1,"wal-append.queue_time_us":248794}
I20260709 13:18:22.762902 10703 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.783049  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:22.785249  7228 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:22.787274  6662 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:22.790678  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.791438  7231 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:22.718219 10902 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:22.686126 10883 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.820109  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:22.820179  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:22.795980  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:22.806770  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.825620  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:22.826073  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 229287us
I20260709 13:18:22.690371 10809 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.735388 10990 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:22.833554  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2199) took 1091 ms. Trace:
I20260709 13:18:22.834088 11022 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.795810 10988 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.835530 10883 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:22.836226 10682 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.856240  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1173) took 1124 ms. Trace:
I20260709 13:18:22.859144  7228 rpcz_store.cc:276] 0709 13:18:21.732063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.051436 (+319373us) service_pool.cc:224] Handling call
0709 13:18:22.768228 (+716792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.785747 (+ 17519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.856061 (+ 70314us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54007a3d50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:22.856187 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526581632}
W20260709 13:18:22.865798  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.837095  6415 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 7, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 7 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:22.841795  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:22.844180  6517 rpcz_store.cc:276] 0709 13:18:21.739770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.740275 (+   505us) service_pool.cc:224] Handling call
0709 13:18:21.740582 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.438551 (+697969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.826266 (+387715us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b5400044f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:22.830766 (+  4500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1517895936}
I20260709 13:18:22.737232 10281 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:22.783910 10690 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.978s	user 0.007s	sys 0.004s
I20260709 13:18:22.857620  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:22.857306  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:22.822908 10994 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.784899 10912 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:22.854713 10993 consensus_meta.cc:287] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.900s	user 0.007s	sys 0.009s
I20260709 13:18:22.877601  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:22.878656  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:22.829804 10440 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.737259 11015 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.876714  6667 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:22.881788  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:22.893415  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.905182  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:22.905881  7231 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:22.844125 10990 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.818189 10694 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:18:22.795660 11023 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:18:22.897156  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:22.898591  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:22.890583  6588 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:22.855288 10856 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:22.786525 10850 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:18:22.911931  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:22.912608  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 226232us
I20260709 13:18:22.911229  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:22.920857  6522 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:22.915153  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.912868  6590 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:22.877727  6597 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:22.855087 10944 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:18:22.917179  6415 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.938093  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:22.945585  6515 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:22.946159  6515 pending_rounds.cc:85] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 8
I20260709 13:18:22.946439  6515 pending_rounds.cc:107] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:18:22.946892  6515 raft_consensus.cc:2889] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:22.922664 10841 consensus_meta.cc:287] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.789s	user 0.012s	sys 0.000s
I20260709 13:18:22.948477  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:22.910938 10682 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:22.912233 10990 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.951154  7200 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:22.954921  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 61715.2us
I20260709 13:18:22.903363 10979 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:22.910704 10912 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:22.952652  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:22.895442 10999 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:18:22.962997  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:22.939926 10709 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:22.969923  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:22.872790 11022 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:22.906953 10440 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:22.888514 10690 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.912148 11032 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:22.940907 10920 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:22.937279 11029 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:22.972594  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 75551.1us
W20260709 13:18:22.974598  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 51138.2us
I20260709 13:18:22.957857 10682 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.979902  6517 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:22.980232  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:22.984131  7228 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.902763 10993 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.985163  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1156) took 1063 ms. Trace:
W20260709 13:18:23.018662  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 299148us
I20260709 13:18:22.950727 11021 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:18:22.980228 10777 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
I20260709 13:18:23.034528  7236 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:22.978762 10709 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:22.907696 10994 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.031651  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:23.037698  7233 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:22.970569 10944 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.981420 11029 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.961608 10841 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:22.982781 10920 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.986596 10682 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.041858  7233 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:22.968353 10880 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:18:23.030880  7232 rpcz_store.cc:276] 0709 13:18:21.922129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.430961 (+508832us) service_pool.cc:224] Handling call
0709 13:18:22.431329 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.442039 (+ 10710us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:22.580770 (+138731us) log.cc:844] Serialized 14 byte log entry
0709 13:18:22.980042 (+399272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:22.980094 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:22.980138 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.980204 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:22.985053 (+  4849us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33574,"prepare.queue_time_us":243683,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":38813,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":34513,"thread_start_us":338428,"threads_started":2}
W20260709 13:18:23.042725 10907 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.004s
I20260709 13:18:23.054505  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1277) took 1202 ms. Trace:
W20260709 13:18:23.081315  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50237.9us
W20260709 13:18:23.084816  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.041436 11032 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:23.086436  6637 leader_election.cc:400] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 13.
I20260709 13:18:23.092506  6637 leader_election.cc:403] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:18:23.078047 10850 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:23.112025 10850 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:23.112900 10850 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.114034 10850 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.115921 10997 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:23.116412 10997 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:23.116652 11043 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:18:23.117420 11038 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:18:23.117261 10997 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.118324 10997 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:22.974165 10979 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:23.106073 11025 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:18:23.107592  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.108276 11046 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 13.
I20260709 13:18:23.108879 10988 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.109318 10709 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.111418 10440 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:22.981644 10661 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:18:23.079846 10777 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:23.077665 10920 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.053606  7237 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:23.083909  7230 rpcz_store.cc:276] 0709 13:18:21.851758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.224064 (+372306us) service_pool.cc:224] Handling call
0709 13:18:22.358013 (+133949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.379404 (+ 21391us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:22.887776 (+508372us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b180041e0c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf
0709 13:18:22.887963 (+   187us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.045284 (+157321us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.045498 (+   214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.045558 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:23.053363 (+  7805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.053642 (+   279us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.054275 (+   633us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":263497,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2394,"raft.queue_time_us":157,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":31266,"spinlock_wait_cycles":2442496,"thread_start_us":476185,"threads_started":2}
I20260709 13:18:23.052744 11029 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.044076 10504 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:18:23.115926 11036 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:18:23.150985  7228 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
W20260709 13:18:23.144429  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 162637us
I20260709 13:18:23.093636 10809 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.091041 10775 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:23.155426  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47905.2us
W20260709 13:18:23.189145  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 227274us
W20260709 13:18:23.166569  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 221014us
I20260709 13:18:23.172518  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1175) took 1428 ms. Trace:
W20260709 13:18:23.181007  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64450.9us
I20260709 13:18:23.190438  7236 rpcz_store.cc:276] 0709 13:18:21.743570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.801309 (+ 57739us) service_pool.cc:224] Handling call
0709 13:18:23.172248 (+1370939us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400646c54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:23.172440 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2707458304}
I20260709 13:18:23.176997 10988 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.200430  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:23.203118  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 218143us
I20260709 13:18:23.202286  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:23.144047 10944 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:18:23.155424 11042 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:23.205067  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.212185  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.203764  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:23.157109 10995 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:18:23.104650 10912 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.219997  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.229007  6597 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:23.232465  7231 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:23.235008  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.235805  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:23.238190  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.239539  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.181140 10777 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.243211  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.247308 10988 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:23.247890 10690 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.250016  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:23.250641 11031 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.342s	user 0.000s	sys 0.002s
I20260709 13:18:23.250921  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1273) took 1400 ms. Trace:
I20260709 13:18:23.251085 10994 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:23.207433 10775 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:18:23.251823  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:23.277280  7234 rpcz_store.cc:276] 0709 13:18:21.850664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.165811 (+315147us) service_pool.cc:224] Handling call
0709 13:18:22.187067 (+ 21256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:22.211346 (+ 24279us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:22.211375 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:22.239398 (+ 28023us) log.cc:844] Serialized 26 byte log entry
0709 13:18:22.445502 (+206104us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:22.445556 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.445642 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.250704 (+805062us) raft_consensus.cc:1685] finished
0709 13:18:23.250723 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.250861 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3953,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":23752,"thread_start_us":57569,"threads_started":1,"wal-append.queue_time_us":330745}
I20260709 13:18:23.252281 10944 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:23.254756 11051 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:18:23.258932 10746 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.874s	user 0.006s	sys 0.006s
W20260709 13:18:23.247311 11039 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:18:23.259855 11035 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:18:23.210516 11046 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:23.264034  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.239854  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:23.217793 10809 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:23.227994 10623 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:23.275458  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1159) took 1291 ms. Trace:
I20260709 13:18:23.293041  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.293758  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.297452  6515 raft_consensus.cc:2368] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11.
I20260709 13:18:23.292273  7235 rpcz_store.cc:276] 0709 13:18:21.984256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.508184 (+523928us) service_pool.cc:224] Handling call
0709 13:18:22.508465 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:22.510193 (+  1728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.510220 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:22.510864 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 13:18:22.770160 (+259296us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:22.770219 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.770317 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.275240 (+504923us) raft_consensus.cc:1685] finished
0709 13:18:23.275264 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.275400 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1150,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":258782,"threads_started":1,"wal-append.queue_time_us":440129}
I20260709 13:18:23.283717 11040 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.298609  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.299023  6513 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:18:23.298908 11040 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.300987 11040 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:23.301884  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.302744 11058 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.302922  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.303273 11058 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.303764  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.304294  6518 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.305388 11058 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.305923 11058 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:23.306252 11058 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:23.306725 11045 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:23.275717  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.295332 10988 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:23.307801 10775 consensus_queue.cc:1243] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:23.307690 10988 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.273842  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.217299 11015 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.210035 10880 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:18:23.284051 11022 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.263365 10907 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:23.310158 10907 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.291436 11042 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:18:23.310559 10907 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.310695 11042 consensus_queue.cc:1243] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:18:23.311960 11052 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:18:23.289294 11046 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:23.313076 11046 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:23.315601  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.316742  6521 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:23.318049  6514 raft_consensus.cc:2410] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:23.318674 10633 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:18:23.288973 10719 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:18:23.329118 10841 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.296654 10623 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.330080 10623 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:23.274502 11016 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:18:23.213920 11032 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:23.307610  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.309762 11022 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:23.340278  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 66593.1us
I20260709 13:18:23.341787  6592 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:23.342700  6518 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:23.285665  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45880.5us
I20260709 13:18:23.307540  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.283455 10746 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:23.344585  6519 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:23.347400  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.348986 11060 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:18:23.350083 11060 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:23.350414 11060 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:23.348562  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.352035  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:23.353168  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.355420  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.346844 10775 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.306586 11058 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.357146  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.360725  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.357311  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.358685  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.359459  6593 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:23.360057 10719 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.361629 10775 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:23.355966 10944 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:23.362207 10841 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.346109 10633 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
W20260709 13:18:23.363659 11019 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:18:23.356280 11023 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:18:23.364989 10979 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.352651 10999 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.346944 11022 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:23.372162 10934 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.022s	user 0.013s	sys 0.000s
I20260709 13:18:23.375909  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.353345 10995 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.381398  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.378353 11045 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.356801 11042 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.359748 11015 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.385690 10994 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:23.362929 10777 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.391039 11022 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.395459  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.398790 10995 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.398667 10934 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.384678 10944 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:23.402331 10866 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:23.403262 11051 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:23.384963 11050 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:23.408331 11028 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.439s	user 0.004s	sys 0.000s
I20260709 13:18:23.408819 10809 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:23.421123 10907 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.468127 10907 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.438859  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:23.442309  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 60992.8us
I20260709 13:18:23.470257  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.443706  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.445909  7235 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:23.455106  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:23.477906  7228 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:23.455956  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 60566.3us
I20260709 13:18:23.457587  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:23.478922  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.380293 11029 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.442229 10988 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:23.464447 11057 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.465251  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:23.483026  7232 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10.
I20260709 13:18:23.477722  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:23.455860 10944 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.469432 10995 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:23.459653 11045 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.456429 10866 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:23.458463 11051 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:23.493721  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.496241  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.493286  7230 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:23.495653  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.501397  6597 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:23.482858 10994 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.507145  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.505435  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:23.517771  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.510145  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.510371  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:23.524865  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.512836  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.525691  6666 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:23.526268  6670 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.494880  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.485141 11022 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:23.443701 11060 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.493811 10999 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:23.546267  6415 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 10, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 10 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.511545  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.528604  6662 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:23.548605  6662 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:23.444666 11042 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.517287 10809 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:23.535321  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.362900  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:23.460832 10990 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.537446  6591 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:23.494100 11015 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:18:23.555725  6662 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:23.561337 11072 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:18:23.562676  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1279) took 1710 ms. Trace:
I20260709 13:18:23.563087  7229 rpcz_store.cc:276] 0709 13:18:21.852316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.281701 (+429385us) service_pool.cc:224] Handling call
0709 13:18:22.367878 (+ 86177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:22.395366 (+ 27488us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:22.395401 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:22.423302 (+ 27901us) log.cc:844] Serialized 26 byte log entry
0709 13:18:22.730692 (+307390us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:22.730758 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.730865 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.407471 (+676606us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:23.562333 (+154862us) raft_consensus.cc:1685] finished
0709 13:18:23.562355 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.562593 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1490,"raft.queue_time_us":5830,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":12109,"spinlock_wait_cycles":8048000,"thread_start_us":283909,"threads_started":1,"wal-append.queue_time_us":454790}
I20260709 13:18:23.539402 11023 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.534979 10995 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.571125 11067 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:18:23.566721 11063 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:23.572062 11063 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:23.572463 11063 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.574134  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.575920 11029 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.577261  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.577888  6516 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.545791 11061 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.543283 11058 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.581219 11058 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.581560 10993 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.583101 11042 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.584964  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.586608  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.588733 10978 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.589242 10978 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.592005 10775 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:23.592407 10775 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:23.592774 10775 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:23.596315 10682 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.002s
I20260709 13:18:23.596740 10682 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.597709  6594 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:23.503593  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 64840.3us
I20260709 13:18:23.601994 10988 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.602504 10988 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.607862  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.519874  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.609006  6590 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:23.540308 11022 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:23.527464  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 48558.8us
I20260709 13:18:23.549612  6415 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:23.531291 10703 consensus_meta.cc:287] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.001s
I20260709 13:18:23.551601  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.552021  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.553741  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.554661  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:23.543500 10902 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.001s
I20260709 13:18:23.538126  6666 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:23.584650 10999 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.636600 10934 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.622080 10866 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.638144 10999 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.638305 10866 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.624646 11042 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.639204 11042 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.641469 11060 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.551303 10912 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.641842 11042 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.642220 10912 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.645233 10912 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.640857 11057 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.646761 11051 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.648267 11051 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.640908 10682 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.648919 10902 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.649969  6596 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:23.620401  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:23.522428  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.622903  6671 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:23.623823  6662 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.624606  6589 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:23.631515  6668 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:23.635931  6595 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10.
I20260709 13:18:23.490932 10709 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:23.657928  6516 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:23.641031 11046 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.621466  6667 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:23.656821  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1165) took 1467 ms. Trace:
I20260709 13:18:23.665488  7237 rpcz_store.cc:276] 0709 13:18:22.192194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.517080 (+324886us) service_pool.cc:224] Handling call
0709 13:18:22.517455 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.177237 (+659782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.656593 (+479356us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b540028cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:23.656760 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206022528}
W20260709 13:18:23.671082  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.622745 11022 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.672964  6588 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:23.674643  6595 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:23.677649  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.693038  6589 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:23.688423  6662 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:23.684901  6663 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:23.656428 10934 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.668334 10993 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.622324 10995 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:23.697679  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 77357.7us
I20260709 13:18:23.632349 10623 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.695489  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.707500  6667 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:23.709673  6670 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:23.712847  6636 leader_election.cc:400] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10.
I20260709 13:18:23.714015  6636 leader_election.cc:403] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:23.713331  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:23.716226  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:23.716771  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:23.718596  6563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.021s
I20260709 13:18:23.718804  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.717103  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:23.722110  6563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.001s	sys 0.021s
I20260709 13:18:23.669179 10709 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:23.705802 10661 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:18:23.723734  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.672844 10898 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 13:18:23.722533  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 520274us
I20260709 13:18:23.725600  7236 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:23.738744  6596 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:23.744936  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.745664  6667 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:23.746604  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.746703 11063 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.747339  6667 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:23.748111  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.748759  6667 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:23.749557  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.750079 11063 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.751379  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:23.752735  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.722697  6662 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:23.814571  6662 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:23.815689  6662 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:23.816680  6662 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:23.754753  7237 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:23.755167  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.817620 11029 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.817919  6594 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:23.724370 10866 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:18:23.751157  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.718321 10934 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.756168  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:23.757380 11066 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:23.822314  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:23.759346 10902 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:23.774899  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76968.5us
I20260709 13:18:23.745712 10979 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.786931 10990 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:23.789219 10995 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.720011 10703 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:18:23.726058 10988 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.800813 10978 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.800673  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.849650  6668 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:23.800691 10623 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.805510  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.805140  6415 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:23.718079 10809 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10.
I20260709 13:18:23.816964  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:23.819056  6592 raft_consensus.cc:2410] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:23.820242  7228 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.819340  6662 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:23.757545 11049 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:23.828387  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.830305  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:23.836512  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.837392  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:23.822578 11029 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:23.840744  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.843341 10934 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.844201  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.846979  6590 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:23.846475  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.850323  6665 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
W20260709 13:18:23.850317  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 126646us
I20260709 13:18:23.858639  6667 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:23.856350  7233 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:23.839262 10979 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:23.858109  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.863235  6662 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:23.862984  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.864240  6668 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.859448  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.864341  6662 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:23.859195  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.861055  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.865679  6669 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:23.867820  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:23.869746  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.870451  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.876281  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.885392  6665 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:23.886958  6662 raft_consensus.cc:1217] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[5.4-11.7]   Dedup: 5.4->[7.5-11.7]
I20260709 13:18:23.867597  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:23.881243  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:23.884902  6637 leader_election.cc:400] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10.
I20260709 13:18:23.855438 10993 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:23.891175  6588 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:23.891667  7228 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:23.890403  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:23.892867  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.895810  6663 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:23.889634  6662 pending_rounds.cc:85] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 4
I20260709 13:18:23.862279 10703 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:23.900092  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.901324  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:23.849032 10887 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:18:23.906916  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.896982  6662 pending_rounds.cc:107] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:18:23.868811 10934 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.901650  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:23.862497 10979 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:23.939474  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:23.903652  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.889310 10841 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:23.887207 11076 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:18:23.943853  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.944463  6522 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:23.950850 10999 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:23.942354  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:23.953542  6664 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:23.895537 10623 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:18:23.876163 10809 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.860478 11078 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:23.895579 10633 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:18:23.901561  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:23.960834  6671 raft_consensus.cc:2410] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:18:23.909500  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:23.894909 11046 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:23.887374 10978 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.928058  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:23.899410 10880 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:23.936947  6662 raft_consensus.cc:2889] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:23.979036  6662 pending_rounds.cc:107] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:18:23.979359  6662 raft_consensus.cc:2889] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:23.887482 10995 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:23.941699  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 60520.7us
I20260709 13:18:23.942322  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:23.985057  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:23.942970 11083 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:18:23.997772  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 69829.3us
I20260709 13:18:23.998471  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.904372 10281 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:23.999125  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:23.999081  6522 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.942781  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:23.861752 11053 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.954718  6666 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: [9.4-9.4]
W20260709 13:18:23.957571  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54007us
I20260709 13:18:24.010169  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:23.967701  6588 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:23.900725 10703 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.011718  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:24.014504  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:24.015668  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:23.937466  6519 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:23.889557 10988 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:18:23.984163  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 74731.7us
W20260709 13:18:24.020673  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.022261  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:24.023087 10902 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:23.906369 11015 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:24.023680 10902 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:24.024906  7232 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:23.913414 11060 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:24.026248 11060 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.026820 11060 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:24.000541  6513 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:24.032647  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:24.006378  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:24.006152  6415 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 9, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 9 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:24.011644 10866 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:24.038647  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:24.012702 10709 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:23.895188 11024 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:18:24.006186 11045 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.912801 10898 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:18:23.993024 11066 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:24.055226  7233 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:24.060165  6522 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:24.062176  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46597.2us
I20260709 13:18:24.000186 11049 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.999291 10912 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:24.069000 10777 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:24.069659 10777 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.070091 10777 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:23.955886 10645 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:18:24.018293 10703 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:24.063508  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:24.024932 10902 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.039382 11042 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:24.020018 10993 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:24.021111 10880 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:24.129802  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:24.120373  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56920.3us
I20260709 13:18:24.131376  7200 leader_election.cc:400] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11.
I20260709 13:18:24.131862  7200 leader_election.cc:403] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:24.019587 10883 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:24.037659 10281 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:24.018744 11053 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:23.993041 11022 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.084108 10661 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:24.085381 10912 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:24.006981 10995 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.126048 11042 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:24.114151 11023 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
I20260709 13:18:24.036592 11063 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:18:24.038714 10866 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:24.138302  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:24.052003 10690 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:18:24.129509 11061 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:24.133538 10920 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.944s	user 0.012s	sys 0.001s
W20260709 13:18:24.132826  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 117412us
I20260709 13:18:24.062211 11054 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:18:24.153136  6513 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
W20260709 13:18:24.143784  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 162312us
W20260709 13:18:24.144032  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 144995us
I20260709 13:18:24.054996 11078 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
W20260709 13:18:24.127512 11050 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.000s
I20260709 13:18:24.146225 11042 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:24.151963  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:24.093088 10979 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:24.184918  6515 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:18:24.145804 10944 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.627s	user 0.006s	sys 0.006s
I20260709 13:18:24.145510 10661 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:24.142032 10993 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11.
I20260709 13:18:24.139994 11037 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:18:24.147500 10866 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:24.161132 11063 consensus_queue.cc:1243] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:18:24.196165 10504 raft_consensus.cc:2764] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:24.191900 11015 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:24.220942  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.235219  7231 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:24.193382 11050 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:24.147601 11023 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:24.186594 10990 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:24.205680 10690 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:24.196326 10978 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:24.219110 11084 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:18:24.161284 10623 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:18:24.161019 10920 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.161844 10645 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:18:24.217296 11029 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:24.280884  7235 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:24.222756 10944 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.234743 11015 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:24.331353  6597 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:18:24.249143 10990 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:24.334668  6668 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:24.251226 10690 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:24.338642  6665 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:24.279098 11032 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:24.345650  6666 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:24.346776  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 138604us
I20260709 13:18:24.347807  7234 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:24.303896 10809 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.303627 10645 consensus_queue.cc:1243] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:24.216563 10880 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.350580  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:24.368255  6670 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:24.309357 11061 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:24.373219  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 151310us
I20260709 13:18:24.336550 11053 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:24.255251 11095 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.127s
I20260709 13:18:24.378599  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:24.381683  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 228426us
I20260709 13:18:24.383687  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.250574 11109 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 13:18:24.384289  7231 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:24.390978  7231 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:18:24.391377  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:24.391963  7237 raft_consensus.cc:2376] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 12. Re-sending same reply.
I20260709 13:18:24.392086  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:24.396349  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.397161  7235 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:24.389878  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:24.396773  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.402341  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:24.259436 10281 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.406101  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:24.409714  6665 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:24.410563  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:24.411072  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:24.412084  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:24.408303  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.317046  6669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:24.422780  6669 raft_consensus.cc:606] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:24.423172  6669 raft_consensus.cc:614] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:24.425567  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.426234  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:24.427404  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:24.370012 10809 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:24.510769  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 132249us
I20260709 13:18:24.511247  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.516911  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.396303 11024 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.413682 11053 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:24.436035 11085 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:18:24.517616  7231 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:24.520604  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:24.521190  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 103823us
W20260709 13:18:24.414848 11091 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:18:24.523706  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.524519  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:24.531486  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 105367us
I20260709 13:18:24.541352  7230 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:24.541821  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:24.542096  7230 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:24.409287 11107 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.382205 10777 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.537528  7200 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:24.435544 11076 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:24.541374  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:24.435456 11022 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:24.534547  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.423817 10898 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:24.546495  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:24.418181 10682 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:24.423101 11058 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:24.512916 10887 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:24.549337  7235 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:24.511445 10995 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:24.400245 10709 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 13:18:24.406371 11029 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.126s
I20260709 13:18:24.523350 11037 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:24.554298  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:24.399963 11050 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 13:18:24.559053  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:24.415016 11045 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.157s
I20260709 13:18:24.563076 10944 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:24.564144  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:24.566609  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:24.523022 11105 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.395701 10661 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.531244 11063 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:24.398036 10990 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:24.572475  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.582155  7237 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15.
I20260709 13:18:24.583019  7235 raft_consensus.cc:2376] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 12. Re-sending same reply.
I20260709 13:18:24.601825  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.588554  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:24.523546 11095 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
W20260709 13:18:24.591104  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 163772us
I20260709 13:18:24.593432  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:24.578445 11089 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
W20260709 13:18:24.592650  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 51353.8us
I20260709 13:18:24.566890 10898 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:24.547147 11061 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.601986  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:24.608160  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:24.604720  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:24.512488 11015 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.620191  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:24.592288  7230 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.580626 11046 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.680051  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:24.545212 11024 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.677362  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:24.584954 10809 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:24.681205  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 76692us
I20260709 13:18:24.584748 11076 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:24.677508  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 69727.9us
I20260709 13:18:24.582998 11022 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:24.592511 10995 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:24.691540  6596 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:18:24.698357  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 298622us
I20260709 13:18:24.600754 10887 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:24.600346 11063 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:24.697013  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:24.740720  6514 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:24.618366 10898 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:24.697365  6487 leader_election.cc:336] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.527s (SENT)
W20260709 13:18:24.699705  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 117929us
I20260709 13:18:24.700220  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:24.747766  6591 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:24.699060  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:24.586433 11021 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:18:24.748745 11021 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:18:24.679884 11087 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
W20260709 13:18:24.698102  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 77987.3us
I20260709 13:18:24.704974  6592 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:24.704909  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:24.750932  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:24.751456 10995 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:24.752091 10995 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:24.617933 11096 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:18:24.711236  7230 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:24.754544 10995 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:24.739660  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "627436404a154d80a882f4900dff2948"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:24.683858 11058 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:24.615175 11108 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:24.759652 11108 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:24.760697 11108 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.764179  7230 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:18:24.765139  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:24.767241  7230 pending_rounds.cc:85] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 9
I20260709 13:18:24.767942  7230 pending_rounds.cc:107] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:18:24.768321  7230 raft_consensus.cc:2889] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:24.555256 11107 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.684599 11076 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:24.767213  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:24.692953 11022 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.692101 10920 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:24.743911 11089 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:18:24.600473 11105 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.783398  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:24.795784  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.805341  7230 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:24.806452  6519 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:24.802330 10887 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.808454 11107 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:24.769652 11095 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:24.691668 11046 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:24.810284  6513 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:24.811159  6514 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:24.813050  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:24.815063  6514 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:24.713426  6669 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:24.712968  6596 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:24.697962  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 96211.8us
I20260709 13:18:24.695996 11116 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.129s
I20260709 13:18:24.835131  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:24.831398  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:24.696182 11078 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:24.836310  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:24.842669  7230 raft_consensus.cc:2368] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11.
I20260709 13:18:24.836310  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:24.838044  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:24.774076 11076 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:24.840618  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:24.845119  6514 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:24.845319  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:24.846010  6669 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:24.841652  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 144305us
W20260709 13:18:24.846725  6487 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.540s (SENT)
I20260709 13:18:24.847087  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:24.848141  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:24.777103 11022 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:24.850028  7235 raft_consensus.cc:2376] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 8. Re-sending same reply.
I20260709 13:18:24.851648  6514 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:24.812040 11095 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:24.853701  6669 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:24.882302  6519 raft_consensus.cc:1217] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Deduplicated request from leader. Original: 8.9->[9.10-9.10]   Dedup: 9.10->[]
I20260709 13:18:24.876811  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:24.883502  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:24.897634  6596 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:24.874423  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 170753us
I20260709 13:18:24.796847 11110 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:18:24.882951  6519 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:18:24.855139 10841 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.910s	user 0.011s	sys 0.000s
W20260709 13:18:24.854827 10775 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.262s	user 0.007s	sys 0.007s
I20260709 13:18:24.853093 11078 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:24.918074  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:24.924618  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 77905.9us
W20260709 13:18:24.925530  6487 leader_election.cc:336] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.564s (SENT)
I20260709 13:18:24.926010  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:24.926328  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.927453  7233 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:18:24.919478  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:24.928645  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:24.929380  7233 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:24.918567  6590 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:24.933591  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:24.934334  6596 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
W20260709 13:18:24.980233  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:24.983397  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:24.918412 11097 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:18:25.050458  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 124973us
I20260709 13:18:25.056804  6590 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:24.917295 10841 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:25.054704 11049 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.959s	user 0.008s	sys 0.003s
I20260709 13:18:24.923815 11103 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:24.924168 11058 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:25.095074  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.049681 10995 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:25.052290 11106 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:24.927218 10775 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.110808 11103 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:25.111794 11103 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.112370 10775 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.113135  6663 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:24.923547 10999 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.159s
I20260709 13:18:25.054600 10641 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:24.893976 11063 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.060534 11119 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.102432 11049 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:25.107861 11126 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:25.108590 11118 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:25.109519 11113 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
W20260709 13:18:25.095019 11081 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:18:25.120850  7233 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:25.122884 10641 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:25.107473 11060 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.080s	user 0.012s	sys 0.000s
I20260709 13:18:25.127810  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:25.129001  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:25.132474  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:25.152102  6590 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
W20260709 13:18:25.130123 11100 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.385s	user 0.000s	sys 0.001s
I20260709 13:18:25.136845  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:25.130414 10866 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.891s	user 0.011s	sys 0.000s
I20260709 13:18:25.171854  6521 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:25.173923  7200 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:25.113615 10995 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.151078 10902 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.171572 11108 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:25.134534 11024 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:25.215224  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 78403.6us
I20260709 13:18:25.221211  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:25.081028 11095 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.233186  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1307) took 1030 ms. Trace:
I20260709 13:18:25.234001  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:25.170642 11060 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.233758  7236 rpcz_store.cc:276] 0709 13:18:24.202594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.202861 (+   267us) service_pool.cc:224] Handling call
0709 13:18:24.220263 (+ 17402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.351352 (+131089us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:24.351397 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:24.369043 (+ 17646us) log.cc:844] Serialized 26 byte log entry
0709 13:18:24.518705 (+149662us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:24.518743 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.518822 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.232905 (+714083us) raft_consensus.cc:1685] finished
0709 13:18:25.232934 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.233129 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59406,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":24586,"thread_start_us":262803,"threads_started":2,"wal-append.queue_time_us":360004}
W20260709 13:18:25.240815  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54523.8us
I20260709 13:18:25.187007 11046 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.191804 11118 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:25.214932 11107 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:25.184307 10866 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.274022  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:25.282191 10898 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 13:18:25.282859 10898 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.285836 10898 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.231161 11105 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:25.290144  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1302) took 1264 ms. Trace:
I20260709 13:18:25.190256 11049 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:18:25.291484  6515 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:25.290634  7232 rpcz_store.cc:276] 0709 13:18:24.025993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.028615 (+  2622us) service_pool.cc:224] Handling call
0709 13:18:24.028995 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.067662 (+ 38667us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:24.068008 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.196858 (+128850us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.318709 (+121851us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:24.345781 (+ 27072us) log.cc:844] Serialized 26 byte log entry
0709 13:18:24.354183 (+  8402us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:24.354238 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.354322 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.289922 (+935600us) raft_consensus.cc:1685] finished
0709 13:18:25.289945 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.290085 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80769,"prepare.queue_time_us":39990,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":157063,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":1854,"spinlock_wait_cycles":108878080,"thread_start_us":205339,"threads_started":2,"wal-append.queue_time_us":190346}
W20260709 13:18:25.251791 11111 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.491s	user 0.000s	sys 0.001s
I20260709 13:18:25.214530 10902 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.303457 10866 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.305183  7230 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:25.252306 11118 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:25.294245 11130 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:18:25.249984 10623 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:18:25.133564 11119 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:25.315114 11117 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.388s	user 0.000s	sys 0.001s
W20260709 13:18:25.325987  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 52040.6us
I20260709 13:18:25.304522 11108 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:25.328639  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:25.330672 11144 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:25.332300 10703 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.198s	user 0.011s	sys 0.000s
I20260709 13:18:25.337298 11119 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.340269 11146 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:25.342321 11032 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.138341 11078 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.241784 11058 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.346176 10902 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.314918 11118 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.347409  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2560) took 1833 ms. Trace:
I20260709 13:18:25.304314 10641 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:25.351501  6670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 843) took 1748 ms (client timeout 2150 ms). Trace:
W20260709 13:18:25.393239  6670 rpcz_store.cc:269] 0709 13:18:23.603559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.710893 (+107334us) service_pool.cc:224] Handling call
0709 13:18:25.351324 (+1640431us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54001e0ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.351424 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241368448}
W20260709 13:18:25.354146 11042 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.153s	user 0.012s	sys 0.000s
I20260709 13:18:25.355230 10995 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.414950 10912 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.416234 10912 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.369848 11146 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.418781 11146 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:25.358284  7236 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:25.360265 11144 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:25.361114  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:25.391443  6593 rpcz_store.cc:276] 0709 13:18:23.514464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.584156 (+ 69692us) service_pool.cc:224] Handling call
0709 13:18:23.584517 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.302362 (+717845us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:24.302407 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.302457 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:24.302508 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.347172 (+1044664us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b540034fb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:25.347220 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.347351 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":862,"raft.queue_time_us":1139,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":28373,"spinlock_wait_cycles":1551564160}
I20260709 13:18:25.413607 10641 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:25.413443 11042 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:25.422421  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 61401us
I20260709 13:18:25.375298 11032 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:25.425808  6487 leader_election.cc:400] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11.
I20260709 13:18:25.423125 11126 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:25.427417  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1295) took 1692 ms. Trace:
I20260709 13:18:25.428746  7229 rpcz_store.cc:276] 0709 13:18:23.735159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.735701 (+   542us) service_pool.cc:224] Handling call
0709 13:18:24.237195 (+501494us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ceee 0000000000375939 00007f04bdfd01e0 00007f04be0990c4 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:18:24.237346 (+   151us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:24.250832 (+ 13486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.250864 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:24.252901 (+  2037us) log.cc:844] Serialized 52 byte log entry
0709 13:18:24.384177 (+131276us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:24.384216 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.384276 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.423454 (+1039178us) raft_consensus.cc:1685] finished
0709 13:18:25.423472 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.427350 (+  3878us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81868,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":20853,"spinlock_wait_cycles":2782976,"thread_start_us":78060,"threads_started":1,"wal-append.queue_time_us":212738}
I20260709 13:18:25.432319  6487 leader_election.cc:403] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:18:25.426581 11143 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:18:25.452636 11092 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:25.375433 10703 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.453215 11092 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:25.454845 10703 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.374748 11076 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.456921  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2557) took 1960 ms. Trace:
I20260709 13:18:25.457162 11076 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.457314  6597 rpcz_store.cc:276] 0709 13:18:23.496229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.518993 (+ 22764us) service_pool.cc:224] Handling call
0709 13:18:23.519734 (+   741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.331968 (+812234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.456691 (+1124723us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54002f71d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:25.456866 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865693952}
W20260709 13:18:25.458073 11022 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.000s
W20260709 13:18:25.473866  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 300001us
I20260709 13:18:25.475149  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:25.476218 10281 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:25.471146 11060 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.480861 11108 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:25.488323 11108 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:25.494450  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 68757.5us
I20260709 13:18:25.480921 10995 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:18:25.498068 11061 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:25.498471 11061 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:25.499171  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.481129 11151 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11.
I20260709 13:18:25.500626  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1193) took 2048 ms. Trace:
I20260709 13:18:25.501181  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2214) took 1614 ms. Trace:
I20260709 13:18:25.501011  7234 rpcz_store.cc:276] 0709 13:18:23.452078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.452239 (+   161us) service_pool.cc:224] Handling call
0709 13:18:23.452623 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.348242 (+895619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.500380 (+1152138us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b5400170a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:25.500553 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2058173184}
W20260709 13:18:25.503026 10883 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:18:25.501770  6516 rpcz_store.cc:276] 0709 13:18:23.887133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.887259 (+   126us) service_pool.cc:224] Handling call
0709 13:18:23.897939 (+ 10680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:23.898281 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:24.189182 (+290901us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18003616c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:18:24.282990 (+ 93808us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.499158 (+1216168us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b18003616c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287
0709 13:18:25.499337 (+   179us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:25.499781 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:18:25.499930 (+   149us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:25.499965 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.500026 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.500996 (+   970us) raft_consensus.cc:1685] finished
0709 13:18:25.501012 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.501122 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63072,"prepare.queue_time_us":292626,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":119068,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":12885,"spinlock_wait_cycles":59790080,"thread_start_us":429881,"threads_started":3,"wal-append.queue_time_us":533934}
I20260709 13:18:25.464149 11042 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.510573  6595 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:25.511835  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2215) took 1622 ms. Trace:
W20260709 13:18:25.512077  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 263943us
I20260709 13:18:25.458592 11022 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.518101  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:25.512336  6517 rpcz_store.cc:276] 0709 13:18:23.889018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.889499 (+   481us) service_pool.cc:224] Handling call
0709 13:18:23.895175 (+  5676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.895370 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:23.911793 (+ 16423us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.251804 (+340011us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.353244 (+1101440us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b180049a020. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:25.511146 (+157902us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:25.511216 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:25.511264 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.511669 (+   405us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.511784 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13725,"prepare.queue_time_us":295361,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":111,"raft.queue_time_us":130316,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":170,"spinlock_wait_cycles":11516160,"thread_start_us":481997,"threads_started":3,"wal-append.queue_time_us":392021,"wal-append.run_cpu_time_us":1534,"wal-append.run_wall_time_us":1005194}
I20260709 13:18:25.512923 11107 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:25.544739 11153 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:25.545390 11153 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:25.529488  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.547363 10281 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:25.548012 10281 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:25.549201 11134 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
I20260709 13:18:25.514124 11076 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:25.550490 11076 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.552276  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1174) took 1153 ms. Trace:
I20260709 13:18:25.552546 11076 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.552747 11153 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:25.552666  7235 rpcz_store.cc:276] 0709 13:18:24.398418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.677073 (+278655us) service_pool.cc:224] Handling call
0709 13:18:25.552109 (+875036us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54006a0754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.552221 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388928384}
I20260709 13:18:25.553514 10281 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:25.555604  7228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1202) took 1654 ms (client timeout 2056 ms). Trace:
W20260709 13:18:25.556018  7228 rpcz_store.cc:269] 0709 13:18:23.901270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.901455 (+   185us) service_pool.cc:224] Handling call
0709 13:18:25.555399 (+1653944us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540057d854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.555536 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3529678208}
I20260709 13:18:25.563180 10978 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:18:25.563362  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.563678 10978 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:25.564492 11118 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.565706  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2228) took 1566 ms. Trace:
I20260709 13:18:25.566051  6521 rpcz_store.cc:276] 0709 13:18:23.999815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.014591 (+ 14776us) service_pool.cc:224] Handling call
0709 13:18:25.565510 (+1550919us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540019b9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.565624 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604502784}
I20260709 13:18:25.454275 11092 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:25.568573 11123 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.652s	user 0.002s	sys 0.000s
I20260709 13:18:25.569391 11022 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.570209  6592 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:25.513229 10883 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.543424  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1188) took 1026 ms. Trace:
I20260709 13:18:25.573875  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:25.573711  7233 rpcz_store.cc:276] 0709 13:18:24.516932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.153603 (+636671us) service_pool.cc:224] Handling call
0709 13:18:25.157846 (+  4243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.182490 (+ 24644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:25.182547 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:25.191309 (+  8762us) log.cc:844] Serialized 26 byte log entry
0709 13:18:25.380878 (+189569us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:25.516421 (+135543us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.543077 (+ 26656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.543157 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.543202 (+    45us) raft_consensus.cc:1685] finished
0709 13:18:25.543218 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.543362 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26373,"prepare.queue_time_us":149259,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":102019,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":22849,"thread_start_us":235430,"threads_started":3,"wal-append.queue_time_us":244205}
I20260709 13:18:25.517035 10623 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.546358 10979 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:25.530447 11108 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.571913 11118 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.575639 10883 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.580410 10623 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.586824 11061 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.588054 11061 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:25.588536 10623 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.611918  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:25.609709 11022 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.625173 11022 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.592715 11146 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.564536 10978 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.632438 10978 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:25.633543 11146 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.635433 11022 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.636623  6514 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:25.644120 11103 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.647295 11103 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.650986 10993 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.650734 11032 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.651479 10993 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.652215  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:25.652928 11032 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:25.653383  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.654301  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2474) took 1971 ms. Trace:
I20260709 13:18:25.654695  6667 rpcz_store.cc:276] 0709 13:18:23.683404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.868244 (+184840us) service_pool.cc:224] Handling call
0709 13:18:23.875679 (+  7435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.053529 (+177850us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:24.123311 (+ 69782us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.280783 (+157472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.280852 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:24.280912 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.654102 (+1373190us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540011dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:25.654143 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.654251 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22094,"prepare.queue_time_us":72927,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":24390,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":11737,"spinlock_wait_cycles":367332224,"thread_start_us":113175,"threads_started":1,"wal-append.queue_time_us":266341}
W20260709 13:18:25.658689  7198 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.150s (SENT) [suppressed 19 similar messages]
W20260709 13:18:25.659649  7237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1169) took 1275 ms (client timeout 1540 ms). Trace:
W20260709 13:18:25.660068  7237 rpcz_store.cc:269] 0709 13:18:24.395438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.402021 (+  6583us) service_pool.cc:224] Handling call
0709 13:18:25.659453 (+1257432us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540078e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.659587 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407009536}
I20260709 13:18:25.660825 10883 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.694532  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:25.698684  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.699204  6516 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:25.703606 10777 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.704290  6521 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:25.705046 10690 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.706039 10690 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:25.707792 11049 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:25.708289 11049 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.710264 11049 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.712812 11121 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.713332 11121 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.715101 11058 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.715289 11057 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.715598 11058 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.715894 11057 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.716717  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.717628 11058 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.715668 11022 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:18:25.720690  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 985) took 1016 ms. Trace:
I20260709 13:18:25.721068  6595 rpcz_store.cc:276] 0709 13:18:24.704604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.704717 (+   113us) service_pool.cc:224] Handling call
0709 13:18:25.720537 (+1015820us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54002f7bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.720633 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727670016}
I20260709 13:18:25.723520  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1186) took 1287 ms. Trace:
I20260709 13:18:25.723927  7230 rpcz_store.cc:276] 0709 13:18:24.436471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.929555 (+493084us) service_pool.cc:224] Handling call
0709 13:18:25.092205 (+162650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.305774 (+213569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.723284 (+417510us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54006e14d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:25.723460 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489162624}
W20260709 13:18:25.732339 11129 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.630s	user 0.002s	sys 0.000s
I20260709 13:18:25.733754 11106 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:25.735545 10883 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:25.735998 10883 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:25.715356 11121 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.738340 10883 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.739393  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 850) took 1219 ms. Trace:
I20260709 13:18:25.742113  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:25.742343  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:25.743932  6594 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:25.739786  6522 rpcz_store.cc:276] 0709 13:18:24.520165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.520333 (+   168us) service_pool.cc:224] Handling call
0709 13:18:25.739210 (+1218877us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b540019bed4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.739339 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393285376}
I20260709 13:18:25.746835  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:25.747128  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.747418  6516 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.747742  6522 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:25.748570  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.750895  6522 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:25.739814 10883 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.751752  6517 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:25.753551  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 858) took 1221 ms. Trace:
W20260709 13:18:25.761005  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.791304  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.790521  6519 rpcz_store.cc:276] 0709 13:18:24.531964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.807382 (+275418us) service_pool.cc:224] Handling call
0709 13:18:24.807658 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.899567 (+ 91909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.753311 (+853744us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b54003bbad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:25.753491 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171374976}
I20260709 13:18:25.749894 11103 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:25.794728  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:25.760561 11109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.245s	user 0.004s	sys 0.018s
I20260709 13:18:25.717907 11057 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.799582  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 847) took 2074 ms. Trace:
I20260709 13:18:25.799945  6663 rpcz_store.cc:276] 0709 13:18:23.724794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.980465 (+255671us) service_pool.cc:224] Handling call
0709 13:18:24.011702 (+ 31237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.114759 (+1103057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.799383 (+684624us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000cce54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:25.799529 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2532546304}
I20260709 13:18:25.802388 10777 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:25.804049 10777 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.806020 10777 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:25.792876 11106 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:25.806686  7236 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:18:25.808146 11169 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:18:25.799328 11109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.286s	user 0.004s	sys 0.019s
W20260709 13:18:25.814294  6419 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 250) took 28877 ms (client timeout 15000 ms). Trace:
I20260709 13:18:25.815634 11115 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:25.828310 11115 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:25.816682 11127 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.631s	user 0.001s	sys 0.000s
I20260709 13:18:25.789698 11076 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:25.793735 11061 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.824913  6521 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:25.827152  6415 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 9, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 9 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:25.832155 11101 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:25.802193  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:25.796926  7237 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:25.835005  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:25.857141  7235 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:25.859529  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 57448.6us
I20260709 13:18:25.860589  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:25.861938  7237 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:25.864675  6415 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.869927  6415 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 11, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 11 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:25.864914  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:25.873003  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:25.829157 11115 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.879518  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:25.880030 11045 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.856056 11076 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.836047 11106 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:25.886272 11046 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.886957 10775 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:25.888283 11101 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.889158  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:25.890750  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:25.891177  6590 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:25.911125  6593 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.891487 11153 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:25.891862 10850 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.891997 10661 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.917408 10661 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.919437 10661 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:25.892349  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:25.864187 10504 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:25.895047  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:25.874053 11119 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.921861 11119 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.923957 11119 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.858381 10841 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:25.897691  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.924618 10841 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.926793 10841 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:25.897809  7235 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:18:25.900110  6415 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 7, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 7 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:25.903754  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.904475  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:25.931155  7237 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:25.912045  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.912511  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2481) took 2001 ms. Trace:
I20260709 13:18:25.932468  6666 rpcz_store.cc:276] 0709 13:18:23.911155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.019004 (+107849us) service_pool.cc:224] Handling call
0709 13:18:24.345404 (+326400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.347612 (+  2208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.912358 (+1564746us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b540005b250. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:25.912463 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749885184}
I20260709 13:18:25.939703  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:25.940538  6522 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:25.913049  6516 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:25.913556 10775 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.916217 11116 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.909300 10809 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.916587 10850 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.946152  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:25.907814 11024 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.901948 11032 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.957381  6670 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:25.908993 10281 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:25.927556 11023 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:25.959712  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 80267.8us
I20260709 13:18:25.960626  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:25.963120  6671 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:25.975334  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:25.966658  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:25.976126  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:25.968700  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:25.913882 10777 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:25.947597  6415 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 6 to 8, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 8 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.914368 11153 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.872027 11061 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.964277  6664 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:25.984304  6666 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:25.988214  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.988653  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:25.990801  6663 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:25.912199 11045 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:25.991477  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.998373  6415 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 8, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:25.967953 10661 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:25.959707 11115 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.002058  6593 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:26.003880  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.004609  6667 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:26.007179  7237 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:25.970865 10281 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:25.969651 10883 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:25.941272 11101 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:25.914777 11046 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.020833  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:25.963752 10841 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:18:25.989648 11153 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:26.024879  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.025986  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2543) took 1431 ms. Trace:
I20260709 13:18:26.026778  6517 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:18:26.028179  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.030331 11132 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:26.034230  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.039949  6562 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:26.083309  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.992482 10995 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:26.064642 11094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.010s	sys 0.004s
I20260709 13:18:26.069417  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2477) took 2384 ms. Trace:
I20260709 13:18:26.072372  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2098) took 1253 ms. Trace:
I20260709 13:18:25.961699 10809 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.097134  6662 rpcz_store.cc:276] 0709 13:18:23.684947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.886418 (+201471us) service_pool.cc:224] Handling call
0709 13:18:23.886761 (+   343us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:18:24.307064 (+420303us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:24.354072 (+ 47008us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.780972 (+426900us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:24.810866 (+ 29894us) log.cc:844] Serialized 78 byte log entry
0709 13:18:24.811094 (+   228us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:24.811422 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.568001 (+756579us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:25.831682 (+263681us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b18003f9d00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04bcf5558f 00007f04b5f94571 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd
0709 13:18:25.951953 (+120271us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.959721 (+  7768us) log.cc:844] Serialized 14 byte log entry
0709 13:18:26.068537 (+108816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.068755 (+   218us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:26.068853 (+    98us) raft_consensus.cc:1685] finished
0709 13:18:26.068883 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.069256 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1134120,"prepare.queue_time_us":60353,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":3371,"raft.queue_time_us":1252254,"raft.run_cpu_time_us":543,"raft.run_wall_time_us":58658,"spinlock_wait_cycles":42148608,"thread_start_us":904144,"threads_started":4,"wal-append.queue_time_us":435803}
I20260709 13:18:26.101640  6415 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:26.076015  6513 rpcz_store.cc:276] 0709 13:18:24.594787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.679783 (+ 84996us) service_pool.cc:224] Handling call
0709 13:18:26.025805 (+1346022us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540014e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:26.025931 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297915136}
W20260709 13:18:26.081521 11111 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:18:25.829725  6419 rpcz_store.cc:269] 0709 13:17:56.936745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.943906 (+  7161us) service_pool.cc:224] Handling call
0709 13:18:25.814081 (+28870175us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:12.707297 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:12.707945 (+   648us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:12.708006 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:18:12.708030 (+    24us) tablet.cc:662] Decoding operations
0709 13:18:12.709022 (+   992us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:12.709093 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:12.709124 (+    31us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:18:12.709919 (+   795us) tablet.cc:701] Row locks acquired
0709 13:18:12.709969 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:18:12.710168 (+   199us) write_op.cc:270] Start()
0709 13:18:12.710312 (+   144us) write_op.cc:276] Timestamp: P: 1783603092710098 usec, L: 0
0709 13:18:12.710335 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:18:12.763607 (+ 53272us) log.cc:844] Serialized 3194 byte log entry
0709 13:18:13.382395 (+618788us) op_driver.cc:464] REPLICATION: finished
0709 13:18:13.911379 (+528984us) write_op.cc:301] APPLY: starting
0709 13:18:13.924380 (+ 13001us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:15.832185 (+1907805us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b18002636e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8dc00ec 00007f04b8dc2af7 00007f04b8dc3fb0 00007f04bcad81bb 00007f04bcad8532 00007f04bcb1fb93 00007f04bc9ccde3 00007f04bc9cc6ab 00007f04bc9cde77
0709 13:18:15.942792 (+110607us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:15.996553 (+ 53761us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:16.265463 (+268910us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:16.396596 (+131133us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:18:16.402610 (+  6014us) write_op.cc:312] APPLY: finished
0709 13:18:16.504401 (+101791us) log.cc:844] Serialized 81 byte log entry
0709 13:18:16.828499 (+324098us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:16.856195 (+ 27696us) write_op.cc:454] Released schema lock
0709 13:18:16.873795 (+ 17600us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:16.941782 (+ 67987us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:16.945171 (+  3389us) write_op.cc:454] Released schema lock
0709 13:18:17.460019 (+514848us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b18002636e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
Metrics: {"mutex_wait_us":60143,"reported_tablets":44,"spinlock_wait_cycles":12672,"child_traces":[["op",{"apply.queue_time_us":508840,"apply.run_cpu_time_us":41432,"apply.run_wall_time_us":3019127,"cfile_cache_hit":30,"cfile_cache_hit_bytes":31335,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6216,"cfile_init":1,"lbm_read_time_us":948,"lbm_reads_lt_1ms":8,"num_ops":8,"prepare.queue_time_us":671229,"prepare.run_cpu_time_us":25091,"prepare.run_wall_time_us":208569,"raft.queue_time_us":148347,"raft.run_cpu_time_us":32412,"raft.run_wall_time_us":308250,"replication_time_us":671867,"spinlock_wait_cycles":26904704,"thread_start_us":1377417,"threads_started":5,"wal-append.queue_time_us":825892,"wal-append.run_cpu_time_us":19244,"wal-append.run_wall_time_us":2315096}]]}
I20260709 13:18:25.989260 11023 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.086025  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:25.959009 11116 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:26.096925 11094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.011s	sys 0.005s
I20260709 13:18:26.089345  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:26.127343  6664 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.970319 11024 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:26.096306  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62133.8us
I20260709 13:18:26.097132  6669 rpcz_store.cc:276] 0709 13:18:24.819225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.847916 (+ 28691us) service_pool.cc:224] Handling call
0709 13:18:26.071941 (+1224025us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b54000e60d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:26.072216 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3588480}
I20260709 13:18:25.975541 11032 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:25.992081 10777 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.081892 10883 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.047019 10281 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:26.018832 11106 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:26.096675 11097 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:18:26.114372  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.120473  6415 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 6 to 9, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 9 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:26.082161 10850 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.123610  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:26.157461  6515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 854) took 1632 ms (client timeout 1893 ms). Trace:
I20260709 13:18:26.157552  6671 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:26.157721  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:26.093196  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:26.158710  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 65592.9us
I20260709 13:18:26.126408  6670 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:26.160590  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:26.129488  6666 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:26.096781 11050 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:26.090236 10995 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:26.074692 11122 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:26.180286  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.104058 10887 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:26.141639  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:26.185722  6669 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:26.142607  6589 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:26.044093  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 980) took 1350 ms. Trace:
I20260709 13:18:26.185842  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.160677  6415 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:26.186578  6593 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:26.186219  6594 rpcz_store.cc:276] 0709 13:18:24.694016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.694185 (+   169us) service_pool.cc:224] Handling call
0709 13:18:24.694595 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.744425 (+1049830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.043481 (+299056us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003a2354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:26.043953 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2412409600}
I20260709 13:18:26.104364 11178 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:18:26.084034  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44383.5us
I20260709 13:18:26.137176 10661 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:26.140290 11015 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.159255  6516 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:18:26.199362  6516 pending_rounds.cc:85] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 7
W20260709 13:18:26.157925  6515 rpcz_store.cc:269] 0709 13:18:24.525281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.554018 (+ 28737us) service_pool.cc:224] Handling call
0709 13:18:26.157272 (+1603254us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54003cc154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:26.157406 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1625988992}
I20260709 13:18:26.199656  6516 pending_rounds.cc:107] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:26.142920 11184 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:18:26.158355  6514 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.203332  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.172467  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:26.140959 11076 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.174685  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:26.210960  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.178879  6595 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:26.179863  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 204586us
I20260709 13:18:26.184531  6597 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:26.153014 11094 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.119244 10841 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:26.187680  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101756us
I20260709 13:18:26.184998 10887 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:26.155434 11119 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.194199  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.192595  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.195043  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:26.242718 11187 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:18:26.179566 11110 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:26.197250  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:26.257499 11180 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:18:26.199918  6516 raft_consensus.cc:2889] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:26.173074 11063 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:26.273427 11189 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:18:26.268589  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.167881  6667 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:26.268554 11110 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.275089 11110 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:26.275259 11179 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:26.276119  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:26.277128 11116 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.183434 10995 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:26.279246 10990 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:26.279932 11023 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.280753 10809 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:26.281489 11174 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:18:26.200443  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:26.205856 11092 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:26.206414  6515 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:18:26.206542  7231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1183) took 1797 ms (client timeout 1965 ms). Trace:
W20260709 13:18:26.267839 11171 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.249s	user 0.002s	sys 0.000s
I20260709 13:18:26.205539  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.289896  6522 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:26.208276 10993 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:26.209089 11046 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.209739 10883 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:26.209842  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 987) took 1427 ms. Trace:
I20260709 13:18:26.294677  6592 rpcz_store.cc:276] 0709 13:18:24.782213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.783071 (+   858us) service_pool.cc:224] Handling call
0709 13:18:26.209670 (+1426599us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400080a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:26.209788 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1741154560}
I20260709 13:18:26.295727  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:26.296352  6592 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:26.211665  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.175833  6664 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:26.222559  6514 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:26.222627  6521 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 13:18:26.189756 11050 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:26.192596 10661 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:26.201632 11132 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.242033  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:26.302948  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61001.3us
W20260709 13:18:26.245311  6487 leader_election.cc:336] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.059s (SENT)
I20260709 13:18:26.246781  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:26.247263  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.307928  7233 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:26.256155 10841 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.292018 11032 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.255040  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.255532 11182 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:26.248317  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.257050  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:26.318015  6669 raft_consensus.cc:2376] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9. Re-sending same reply.
W20260709 13:18:26.258808  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.259737  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.318495  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:26.319547  6664 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:26.222882 11152 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:26.292335 11076 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:26.263528  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:26.269711 11119 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.267079  6597 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:26.326426  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 81147.3us
I20260709 13:18:26.197111  6415 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 5, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:26.327095  6487 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.213s (SENT)
W20260709 13:18:26.261927 11181 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:18:26.286168  7231 rpcz_store.cc:269] 0709 13:18:24.409284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.534233 (+124949us) service_pool.cc:224] Handling call
0709 13:18:26.206360 (+1672127us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54004de254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:26.206487 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328274688}
I20260709 13:18:26.207222  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:26.332230  6594 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:26.334275  7233 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:26.285626  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:26.288113  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 87729.9us
I20260709 13:18:26.288825  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2096) took 1470 ms. Trace:
I20260709 13:18:26.261169 11061 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:26.292562  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.339967  6596 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:26.292111 10993 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.298604  6522 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:26.300772  6588 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:26.312780  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:26.242413 11135 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.000s	user 0.003s	sys 0.000s
I20260709 13:18:26.321981  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:26.349292  6593 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:26.293134 11058 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:26.259649 11094 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.323086  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:26.165874 10682 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:26.290949 11045 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.295293 11101 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.270774 11190 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:18:26.355146 11058 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.405980 11058 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:26.407419 11190 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:26.408262 11190 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:26.408615 11190 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:26.408978 11190 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.330379  6415 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 7, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 7 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:26.335784  6665 rpcz_store.cc:276] 0709 13:18:24.818609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.931591 (+112982us) service_pool.cc:224] Handling call
0709 13:18:24.931895 (+   304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:25.832647 (+900752us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:25.994903 (+162256us) log.cc:844] Serialized 14 byte log entry
0709 13:18:26.131490 (+136587us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:26.138251 (+  6761us) log.cc:844] Serialized 52 byte log entry
0709 13:18:26.138391 (+   140us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:26.138429 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.138496 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:26.273382 (+134886us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b180008a340. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:26.284209 (+ 10827us) raft_consensus.cc:1685] finished
0709 13:18:26.288586 (+  4377us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.288750 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":962878,"prepare.queue_time_us":318443,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":951103,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":10194,"spinlock_wait_cycles":5011328,"thread_start_us":159840,"threads_started":2,"wal-append.queue_time_us":57130}
I20260709 13:18:26.324873 10887 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.412557  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:26.413604  6665 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:26.321453 11023 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:26.342407  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56862us
I20260709 13:18:26.358777 10777 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.315821 11132 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.416671 10856 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:26.324443 11107 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:26.417634 11132 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.417723 11107 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.419256  6662 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:26.419055 11190 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.357353  6667 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:26.361881  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:26.421574  6669 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:26.363714  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:26.344895 11061 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:18:26.366681 11094 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:26.368108  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.376377 11119 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.324668 10883 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:18:26.451756  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.354089 10993 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.412278  6415 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:26.458230  6415 catalog_manager.cc:5697] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 8, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 8 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:26.414755 11023 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:26.422151  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 58547.4us
I20260709 13:18:26.465215  6593 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:26.465683  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:26.414657  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.324514 11046 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:18:26.455168 11076 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:26.347921 10690 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.641s	user 0.013s	sys 0.000s
I20260709 13:18:26.344060 11057 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:26.346031 11024 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.471812  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.504076  6669 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 7.
I20260709 13:18:26.445618 10777 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:26.522982  7236 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
W20260709 13:18:26.523013  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 204584us
I20260709 13:18:26.529318  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.536844  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:26.470825 11023 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.469110  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:26.545275  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76264.8us
I20260709 13:18:26.471007 11108 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.467470  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:26.469129  6415 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 9, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 9 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:26.467348 11193 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:26.468242 11046 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:26.468168 10841 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.468613 11076 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.478173 11053 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:26.567523 11053 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:26.567994 11076 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.568392 11053 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.569664 11053 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:26.480521  6671 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:26.488703  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:26.575394  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 86790.5us
I20260709 13:18:26.576951  6671 raft_consensus.cc:1217] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[9.5-11.6]   Dedup: 9.5->[11.6-11.6]
I20260709 13:18:26.576987  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.577634  6671 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 13:18:26.489840  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.489609  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.578243  6518 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 7.
I20260709 13:18:26.490694  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.492005  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.490903  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.496918  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1189) took 1745 ms. Trace:
I20260709 13:18:26.464928 10995 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.580583  6662 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:26.581003  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.582074  6513 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:26.581185  7232 rpcz_store.cc:276] 0709 13:18:24.751242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.295335 (+544093us) service_pool.cc:224] Handling call
0709 13:18:25.570995 (+275660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.571338 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:25.571375 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:25.576816 (+  5441us) log.cc:844] Serialized 26 byte log entry
0709 13:18:25.760327 (+183511us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:25.799668 (+ 39341us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.830278 (+ 30610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.830350 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.830395 (+    45us) raft_consensus.cc:1685] finished
0709 13:18:25.830409 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.173378 (+342969us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b180036f100. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4
0709 13:18:26.496744 (+323366us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54006a0750. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:26.496858 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30212,"prepare.queue_time_us":37995,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":759,"raft.queue_time_us":32160,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":6132,"spinlock_wait_cycles":584174720,"thread_start_us":150194,"threads_started":1,"wal-append.queue_time_us":237166}
I20260709 13:18:26.499495  6668 raft_consensus.cc:606] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:26.586122  6668 raft_consensus.cc:614] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:26.501957  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 174874us
W20260709 13:18:26.586943  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.588620  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:26.502327  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.589901 11109 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:26.590314  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.590747 11054 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:26.592321 10993 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:26.474644 11184 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:18:26.547103  6665 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.593043 11115 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:26.593464 11132 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:26.594077 11115 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:26.594425 11115 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:26.595146 11115 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.469643 10690 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:26.546221  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.521896 11057 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:26.599627  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53459.7us
I20260709 13:18:26.599961 11057 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.579025  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.603346  7228 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:26.604969  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:26.605566  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.606340  7233 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:26.614852  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.614852  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.618480  6517 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:26.560547  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:26.503485 11107 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:26.565119  6514 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:26.563191  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.621233  6665 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:26.554270 11193 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:26.566049 11095 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:26.557595 11046 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:26.566417 10775 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.567001  6664 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:26.558992 11061 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:26.577544  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.642987  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.643514  6593 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:26.643699  6517 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:26.495429  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.578856  6671 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:26.644291  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.592527  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.498056  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.645932  6670 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:26.523149 10777 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:26.559370 11108 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.626637 11115 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:26.607677  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:26.548055  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.579586  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.578474  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.605062  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.554304  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.617244  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:26.453209  6594 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:26.625782  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.652822  6666 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:26.653543  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.625753  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.628237  7235 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:18:26.628895  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:26.633009  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 72529.1us
I20260709 13:18:26.635883  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.636093  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.637380  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.657565  6521 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:26.567119  6667 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:26.641117  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.659111  7231 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:26.642339 11193 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.659652  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.607227  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.654485  6516 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:26.465996 10661 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.631278 10993 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:26.608477 10988 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.650152  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.579564 11190 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:26.655018  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.659974  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.564133 11063 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:26.512188 11050 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.627671 11054 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:18:26.661216  6669 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:26.664124  7230 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:26.738343  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.739372  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.740101  7230 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:26.654964  6589 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:26.626008 11132 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:26.699316 10934 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.675339  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.676570  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:26.677124 10997 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:26.654090 10777 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:26.679248  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:26.679723 10645 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:26.684188  6519 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:26.749274  7235 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:26.688136  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52622 (request call id 27) took 1949 ms. Trace:
I20260709 13:18:26.689857 11194 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:18:26.647768  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 55296.2us
I20260709 13:18:26.753015  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.636690 11103 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:26.694300  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.702090  6662 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:26.647228 11152 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:26.700901  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:26.647696 11061 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.705637  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:26.708149  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.590349 11109 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.733304 11015 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:18:26.734153 10988 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.779781 10850 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:26.780248 10850 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.780618 10850 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:26.780848 11109 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:26.636991 10504 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:26.781488 11023 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.782022 10504 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:26.783090 11152 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.655064 11107 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:26.784678 11107 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.785070 11107 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.729811  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:26.673171  6591 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [7.4-10.6]
I20260709 13:18:26.746318  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.685361  6666 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
W20260709 13:18:26.750284  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53457.2us
I20260709 13:18:26.639372 10281 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.691442 10703 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.749842  6668 rpcz_store.cc:276] 0709 13:18:24.739136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.739476 (+   340us) service_pool.cc:224] Handling call
0709 13:18:26.687950 (+1948474us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54000e56d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b907b853 00007f04be04960a 00007f04b75cf7d4 00007f04b75cf762 00007f04b75cf691 00007f04b75cf60c 00007f04b75ce8c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:26.688055 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4002554240}
I20260709 13:18:26.718430 11092 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.733525 10775 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:18:26.764218  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:26.813683  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49585.5us
I20260709 13:18:26.768982  7230 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:26.818136  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.820802  7230 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:26.649187 11110 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:26.777160  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71590.9us
I20260709 13:18:26.750264 11132 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.777680  6521 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:26.751500 10995 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:26.717348 11076 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:18:26.786682 11061 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.712430 11058 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:26.796177  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.753789  7231 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:26.718837 10993 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.801214 10703 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.753170 10934 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.802919  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.783030 10504 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.803257  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:26.805881  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:26.897800  6595 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:26.805789 10850 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.801142 11194 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:18:26.806692  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:26.810989  7237 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:26.951485  6515 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:26.952338  7235 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:18:26.952821  7235 pending_rounds.cc:85] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 8
I20260709 13:18:26.953119  7235 pending_rounds.cc:107] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:18:26.951485  6588 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:26.951526  7231 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:18:26.860251 10645 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:26.823732  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:26.960005  7230 raft_consensus.cc:2410] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:18:26.832791  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:26.832947  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:26.793393  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.765359 11190 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:18:26.812034 10281 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:26.802242  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72568.5us
I20260709 13:18:26.830814 11092 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:26.953042 11153 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.852s	user 0.006s	sys 0.005s
I20260709 13:18:26.953431  7235 raft_consensus.cc:2889] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:26.811008 11107 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:26.960912  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 128022us
W20260709 13:18:26.962800  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 130027us
I20260709 13:18:26.897835  6668 raft_consensus.cc:606] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:26.744419 11046 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.972585  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.975674  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.967320  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:26.972784  6668 raft_consensus.cc:614] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:26.977180  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:26.978145  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 274209us
I20260709 13:18:26.982991  6516 raft_consensus.cc:2368] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
I20260709 13:18:26.985044  7230 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:26.965719 11024 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:26.970932 11106 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:26.969640 10645 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:26.991180  6513 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:26.972554 11063 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:27.000738  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 307515us
I20260709 13:18:26.982028 10883 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:26.985800 10775 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:26.980227 11184 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:26.995299  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 301053us
I20260709 13:18:26.989882 11024 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.010929  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:26.974920 11153 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.978200 10841 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:18:26.979707 10993 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:26.999600 11076 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:27.041277  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.045971 10997 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.046561 10997 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.048662 10997 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.049171 10997 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.049569 10997 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:26.985301 11196 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:18:27.054535  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:27.006870 11023 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:27.055217  6668 raft_consensus.cc:606] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:27.059924  6668 raft_consensus.cc:614] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:27.009692 10775 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.999435 11198 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:27.003748 11106 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:27.006145 10883 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:27.018086 11024 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:27.067358  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56573us
I20260709 13:18:27.058857 10993 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:27.027173 11184 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.071272 10993 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.071713 10997 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.073391 10993 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:27.074396 11193 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.088127 10698 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:27.088716 10698 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:27.063457 10775 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:27.095157 11199 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:18:27.096819 11197 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:18:27.107455 11046 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.128499  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1326) took 1104 ms. Trace:
I20260709 13:18:27.211714  7237 rpcz_store.cc:276] 0709 13:18:26.025984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.072395 (+ 46411us) service_pool.cc:224] Handling call
0709 13:18:26.072967 (+   572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.951744 (+878777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.128275 (+176531us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54005a3554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:27.128438 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696772864}
I20260709 13:18:27.131875  7228 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:18:27.131212  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:27.118556  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.215497  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:27.226794  6517 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:18:27.119866 10997 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.228744 10997 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.040143 11095 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:18:27.016579 11063 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.069289 10883 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:27.218811 11023 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.066591 11198 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:27.237905 11066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.034s	sys 0.048s
I20260709 13:18:27.239219 11110 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:27.259074 11066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.034s	sys 0.049s
I20260709 13:18:27.284319  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:27.242717 11058 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.210852 11061 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.120009 10990 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:27.126732 10703 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.258925 11198 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.291464  6668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:27.322794  6668 raft_consensus.cc:606] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:27.323298  6668 raft_consensus.cc:614] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:27.293038  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:27.310546  6488 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.036s
W20260709 13:18:27.324299  6488 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.001s	sys 0.037s
W20260709 13:18:27.324694  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 572919us
I20260709 13:18:27.325896  7236 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:27.284327 10850 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:27.332687  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.284206 10934 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.208891 10645 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.300179 11058 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.302112 10990 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.305949 11092 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.363646  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:27.363217 11076 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:27.320855 10997 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.345614 11156 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.385102 11092 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.389343  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:27.377247 11209 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:27.401415 11076 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:18:27.424219  6597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.063s	sys 0.026s
W20260709 13:18:27.429040  6597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.063s	sys 0.026s
I20260709 13:18:27.287503 11106 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.377213 10934 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:27.392381 11115 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.727s	user 0.007s	sys 0.004s
I20260709 13:18:27.386902 10850 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.432511 11092 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.377746 10990 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:27.451231 11193 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.451833 11193 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.454233 11193 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:27.443264  6665 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.021s	sys 0.122s
I20260709 13:18:27.443297 11209 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:27.456691  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.054s	sys 0.066s
W20260709 13:18:27.457005  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.054s	sys 0.066s
W20260709 13:18:27.457531  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 756642us
I20260709 13:18:27.427496 11210 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.379856 11058 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:27.462803  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 487171us
W20260709 13:18:27.456605  6665 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.021s	sys 0.122s
I20260709 13:18:27.465782  7231 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:27.434720 11153 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.454293 11132 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.456483 11092 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.427910 11156 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.478031 10902 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:18:27.479679 11152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.008s	sys 0.028s
I20260709 13:18:27.478451 10698 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.464210  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:27.462462 10682 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:27.443608 10934 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:27.490595  6636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.022s
W20260709 13:18:27.500094  6636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.005s	sys 0.022s
I20260709 13:18:27.434092 10997 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.490986  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:27.458217 10850 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.455749 11209 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.492761  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:27.473935 11193 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.506316  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:27.469637 11015 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:18:27.507987  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:27.493386 10912 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.022s	sys 0.007s
W20260709 13:18:27.500910  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 533625us
I20260709 13:18:27.525092  6668 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:27.525889  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:27.527726  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:27.529623  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:27.530377  6663 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:27.498155 10682 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:27.534400 10682 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.535706 10682 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:27.490406 10902 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:27.537366  6662 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:27.500367 11152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.008s	sys 0.028s
W20260709 13:18:27.537783  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46884.6us
I20260709 13:18:27.575940  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:27.541484  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:27.514806  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 497652us
I20260709 13:18:27.543095  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:27.552745  6515 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.001s
I20260709 13:18:27.579212  6515 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:27.567972  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 108609us
I20260709 13:18:27.489962 11153 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.469990 11210 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.584923  6664 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:18:27.585711  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:27.585947  6664 pending_rounds.cc:85] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 8
I20260709 13:18:27.580355  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:27.581387  6519 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:27.586578  6664 pending_rounds.cc:107] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:18:27.588801  6664 raft_consensus.cc:2889] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:27.579246  6668 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
W20260709 13:18:27.537142 10912 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.023s	sys 0.007s
I20260709 13:18:27.519838 10698 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.590464  6669 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:18:27.595813  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:27.518852 10997 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.598242 11092 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
W20260709 13:18:27.608480 11206 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:18:27.611279 10883 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.613387 10883 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.608513  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:27.617755 11046 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.003s
I20260709 13:18:27.580753 11061 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.573696 11211 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:27.523862 11193 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.594444 11053 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:27.628687  6513 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:18:27.629506 11204 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:18:27.629779 10997 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:27.630769 11203 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:18:27.630832 11214 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:18:27.612833 11092 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:27.643867 10850 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:27.648095 11202 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:18:27.598843 11063 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:27.601938 11110 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:27.657752  6563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.015s	sys 0.031s
W20260709 13:18:27.674568  6563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.015s	sys 0.031s
I20260709 13:18:27.660763 11046 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:27.663667  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2577) took 1087 ms. Trace:
I20260709 13:18:27.649941 10703 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:27.650594 11023 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:27.659929 11057 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.007s	sys 0.041s
I20260709 13:18:27.643903 11210 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.599490 11153 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.699373 11092 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.700621 11092 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:27.683084 11057 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.007s	sys 0.042s
I20260709 13:18:27.705727 11092 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:27.664775 10856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.012s	sys 0.022s
I20260709 13:18:27.722921 11193 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:27.727294  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2102) took 1422 ms. Trace:
I20260709 13:18:27.727834 11153 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:27.728828 10856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.012s	sys 0.023s
I20260709 13:18:27.730098  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1217) took 1083 ms. Trace:
I20260709 13:18:27.732177  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2101) took 1427 ms. Trace:
I20260709 13:18:27.730938  7233 rpcz_store.cc:276] 0709 13:18:26.647183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.659394 (+ 12211us) service_pool.cc:224] Handling call
0709 13:18:27.729588 (+1070194us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54007a0154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:27.729934 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155631488}
I20260709 13:18:27.728305  6670 rpcz_store.cc:276] 0709 13:18:26.304565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.678589 (+374024us) service_pool.cc:224] Handling call
0709 13:18:26.678926 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:26.679261 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:26.679291 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:26.679756 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:18:26.980249 (+300493us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:26.980314 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.980427 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.726831 (+746404us) raft_consensus.cc:1685] finished
0709 13:18:27.726875 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.727147 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1202,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":54,"thread_start_us":298088,"threads_started":1,"wal-append.queue_time_us":813792}
I20260709 13:18:27.737392 10934 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:27.740967 11108 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.741219 10990 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.742573  6664 raft_consensus.cc:2368] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
W20260709 13:18:27.734462 10777 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.967s	user 0.009s	sys 0.002s
I20260709 13:18:27.743737 11198 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.744354 10777 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.745890 11198 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.680836 11046 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.746646 11046 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.747936 10990 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.746399 10777 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.672811 10850 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.732997  6667 rpcz_store.cc:276] 0709 13:18:26.304305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.673762 (+369457us) service_pool.cc:224] Handling call
0709 13:18:26.674124 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:26.674527 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.674559 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:26.675082 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:18:26.960301 (+285219us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:26.960358 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.960446 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.731705 (+771259us) raft_consensus.cc:1685] finished
0709 13:18:27.731750 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.732032 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5768,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":284738,"threads_started":1,"wal-append.queue_time_us":620104}
I20260709 13:18:27.752128  7230 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:27.659858  6634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.006s	sys 0.019s
I20260709 13:18:27.659756 11015 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:27.658581 11220 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:27.681628  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2103) took 1376 ms. Trace:
W20260709 13:18:27.686628  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.683693 10703 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.681753  6592 rpcz_store.cc:276] 0709 13:18:26.576114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.634014 (+ 57900us) service_pool.cc:224] Handling call
0709 13:18:26.634358 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.634588 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:26.972080 (+337492us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.663338 (+691258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:27.663392 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:27.663434 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.663499 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.663610 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5404,"prepare.queue_time_us":354871,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":193739,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":1004126,"threads_started":3,"wal-append.queue_time_us":506275}
I20260709 13:18:27.684370 11110 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.664798 11053 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:27.665477 11224 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:27.763856  6634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.007s	sys 0.020s
I20260709 13:18:27.806291  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:27.767556  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2501) took 1447 ms. Trace:
I20260709 13:18:27.772162  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:27.806481  6671 rpcz_store.cc:276] 0709 13:18:26.304824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.682429 (+377605us) service_pool.cc:224] Handling call
0709 13:18:26.682739 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.682961 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:26.683251 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.329941 (+646690us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b1800341080. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:27.678058 (+348117us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.681206 (+  3148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:27.681268 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:27.681333 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.681408 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.681556 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28335,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":16635,"raft.queue_time_us":121969,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":22004992,"thread_start_us":347684,"threads_started":1,"wal-append.queue_time_us":1011923}
I20260709 13:18:27.749274 10661 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.774211  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:27.674940  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 605481us
I20260709 13:18:27.817282  7232 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:27.812536  7200 leader_election.cc:400] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
I20260709 13:18:27.818063  7200 leader_election.cc:403] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:27.818326  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:27.808223  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1212) took 1203 ms. Trace:
I20260709 13:18:27.772868  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:27.820642  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:27.821437  7237 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:27.822795  7237 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:27.660524 10902 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.814699  6665 rpcz_store.cc:276] 0709 13:18:26.320669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.701191 (+380522us) service_pool.cc:224] Handling call
0709 13:18:26.701492 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.701725 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:27.468888 (+767163us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.767140 (+298252us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:27.767207 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:27.767270 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.767360 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.767484 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53906,"prepare.queue_time_us":63644,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":93115,"thread_start_us":970397,"threads_started":2}
I20260709 13:18:27.813010 10887 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:27.814875 11058 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.771055 10281 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.811081 10703 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:27.817551 10775 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.000s
I20260709 13:18:27.806561 11124 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:27.805992 10934 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.815511  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:27.831874  7228 rpcz_store.cc:276] 0709 13:18:26.604864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.607255 (+  2391us) service_pool.cc:224] Handling call
0709 13:18:27.807991 (+1200736us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540027d8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:27.808153 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104803712}
I20260709 13:18:27.811194 11220 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.813962 11196 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:27.838023  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.839534  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:27.863952  7235 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.011s	sys 0.008s
I20260709 13:18:27.824707 11108 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
W20260709 13:18:27.868168 11024 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.002s
W20260709 13:18:27.875844  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 280124us
W20260709 13:18:27.862030  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 283301us
I20260709 13:18:27.863000  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:27.814591 10777 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:18:27.805817 10850 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.869936  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:27.887509  7228 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:27.872210  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:27.859443 10902 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.864385 11193 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:18:27.833864 10887 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:27.873705  7235 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.011s	sys 0.008s
I20260709 13:18:27.848244  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:27.865916  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 357967us
I20260709 13:18:27.866182 11050 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.865756 10775 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.883569  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:27.891938  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.869385 11226 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:18:27.897562  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:27.902905  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:27.903615 11015 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:27.903981  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:27.903981 11015 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.904178 10850 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.904573 11015 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:27.872915 11220 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:27.874270  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:27.898984  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50867.7us
I20260709 13:18:27.905253 11108 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:27.905493 11237 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:18:27.899451  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 319155us
I20260709 13:18:27.909555 11095 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:27.909947 11095 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.910310 11095 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:27.901583 11050 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.865885 10281 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.912711 11050 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.913049 10281 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:27.913668  7232 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:27.914397  7237 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:27.895336 10690 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.298s	user 0.014s	sys 0.000s
I20260709 13:18:27.877311 11066 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:27.906997 11108 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.915416  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1332) took 1145 ms. Trace:
I20260709 13:18:27.919842  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:27.923534  6513 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:27.924153  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:27.924721  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1211) took 1320 ms. Trace:
I20260709 13:18:27.927039 10887 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.928680 10504 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:27.935385 10988 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.968379  6667 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:27.968793  6667 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:27.936285 11107 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:27.937238  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:27.896275 11224 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:18:27.896763 10856 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:27.876530 11196 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.876344 11053 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.880752 11024 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:27.939612  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:27.941184 11184 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.942688  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:27.943851  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:27.944478  6516 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:18:27.987700  6516 pending_rounds.cc:85] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 7
I20260709 13:18:27.988042  6516 pending_rounds.cc:107] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:18:27.988284  6516 raft_consensus.cc:2889] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:27.971551 11063 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:27.907837  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:27.968501 10504 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:27.989493  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:27.991371  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:27.992581  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:27.993163  6668 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:27.950438 11209 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:27.950811 11226 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.951383 11132 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:27.954944  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:27.873356 10934 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:27.954946  6662 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:18:27.917928  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.956403  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:27.906023  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:27.957114  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.001077  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:28.001550  6591 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:27.957828  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:27.958698  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:27.950491  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 67030.7us
I20260709 13:18:28.002475  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:27.961148  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:27.962046  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:28.004019  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.004387  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.004920  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.005532  6595 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
W20260709 13:18:28.005957  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 100021us
I20260709 13:18:28.006359  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:27.964671  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44079.4us
I20260709 13:18:28.007010  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:27.904902 10990 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:27.964942  7229 rpcz_store.cc:276] 0709 13:18:26.604358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.604664 (+   306us) service_pool.cc:224] Handling call
0709 13:18:27.924500 (+1319836us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540078e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:27.924656 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2838691072}
I20260709 13:18:27.962978  7231 rpcz_store.cc:276] 0709 13:18:26.770467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.955634 (+185167us) service_pool.cc:224] Handling call
0709 13:18:27.465573 (+509939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.466315 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.915220 (+448905us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b540005bc50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:27.915357 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1139988864}
I20260709 13:18:27.966544  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.008576  6665 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:27.967029  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:27.967082  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:27.970348  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:27.955085 11212 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:27.970774  6667 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.010681  6667 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.977808  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:27.978011  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:28.011474  6522 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:28.011898  6671 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:27.960723 11066 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.981393  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:27.966528 11108 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.962574 10690 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:28.022043  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:28.020987  6519 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:27.989673  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81890.1us
I20260709 13:18:27.913703 11156 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:27.996440  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:27.973017 11245 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:27.978794 10856 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:28.005226  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:28.027271  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.027771  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.028241  7236 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:28.028330  6590 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:27.977371 11241 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:18:28.019373  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:28.020304  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:28.005223  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.029793  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:28.030310  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1201) took 1060 ms. Trace:
I20260709 13:18:27.996271 11193 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:27.990299 11107 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:28.025744  6664 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:28.032569  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:28.021121 11066 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:28.038730  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.039244  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.038755  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:28.033483  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.040867  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:28.041281  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:28.034790  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:28.035095  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.042062  6519 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:28.042138  6664 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:28.036330  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:27.999795 11132 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:28.045569 11124 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:28.046782  7198 leader_election.cc:400] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
I20260709 13:18:28.047081  7198 leader_election.cc:403] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:18:28.049065 10841 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:28.049500 10993 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:28.049893  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:28.050434  6513 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.050554 11106 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:28.052007 11239 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.052788 11236 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:18:28.052582 11239 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.053608 11228 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:28.054111 11228 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.048926 11221 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:28.056614 11248 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:18:28.055910 11244 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:28.048904 11103 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:18:28.002707 11063 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:28.067454  7234 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:28.087440  6670 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.074438  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.024494 10988 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:28.019024 11110 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:28.032383 11107 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:28.148108  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.148535 11198 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.150705 10690 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:28.082860 11184 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:28.098662 11092 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:28.000665 11209 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.024042 10682 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:28.021616 11243 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:28.033154 10856 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:28.032218 11058 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.155s
I20260709 13:18:28.174434  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.230072  6666 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:28.027706 10990 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:28.180860 11107 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:28.180872 11076 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.180305 11103 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:28.082690 10661 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 13:18:28.186862 11198 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.151743 10850 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.153532  6669 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:28.064370 11132 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:28.099681 11193 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:28.042588 11245 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:28.208721  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 170067us
W20260709 13:18:28.208972  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 167743us
I20260709 13:18:28.214550  7233 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:28.002012 11226 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.215004  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:28.278425  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:28.172434  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:28.279333  6516 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:28.215813  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:28.210767  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:28.222709  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.225057  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:28.237776  6513 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:28.238287  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:28.059267 10912 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.135s
I20260709 13:18:28.022744 11212 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.239838  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:28.240912  6519 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:28.243064  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.185683 11063 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.246924  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:28.286634  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:28.252343  6593 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:28.183954 11184 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:18:28.246356  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 164956us
I20260709 13:18:28.084022 11207 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.144s
I20260709 13:18:28.234189 10682 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.259903  6594 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:28.301802  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.241295 10988 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:28.253444 11076 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.268227 11095 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.253755 11024 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:28.270159 11046 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.007s
I20260709 13:18:28.203563  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:28.275590  6667 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:18:28.329612  6667 pending_rounds.cc:85] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 5
I20260709 13:18:28.329947  6667 pending_rounds.cc:107] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:18:28.330248  6667 raft_consensus.cc:2889] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:28.277426  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.281279  6663 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
W20260709 13:18:28.281396  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56423.4us
I20260709 13:18:28.277482  6521 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:28.286533  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.030705  7235 rpcz_store.cc:276] 0709 13:18:26.969779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.987361 (+ 17582us) service_pool.cc:224] Handling call
0709 13:18:26.987720 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.133541 (+145821us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:27.896486 (+762945us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.973182 (+ 76696us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.973460 (+   278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:27.973521 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:27.973583 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.017864 (+ 44281us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b1800429b20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419
0709 13:18:28.030122 (+ 12258us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400575dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:28.030167 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.030262 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6535,"prepare.queue_time_us":782749,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":3525,"raft.queue_time_us":185,"raft.run_wall_time_us":8839,"spinlock_wait_cycles":362542848,"thread_start_us":732094,"threads_started":2,"wal-append.queue_time_us":137132}
I20260709 13:18:28.288609  7228 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:28.290292  6596 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:28.272830 11252 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:28.151899 10504 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:28.297986  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59816.2us
I20260709 13:18:28.303124  6665 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:28.234860 11050 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:18:28.273351 10775 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:28.235049 11253 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:18:28.302480 10682 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.290371 10690 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:28.285964 11243 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.313532 11109 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:28.343159  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:28.340108  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.434103 11109 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.436264 11109 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.276993 10850 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.418653 10661 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:28.294950 11054 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.326063 10988 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.255677 11103 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.440122 10988 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:28.440485 11103 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:28.440052 11220 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.437979 11054 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.443210 11054 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.443743 11193 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
I20260709 13:18:28.446503 11212 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:28.380409 10990 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.450600 11245 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.381940  6520 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:28.340224 11046 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.377321  6671 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:28.390064  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.451810  6592 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:28.402187  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:28.452348 11220 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.452663 11046 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:28.396914  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 357300us
I20260709 13:18:28.377489 11024 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:28.393906 11152 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:18:28.435978 11254 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:28.455601  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.425911 11228 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.429250 10690 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.460052 10690 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.463186 11095 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.418591 11196 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:28.345367  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:28.295758 11184 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.295132 10902 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:28.284963 11015 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.378681  6516 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:18:28.450663 10850 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.436784 11109 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.502970  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 47530.1us
W20260709 13:18:28.473260 10703 consensus_meta.cc:287] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.006s
I20260709 13:18:28.532392  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.548151  7229 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:28.470862 10912 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:28.581002 11061 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.005s	sys 0.010s
W20260709 13:18:28.588012  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 55710.7us
I20260709 13:18:28.502405 11103 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:28.602428  6515 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:18:28.605283  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:28.533900 11193 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:28.584344 10934 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:28.629793  6522 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:18:28.493507 10902 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.470532 11243 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.610872 10281 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:28.590881 10912 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:28.636087  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:28.636940  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.651227  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.652709  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.612063 11209 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.619081 11057 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.530995 11109 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.656755  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.620703 11193 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:28.581888 10703 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.690956  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.595379 10856 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.667651  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:28.698611  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:28.700342  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:28.669925 11053 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.626154 11103 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.673573 11107 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:28.681473  7234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1210) took 2077 ms (client timeout 2071 ms). Trace:
I20260709 13:18:28.695564  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:28.733027  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 454629us
W20260709 13:18:28.736523  7234 rpcz_store.cc:269] 0709 13:18:26.603897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.606824 (+  2927us) service_pool.cc:224] Handling call
0709 13:18:28.681259 (+2074435us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540051d1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:28.681394 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3235337472}
I20260709 13:18:28.737862  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:28.702546  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:28.734335  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.675307 11193 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:28.744740  6668 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:28.744329  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:28.670892 11196 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:28.671964 10281 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:28.747977  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:28.759755  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 482340us
I20260709 13:18:28.771428  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:28.686739 11153 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.773559  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:28.779009  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:28.733779 10934 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:28.776656  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 387884us
I20260709 13:18:28.777647  6514 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:28.738786 11237 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:28.771940 11108 consensus_meta.cc:287] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.022s
W20260709 13:18:28.667196  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 346320us
I20260709 13:18:28.770996 11054 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:28.733023 11259 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:28.682952 11057 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:28.782512 11221 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.724s	user 0.017s	sys 0.010s
I20260709 13:18:28.775097 11124 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:28.805071  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:28.776633 11066 consensus_meta.cc:287] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.008s
W20260709 13:18:28.808125  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.761057 11264 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:28.747521 11095 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:28.785496  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:28.787288 11061 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:28.850996  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.835693  6663 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:28.851814  6667 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:28.769841 10777 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:28.840335  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:28.867306  6592 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.806721 11054 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:28.811489 11221 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:28.831339 11209 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:28.777092 11196 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.805640 11253 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:18:28.805316 11108 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.747575 10912 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.848502 10856 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:28.849908  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 463387us
I20260709 13:18:28.876223  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:28.840549 11207 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:28.812363 11124 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:28.881326  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:28.881848 10645 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.877971  6521 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:18:28.883033  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 180563us
I20260709 13:18:28.832185 11015 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:28.773228 11053 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:28.784272 11153 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.900800  6592 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:28.841898 11095 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:28.876009 11269 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:28.909662  6594 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
W20260709 13:18:28.883561 11257 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:18:28.882498 11054 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:28.877655  6588 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.11-10.11]
W20260709 13:18:29.032542  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.906781 11258 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
W20260709 13:18:29.066107  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.844308 11066 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:28.993810  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:28.873034 11242 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:28.985105 11207 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:29.240198 11263 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:18:28.976517 11209 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:29.240993  6516 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.017s	sys 0.043s
W20260709 13:18:29.278864  6516 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.018s	sys 0.043s
W20260709 13:18:29.242084  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.988298 11284 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.991230 11124 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:29.247862  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 254112us
I20260709 13:18:29.250218  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.291373  6667 raft_consensus.cc:2410] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:18:29.256196 11261 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.585s	user 0.001s	sys 0.000s
I20260709 13:18:29.251466  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:29.306105  6671 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 8: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:28.893843 10281 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.012148 11095 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.988409 11270 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:29.292578  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.250671  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.294137  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:29.346743  6667 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:29.348416  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.349689  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:29.006996 10645 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:29.277060 10703 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.013073 10856 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.278s
I20260709 13:18:28.992123 11241 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.241612 11269 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:29.389652 11269 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:29.307862  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 522416us
I20260709 13:18:29.306948  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.270550 11207 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:29.403970  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 109902us
I20260709 13:18:29.404875  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:29.270820 11277 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.317761  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1004) took 1364 ms. Trace:
I20260709 13:18:29.270393 11061 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:29.032130 11275 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.261s
I20260709 13:18:29.271399 11024 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.351830  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.417903  6597 rpcz_store.cc:276] 0709 13:18:27.952864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.955418 (+  2554us) service_pool.cc:224] Handling call
0709 13:18:27.955762 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.971484 (+ 15722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.971514 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.971960 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.307264 (+335304us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:28.307319 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.307411 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.317551 (+1010140us) raft_consensus.cc:1685] finished
0709 13:18:29.317569 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.317694 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28966,"raft.queue_time_us":15941,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5887,"thread_start_us":307425,"threads_started":1,"wal-append.queue_time_us":611024}
W20260709 13:18:29.270328 11265 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.462s	user 0.000s	sys 0.001s
W20260709 13:18:29.374203  6666 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.038s	sys 0.066s
I20260709 13:18:29.291019 11243 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.387864 11241 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:29.388316  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:29.332230 11267 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.589s	user 0.001s	sys 0.000s
W20260709 13:18:29.344833 11278 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 13:18:29.392216  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:29.452188  6669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.005s	sys 0.016s
I20260709 13:18:29.434067  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2254) took 1448 ms. Trace:
I20260709 13:18:29.330538 11279 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.344615 11229 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:29.346331 11270 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:29.436625  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1336) took 1224 ms. Trace:
W20260709 13:18:29.442682  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.045s	sys 0.080s
W20260709 13:18:29.442878  6666 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.038s	sys 0.066s
W20260709 13:18:29.449165  7233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.010s	sys 0.030s
W20260709 13:18:29.505004  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.046s	sys 0.080s
W20260709 13:18:29.505461  7233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.011s	sys 0.030s
W20260709 13:18:29.505563  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 659736us
I20260709 13:18:29.313174 11109 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:29.437973 11260 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:29.404204 11103 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:29.459443  6518 rpcz_store.cc:276] 0709 13:18:27.985311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.985455 (+   144us) service_pool.cc:224] Handling call
0709 13:18:27.997892 (+ 12437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.998334 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.998408 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.017444 (+ 19036us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.048022 (+ 30578us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b18003ef0e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:28.256187 (+208165us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:28.256221 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.256278 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.394181 (+1137903us) raft_consensus.cc:1685] finished
0709 13:18:29.394206 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.433968 (+ 39762us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26766,"raft.queue_time_us":134,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":23892,"spinlock_wait_cycles":13122304,"thread_start_us":207993,"threads_started":1,"wal-append.queue_time_us":401001}
I20260709 13:18:29.504981  7232 rpcz_store.cc:276] 0709 13:18:28.212611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.212739 (+   128us) service_pool.cc:224] Handling call
0709 13:18:28.213046 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.213312 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.213348 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.230709 (+ 17361us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.527186 (+296477us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:28.527222 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.527318 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.436360 (+909042us) raft_consensus.cc:1685] finished
0709 13:18:29.436384 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.436547 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":296192,"threads_started":1,"wal-append.queue_time_us":649778}
I20260709 13:18:29.436121 11061 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:29.452574 11063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.016s	sys 0.009s
I20260709 13:18:29.307489 11284 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.439313 11277 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.444264 11057 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.522897 11057 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.523381 11057 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.523784 10902 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:29.524289 11184 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.031s	user 0.009s	sys 0.001s
I20260709 13:18:29.524664 11184 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.525364 11057 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.526927  7236 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:29.527846 11198 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.528503 11241 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.529124 11184 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.479197 11279 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:29.531731 10902 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:29.533174 11279 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:29.536707 11239 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.538093 10682 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:29.538483 10682 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:29.539265 10682 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.540457 11153 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.540578  6517 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:29.540915  6665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 886) took 1503 ms (client timeout 1605 ms). Trace:
W20260709 13:18:29.541306  6665 rpcz_store.cc:269] 0709 13:18:28.037820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.039020 (+  1200us) service_pool.cc:224] Handling call
0709 13:18:29.540771 (+1501751us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000b3954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:29.540864 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605341568}
I20260709 13:18:29.542431  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.543145  6665 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:29.434854 11296 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:29.417433 11253 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.497902 11132 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.214s	user 0.011s	sys 0.002s
I20260709 13:18:29.473654  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.451663 11281 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.510159  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.483803 11270 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:29.519644 11288 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 13:18:29.548025  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 112398us
W20260709 13:18:29.540459  6520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 886) took 1602 ms (client timeout 1748 ms). Trace:
I20260709 13:18:29.444562 11024 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.563246  6663 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:29.557250  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:29.512519 11063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.016s	sys 0.009s
I20260709 13:18:29.510602 11287 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:29.514288 11061 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:29.603871  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:29.544481 11108 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.517401 11103 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:29.601461  6665 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:29.552946 11301 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.604593  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:29.604959  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:29.577893  6520 rpcz_store.cc:269] 0709 13:18:27.938609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.942421 (+  3812us) service_pool.cc:224] Handling call
0709 13:18:29.540255 (+1597834us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54003a75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:29.540400 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863937920}
I20260709 13:18:29.480569 11209 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:29.609936  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:29.610742  7232 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:29.516077 11196 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.614578 11299 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:29.664680  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 54812.5us
I20260709 13:18:29.591183 11281 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.615949 11106 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.614260  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:29.620100  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2261) took 1301 ms. Trace:
I20260709 13:18:29.620720  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 895) took 1028 ms. Trace:
I20260709 13:18:29.627416  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2253) took 1642 ms. Trace:
I20260709 13:18:29.629621  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:29.484573 11229 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.609599 11053 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.600931 11280 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:29.578373 11193 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.799s	user 0.010s	sys 0.000s
W20260709 13:18:29.635087  7236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1223) took 1578 ms (client timeout 2074 ms). Trace:
I20260709 13:18:29.699724  6513 rpcz_store.cc:276] 0709 13:18:28.592561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.592674 (+   113us) service_pool.cc:224] Handling call
0709 13:18:28.622781 (+ 30107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.623232 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.623271 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.623820 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.891026 (+267206us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:28.891070 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.891159 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.620487 (+729328us) raft_consensus.cc:1685] finished
0709 13:18:29.620513 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.620658 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5585,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2472,"thread_start_us":258526,"threads_started":1,"wal-append.queue_time_us":646236}
W20260709 13:18:29.706473  7236 rpcz_store.cc:269] 0709 13:18:28.209159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.209280 (+   121us) service_pool.cc:224] Handling call
0709 13:18:29.624055 (+1414775us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54007a3d54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:29.635022 (+ 10967us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866968448}
I20260709 13:18:29.637692  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.592945  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:29.647645  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 255472us
I20260709 13:18:29.709357  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.710216  6663 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:29.655405  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 141686us
I20260709 13:18:29.662091  6667 raft_consensus.cc:2376] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11. Re-sending same reply.
I20260709 13:18:29.666926  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 893) took 1075 ms. Trace:
W20260709 13:18:29.667320  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48997.3us
I20260709 13:18:29.663736  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:29.718501  6520 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:29.670322  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 890) took 1450 ms. Trace:
I20260709 13:18:29.718946  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:29.632601 11277 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:29.702337  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 189686us
I20260709 13:18:29.667893 11299 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.729113  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:29.668908  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:29.636472 11066 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.577322 11132 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.700148  6519 rpcz_store.cc:276] 0709 13:18:28.318800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.321922 (+  3122us) service_pool.cc:224] Handling call
0709 13:18:28.322346 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.638680 (+316334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.638724 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.639167 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.855244 (+216077us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:28.855281 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.855347 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.619846 (+764499us) raft_consensus.cc:1685] finished
0709 13:18:29.619875 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.620032 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108530,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":113,"thread_start_us":521621,"threads_started":2,"wal-append.queue_time_us":859532}
I20260709 13:18:29.702201  6517 rpcz_store.cc:276] 0709 13:18:27.984949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.985134 (+   185us) service_pool.cc:224] Handling call
0709 13:18:29.540387 (+1555253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.541168 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.627176 (+ 86008us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54003a75d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:29.627349 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3543778176}
I20260709 13:18:29.655000  7228 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:29.715516  6521 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:29.719928  6665 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:29.718905  6516 rpcz_store.cc:276] 0709 13:18:28.591923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.592107 (+   184us) service_pool.cc:224] Handling call
0709 13:18:28.592500 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.592711 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:29.279406 (+686695us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.666507 (+387101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.666573 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:29.666644 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.666724 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.666864 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":978803,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":115550,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":1036865,"threads_started":3,"wal-append.queue_time_us":242245}
I20260709 13:18:29.571511 11253 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:29.778280  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:29.778697  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 60316.6us
I20260709 13:18:29.719398  6666 rpcz_store.cc:276] 0709 13:18:28.219786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.239356 (+ 19570us) service_pool.cc:224] Handling call
0709 13:18:28.239747 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:28.253182 (+ 13435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.253209 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:28.253763 (+   554us) log.cc:844] Serialized 52 byte log entry
0709 13:18:28.647343 (+393580us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:29.507455 (+860112us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.507670 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.507754 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.530911 (+ 23157us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b18001a9380. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:29.648923 (+118012us) raft_consensus.cc:1685] finished
0709 13:18:29.670109 (+ 21186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.670260 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20956,"prepare.queue_time_us":862750,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":14239,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":27936,"spinlock_wait_cycles":7150848,"thread_start_us":1153548,"threads_started":2,"wal-append.queue_time_us":583489}
W20260709 13:18:29.777958  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 59848.3us
I20260709 13:18:29.730671  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.698999 11076 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:29.786912  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.721940  6671 raft_consensus.cc:2410] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:18:29.790571  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:29.791529  6517 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:29.705410 11272 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.797219  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.797331  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.670714 11108 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:29.798357  6666 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:29.800992  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 132172us
I20260709 13:18:29.699735 11196 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:18:29.720813 11109 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.790063  6663 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:29.718442 11296 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.805662  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:29.713487 10645 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.808039  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:29.807875  6591 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:29.809900  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:29.806865  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:29.802313 11278 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:18:29.777154 11299 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:29.815385  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:29.803402 11277 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.706068 11106 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.669360 11301 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.828965  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1340) took 1604 ms. Trace:
I20260709 13:18:29.811803 11050 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:29.718078 11193 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.729602 11287 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:18:29.813880 11076 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:18:29.813367 11303 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:29.826917 11058 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:29.856673  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 892) took 1054 ms. Trace:
I20260709 13:18:29.857131  6670 rpcz_store.cc:276] 0709 13:18:28.802738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.803093 (+   355us) service_pool.cc:224] Handling call
0709 13:18:28.803493 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.803916 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.803958 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.804479 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.286475 (+481996us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:29.286530 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.286633 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.856452 (+569819us) raft_consensus.cc:1685] finished
0709 13:18:29.856475 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.856607 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":996,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5154,"thread_start_us":481656,"threads_started":1,"wal-append.queue_time_us":911065}
I20260709 13:18:29.857940 11095 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.829926 11103 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:29.848837 10856 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.853075 11054 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.000s
I20260709 13:18:29.814011 11109 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.813525 11312 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:29.859004  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.905800  6670 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:29.838987 11282 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:29.914332 11282 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:29.915350 11282 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:29.860229 11024 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.924043 11107 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.926308  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.927048  6663 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:29.927896  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.928689  6663 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:29.928393 11061 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.929597  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.930450  6663 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:18:29.804772 11272 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.830559 11285 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:29.863397 10690 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.403s	user 0.013s	sys 0.000s
I20260709 13:18:29.865284 10912 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:29.866103 11106 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:29.866897  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1008) took 1912 ms (client timeout 2099 ms). Trace:
W20260709 13:18:29.940701  6593 rpcz_store.cc:269] 0709 13:18:27.954082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.957700 (+  3618us) service_pool.cc:224] Handling call
0709 13:18:29.866724 (+1909024us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540023cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:29.866839 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557704576}
I20260709 13:18:29.869362 10504 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:29.716244 11280 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.847808 11303 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:29.831157 11277 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:29.859858 11304 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 13:18:29.908258  6522 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:29.903692 11054 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.815179 11296 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.917349 11023 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.861467 11281 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.931519  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:29.876410 11301 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.980363  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:29.980996  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:29.981580  7228 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:29.982385  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:29.982954  7232 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:29.987130  7235 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:29.931329 11109 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:29.957927  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:29.902032 10856 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:29.964098  6665 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:29.992120  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:29.949721 11303 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:29.974475  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:29.975495  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.996897  6670 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:29.997714  6664 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:29.992756  6517 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:29.999538  7232 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:30.000468  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:29.976212  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81809.5us
I20260709 13:18:29.977468  6520 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:18:29.971879  6407 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 921c423406da4f289884afca51ca198f (127.5.72.129:34121) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 921c423406da4f289884afca51ca198f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:29.991397  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:29.949179 11253 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:29.940809 10690 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.002460  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.006731  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.006928  6520 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:30.007644  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.002924  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2422) took 1982 ms. Trace:
I20260709 13:18:30.002425  7236 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:29.995688  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.008610  7235 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:30.008210  6596 rpcz_store.cc:276] 0709 13:18:28.020071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.032466 (+ 12395us) service_pool.cc:224] Handling call
0709 13:18:28.152927 (+120461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.386799 (+233872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.996551 (+1609752us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54002fdad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:30.002862 (+  6311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315000576}
I20260709 13:18:30.005669  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:30.014667  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.015268  6520 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:29.995098 10887 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:30.027024 11152 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:30.027318  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.028019  6513 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:30.028721 11242 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:30.029189 11242 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.030826 11240 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:30.031075 11313 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:30.031244 11240 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:30.031678 11313 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:30.032122 11313 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.032239 11240 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.033517 11240 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:29.999055 11132 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.036859  6670 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:18:30.003683 11277 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:30.098145  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.055717  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:30.055791  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.065791  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:30.052130  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 891) took 1823 ms. Trace:
I20260709 13:18:29.977993 11023 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.099483  6669 rpcz_store.cc:276] 0709 13:18:28.228567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.256418 (+ 27851us) service_pool.cc:224] Handling call
0709 13:18:28.271984 (+ 15566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.632299 (+360315us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:28.912448 (+280149us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.915954 (+1003506us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b18003eab80. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a
0709 13:18:29.932613 (+ 16659us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:29.933203 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.933432 (+   229us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:29.933484 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.933576 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.035761 (+102185us) raft_consensus.cc:1685] finished
0709 13:18:30.051323 (+ 15562us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.052063 (+   740us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":818818,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":90810,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":2760,"spinlock_wait_cycles":7325952,"thread_start_us":1162096,"threads_started":3,"wal-append.queue_time_us":631882}
I20260709 13:18:30.101432  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.063386 11280 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.064525  6517 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:30.105340  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.038483  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:30.064541 11207 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.000s
I20260709 13:18:30.108566  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:30.103466  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47815.6us
I20260709 13:18:30.104364 10281 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:18:30.110483  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:30.054744 11057 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.111640  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.106132 11092 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:30.119040  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:30.119658  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.120664  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.095247 11196 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.127173  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.095871 11061 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:30.135262  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.120814 11277 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.098057 11301 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:30.104148 11124 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.759s	user 0.011s	sys 0.001s
I20260709 13:18:30.135883  6596 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:30.104538 11108 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:30.146953  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.171708  6596 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:30.125805 11092 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.133365 11271 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:30.142828 11272 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.186100  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.172350 11156 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.188215  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:30.188400  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 69429.8us
I20260709 13:18:30.126051 11207 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.124805 11280 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:18:30.176548 11193 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:30.134418 11057 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:30.178273  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77904.6us
I20260709 13:18:30.190517  6596 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:30.190575  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.204461  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.146499 11108 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:30.174722 11092 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.148411 11301 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:30.184593 11270 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.575s	user 0.004s	sys 0.006s
I20260709 13:18:30.144011 11284 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.137192 11196 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.184954 11271 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:30.207175  7231 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:18:30.226146  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.226747  7235 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.207517  6662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:30.227773  6662 raft_consensus.cc:606] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:30.228123  6662 raft_consensus.cc:614] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:30.197618  6669 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:30.146041 11124 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.209547 11108 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.211108  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.146435 11277 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.215539  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.202677 11054 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.199209 11156 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.211560 10993 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:18:30.223879 10856 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.210852  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:30.243163  6662 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:30.242300 11061 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.250489 11207 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.251298 11296 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:30.251652 11328 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:18:30.272922 11103 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:30.225517 11095 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.226500 10777 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.295274 10777 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.295634 11313 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.296756  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.237430  6513 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:30.259227  6664 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:30.226729 10690 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:30.262555  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 51793.2us
I20260709 13:18:30.215372 11270 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:30.266275 11330 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:30.236792 11272 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:30.277761 11156 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.278820 11334 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:30.302495 11272 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.303509 11270 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.304026  6518 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:30.279139  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.280795 11124 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:30.283390  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 832447us
I20260709 13:18:30.244192 11024 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:30.306952  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:30.307369  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.284997  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:30.286906  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2127) took 1034 ms. Trace:
I20260709 13:18:30.309235  6667 rpcz_store.cc:276] 0709 13:18:29.252672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.349441 (+ 96769us) service_pool.cc:224] Handling call
0709 13:18:30.286739 (+937298us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54000cba54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.286848 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716963584}
I20260709 13:18:30.310257  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.287276  6671 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:30.288190  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2425) took 2258 ms. Trace:
I20260709 13:18:30.281546  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.288910 11282 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:30.289897  7231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1224) took 2080 ms (client timeout 1694 ms). Trace:
I20260709 13:18:30.291512  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.289937 11023 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.299343  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:30.299300  7236 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:30.300837  6596 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:30.310472  7235 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:30.332188  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.332866  6596 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
W20260709 13:18:30.330106  7231 rpcz_store.cc:269] 0709 13:18:28.209587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.209694 (+   107us) service_pool.cc:224] Handling call
0709 13:18:30.289654 (+2079960us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b540057d854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.289824 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4421503744}
I20260709 13:18:30.333976  6669 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:30.330173  6595 rpcz_store.cc:276] 0709 13:18:28.030261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.030366 (+   105us) service_pool.cc:224] Handling call
0709 13:18:29.870078 (+1839712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.870276 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.870299 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.870341 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:29.870373 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.870426 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.287982 (+417556us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b540023cdd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.288122 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4230243712}
I20260709 13:18:30.337476  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.337915  6667 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:30.342334  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:30.342579  7235 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:30.343029  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.343307  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.343850  7228 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.343927  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.344231  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.343604  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.346331  7237 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:30.345806  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:30.347030  6514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 890) took 2386 ms (client timeout 1592 ms). Trace:
I20260709 13:18:30.347863  6669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:30.349123  6669 raft_consensus.cc:606] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:30.349457  6669 raft_consensus.cc:614] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:30.348579  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.344537  7230 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
W20260709 13:18:30.347419  6514 rpcz_store.cc:269] 0709 13:18:27.960814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.960934 (+   120us) service_pool.cc:224] Handling call
0709 13:18:30.346842 (+2385908us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54003f6bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.346970 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778771200}
I20260709 13:18:30.350533  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.301914 11313 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:30.303005 11334 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.352584  6520 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:30.352885  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.371825  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2595) took 1526 ms. Trace:
I20260709 13:18:30.372256  6592 rpcz_store.cc:276] 0709 13:18:28.845078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.352596 (+1507518us) service_pool.cc:224] Handling call
0709 13:18:30.371748 (+ 19152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:30.373389  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.358017  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.356814  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.358660  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.356824  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:30.355803  7228 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:30.358054  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:30.368660  6667 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:30.368695  6593 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:30.335259 11268 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:30.375097  7234 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:30.381170  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.334167 11331 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:30.383705  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.378331  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.374186  6592 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:30.339841 11156 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:30.386379  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.274909 11061 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.351511 11023 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:30.366483 11329 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:18:30.332074 11270 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:18:30.388202  7228 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:18:30.335685 11024 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.399972  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.404605  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2121) took 1153 ms. Trace:
I20260709 13:18:30.419919  6663 rpcz_store.cc:276] 0709 13:18:29.251075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.939717 (+688642us) service_pool.cc:224] Handling call
0709 13:18:29.953771 (+ 14054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.954032 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:29.954395 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.261935 (+307540us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.262233 (+   298us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.404292 (+142059us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:30.404343 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:30.404386 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.404443 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.404553 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2161,"prepare.queue_time_us":7501,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":96,"raft.queue_time_us":168827,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":13299,"spinlock_wait_cycles":42752,"thread_start_us":446478,"threads_started":3,"wal-append.queue_time_us":237186}
I20260709 13:18:30.339213 11207 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:18:30.404075  6595 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.420051  7234 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:30.417915  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.425132  7228 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:30.426998  7228 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:30.351816 11338 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:30.379436 11334 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:30.359952 11274 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:18:30.434140 11301 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.413231 11240 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:30.405079 11297 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:30.435046 11297 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:30.429821  7236 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
W20260709 13:18:30.429979  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 89674.2us
I20260709 13:18:30.393997 11280 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:30.433404 11268 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:30.434856 11240 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.391748 11110 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:30.445945 11110 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.446370 11110 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.437234  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.417817  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.440112  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.447647  6667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:30.448117  6596 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:30.448220  6667 raft_consensus.cc:606] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:30.448581  6667 raft_consensus.cc:614] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:30.442245  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:30.443279 11335 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:18:30.444788 11240 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.445427  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.459873  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.461787  6667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:30.448523  6592 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:30.369357 11336 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:18:30.464632  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.466032  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.449734  6517 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:30.450276  6595 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:30.345835 11271 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:30.451539  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.449136  6663 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:30.425118 11156 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.457666  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.468084  7228 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:18:30.467908  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.469954  6667 raft_consensus.cc:606] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:30.387156 11242 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.472599  6667 raft_consensus.cc:614] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:30.475832  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.476486  7228 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:30.477484  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.478812  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.473881  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.415495 11023 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.437655 11338 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:30.441859 11268 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:30.481043  7234 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:30.451335 11280 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:30.483670  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.486749  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.485253  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.486790  6592 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:30.466802 11343 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:30.485205  7234 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:30.490468  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.487810  7228 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:30.490722  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:30.482066 11299 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.000s
I20260709 13:18:30.491389  7231 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:30.492347  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:30.493479 10777 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.493860 11282 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.495556 11023 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.482229 11338 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.481891 11156 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.498919 11156 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.501179 11299 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.503057  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.503628  6670 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:30.503705 11299 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.505417  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.506008  6670 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.507962 11268 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.505106 11280 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.509119 11268 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.511574  7234 raft_consensus.cc:2368] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15.
I20260709 13:18:30.521674  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.524391  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.525221  6670 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:30.527781  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.528452  6595 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:30.529230  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.529392  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.529892  6670 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:30.530088  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:30.531008  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.531159  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:30.531657  6595 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:30.469110 11297 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.532280  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.534847  6596 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:30.532680  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.532974  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.536239  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.533308  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:30.536834  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.539856  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.536146  6595 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:30.527995 11299 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.548419 11299 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.550526 11299 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.511554 11343 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.552752 11343 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.552958  6595 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:30.553285 11343 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:30.553700 11343 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.554061 11343 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:30.554821  7229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1225) took 2331 ms (client timeout 2439 ms). Trace:
W20260709 13:18:30.555269  7229 rpcz_store.cc:269] 0709 13:18:28.209956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.210060 (+   104us) service_pool.cc:224] Handling call
0709 13:18:30.540690 (+2330630us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400571554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.540822 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362234112}
I20260709 13:18:30.556340 11024 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.557899  6592 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:30.554867  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.559343  6514 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:30.561221  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.561940  6514 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:30.533892 11023 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:18:30.563405 11023 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:30.563598  6592 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:30.563844 11023 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:30.564761 11023 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.562687  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.566432  6514 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:30.539549 11108 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.567284  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.560909 11092 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.560348  6595 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:30.569067  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.566032 11023 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.569887 11092 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.570994  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.563550 11334 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.571614 11242 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.571936  7228 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:18:30.572841  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.568305  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.574595  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:30.576869  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.537098 11124 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:30.583046  6670 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:30.586046  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:30.574450 11303 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:18:30.540769 11226 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:30.597958  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.599411  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.599961  6521 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:30.574177 11110 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.602344 11110 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.603178  6513 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:30.595113 11124 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:30.604938 11124 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.583148 11338 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.608237  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.609056  6520 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:30.610071  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.584282 11024 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:30.595166 11303 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.619469  6521 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:30.620539  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.621728  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.622988 11212 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:30.623445 11212 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:30.624316 11212 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.625428 11212 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:30.583662 11092 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.588055  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.591053  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1009) took 2636 ms. Trace:
I20260709 13:18:30.627192  6514 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:30.627543  6589 rpcz_store.cc:276] 0709 13:18:27.954359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.959794 (+  5435us) service_pool.cc:224] Handling call
0709 13:18:27.960170 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.619193 (+659023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.619232 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.619722 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.931521 (+311799us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:28.931573 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.590529 (+1658956us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54001b6f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:30.590596 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.590699 (+   103us) raft_consensus.cc:1685] finished
0709 13:18:30.590719 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.590980 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120967,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":91,"spinlock_wait_cycles":813126528,"thread_start_us":563500,"threads_started":2,"wal-append.queue_time_us":1167804}
I20260709 13:18:30.636129  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.639438  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.640190  6514 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
W20260709 13:18:30.648980  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1012) took 2687 ms (client timeout 2203 ms). Trace:
I20260709 13:18:30.652838  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2604) took 1041 ms. Trace:
W20260709 13:18:30.657198  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.657985  7230 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:30.659159  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.660131  6662 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:30.649417  6591 rpcz_store.cc:269] 0709 13:18:27.961916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.004991 (+ 43075us) service_pool.cc:224] Handling call
0709 13:18:30.648811 (+2643820us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b540039f8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.648912 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4088007936}
I20260709 13:18:30.653373  6596 rpcz_store.cc:276] 0709 13:18:29.611462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.535540 (+924078us) service_pool.cc:224] Handling call
0709 13:18:30.571567 (+ 36027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:30.571774 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:30.572088 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.652198 (+ 80110us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.652390 (+   192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:30.652431 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:30.652469 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.652553 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.652686 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":656,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":829,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":154,"thread_start_us":79498,"threads_started":1,"wal-append.queue_time_us":79873}
I20260709 13:18:30.610922 11024 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.613780 11303 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.610608 11338 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.665023  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:30.665210  6668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2515) took 2659 ms (client timeout 1843 ms). Trace:
I20260709 13:18:30.647993 11228 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:30.674223  6668 rpcz_store.cc:269] 0709 13:18:28.000091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.000809 (+   718us) service_pool.cc:224] Handling call
0709 13:18:30.655872 (+2655063us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54000b3454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.656045 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1708268416}
I20260709 13:18:30.675922  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:30.677465  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.678575  6668 raft_consensus.cc:2410] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:18:30.678859  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.679505  6669 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:30.677560  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.681617  6664 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:30.682060  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:30.682813  6662 raft_consensus.cc:2368] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15.
I20260709 13:18:30.682997  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.683506  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.626703 11092 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.681634  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:30.683074  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:30.689800  7230 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:30.689882  6669 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:30.668483 11024 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:30.687093 11340 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.220s	user 0.004s	sys 0.000s
I20260709 13:18:30.689208 11023 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:30.696211 11023 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.696602 11023 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.701448 11092 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.675462 11228 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:30.703372 11228 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.704509 11228 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.709139 11272 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.710028 11229 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.691198 11124 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:30.717368 11343 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.719625 11343 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.721135 10682 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.722826 11124 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:30.723001 11196 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.724733 11124 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:30.727430 11024 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.727537 11023 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.763007  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:30.765688  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 191510us
I20260709 13:18:30.766155  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.775460  6514 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:30.769490  6521 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:30.768180  6520 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:18:30.778457  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.780516  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.781590  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.783488  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.786734  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.787290  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.787724  6517 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:30.784932  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2266) took 2012 ms. Trace:
I20260709 13:18:30.786993  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 897) took 1180 ms. Trace:
I20260709 13:18:30.789942  6518 rpcz_store.cc:276] 0709 13:18:29.606521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.606753 (+   232us) service_pool.cc:224] Handling call
0709 13:18:29.655224 (+ 48471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:30.304395 (+649171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.786474 (+482079us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b5400355054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:30.786837 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491636480}
I20260709 13:18:30.792984  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2258) took 2806 ms. Trace:
I20260709 13:18:30.789016  6522 rpcz_store.cc:276] 0709 13:18:28.771902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.773055 (+  1153us) service_pool.cc:224] Handling call
0709 13:18:28.800593 (+ 27538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.952753 (+1152160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.784315 (+831562us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400044f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:30.784705 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2546424704}
I20260709 13:18:30.781934  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:30.789505  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.793486  6515 rpcz_store.cc:276] 0709 13:18:27.986474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.045158 (+ 58684us) service_pool.cc:224] Handling call
0709 13:18:28.173493 (+128335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.612213 (+438720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.792291 (+2180078us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b5400354b54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:30.792753 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985628928}
I20260709 13:18:30.795595  6514 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:30.667824  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:30.801314  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 133466us
I20260709 13:18:30.794958  6520 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:30.803907  6517 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:30.804229  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:30.813715  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.815461  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.816378  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.816871  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:30.818312  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.819197  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:30.821991  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:30.817535  6514 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:30.827196  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:30.828634  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.832752  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.833432  6596 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:30.833952  6589 raft_consensus.cc:2410] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:30.835479  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.836436  6590 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:30.838099  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.844877  6669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:52622
I20260709 13:18:30.684470  6662 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:18:30.862032  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.863478  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.860651  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.868898 11226 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:30.869269 11268 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:30.869516 11023 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.868152  6588 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:30.859120  6669 raft_consensus.cc:606] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:18:30.875304 11230 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:30.875607  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.875917 11230 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.877214  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.878894  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.729722 11024 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.880298 11224 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:18:30.876397 11230 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:30.887179  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:30.888469  6590 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:30.889467  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2526) took 1160 ms. Trace:
I20260709 13:18:30.890131  6665 rpcz_store.cc:276] 0709 13:18:29.728658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.790631 (+ 61973us) service_pool.cc:224] Handling call
0709 13:18:30.861928 (+1071297us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000e5bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.889161 (+ 27233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381504896}
I20260709 13:18:30.891039 10856 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:30.891497  6594 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:30.892297  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:30.892661  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 359255us
I20260709 13:18:30.863480 10682 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:30.896229  6595 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:18:30.896649  6595 pending_rounds.cc:85] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 8
I20260709 13:18:30.897008  6595 pending_rounds.cc:107] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:18:30.897284  6595 raft_consensus.cc:2889] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:30.896222 11156 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:30.898188 11156 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.906013 11061 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:30.907819  6596 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:30.909502  7231 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:30.911121 11108 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.911805 11109 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:30.892343 11212 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:18:30.916225 11212 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:30.916785 11212 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:30.924237 11277 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.927845 11287 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:30.928399 11287 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:30.933817 11156 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.860431  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.935421  6592 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:30.935776 11224 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:30.887240  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:30.894419  6597 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:30.939184  6669 raft_consensus.cc:614] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:30.946988 11212 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.949148 11212 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:30.949395  6637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 1.949s (SENT) [suppressed 26 similar messages]
I20260709 13:18:30.952538 11331 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.953353 11331 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:30.954671  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2599) took 1383 ms (client timeout 1553 ms). Trace:
I20260709 13:18:30.954988 11196 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:30.955263  6593 rpcz_store.cc:269] 0709 13:18:29.571343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.947756 (+376413us) service_pool.cc:224] Handling call
0709 13:18:30.954186 (+1006430us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54002fe4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.954442 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868467584}
I20260709 13:18:30.956013 11196 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:30.957755  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2438) took 1448 ms (client timeout 1507 ms). Trace:
I20260709 13:18:30.936331 11224 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:30.958232  6594 rpcz_store.cc:269] 0709 13:18:29.510412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.629144 (+118732us) service_pool.cc:224] Handling call
0709 13:18:30.957487 (+1328343us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400210a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.957673 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636524928}
I20260709 13:18:30.959429 11224 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:30.961596 11224 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.961894 11196 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.967200  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:30.971843 11268 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:30.972036 11057 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:30.972352 11268 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:30.972828 11268 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.943953  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:30.975387 11281 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:30.975632  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 59547.2us
I20260709 13:18:29.835215  7233 rpcz_store.cc:276] 0709 13:18:28.224295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.287444 (+ 63149us) service_pool.cc:224] Handling call
0709 13:18:28.287724 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.287961 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:28.312395 (+ 24434us) log.cc:844] Serialized 14 byte log entry
0709 13:18:28.397014 (+ 84619us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b180046dda0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:29.827624 (+1430610us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.828602 (+   978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.828659 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:29.828706 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.828783 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.828903 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":679,"prepare.queue_time_us":24338,"prepare.run_cpu_time_us":573,"prepare.run_wall_time_us":38169,"raft.queue_time_us":30651,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":5965,"spinlock_wait_cycles":10214784,"thread_start_us":1429926,"threads_started":3,"wal-append.queue_time_us":1291177}
W20260709 13:18:30.979130 11325 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.878s	user 0.005s	sys 0.000s
I20260709 13:18:30.981724  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 899) took 1362 ms. Trace:
I20260709 13:18:30.982162  6519 rpcz_store.cc:276] 0709 13:18:29.618913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.785404 (+166491us) service_pool.cc:224] Handling call
0709 13:18:29.785734 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:29.786081 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.786118 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:29.797208 (+ 11090us) log.cc:844] Serialized 52 byte log entry
0709 13:18:29.934529 (+137321us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:29.934568 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.934647 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.978225 (+1043578us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:30.981073 (+  2848us) raft_consensus.cc:1685] finished
0709 13:18:30.981101 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.981498 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1736,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":16732,"spinlock_wait_cycles":5649280,"thread_start_us":104194,"threads_started":1,"wal-append.queue_time_us":187321}
I20260709 13:18:30.985400 11334 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:30.985559  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:30.995944  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2133) took 1722 ms. Trace:
I20260709 13:18:30.996479  6671 rpcz_store.cc:276] 0709 13:18:29.273289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.993877 (+720588us) service_pool.cc:224] Handling call
0709 13:18:30.286976 (+293099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:30.329947 (+ 42971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.995406 (+665459us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54000cba50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:30.995692 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673448192}
I20260709 13:18:31.010569  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:31.012418  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51521.2us
I20260709 13:18:30.930169 11287 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.021276  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:31.008680 11230 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.033267 11230 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:31.037887  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.013242  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.029628  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:31.019973 11303 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:18:31.054155 11230 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:31.060359 11230 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.060475 11303 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:31.060997 11230 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.061107 11303 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:31.063462  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48858.4us
I20260709 13:18:31.063035  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:31.075510  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:31.082787  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.084214  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:31.084376 11343 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:31.084923 11343 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:31.085502 11343 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.085525 11124 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:31.086225 11124 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:31.086730  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.086835 11303 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.086720 11124 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.088574 11303 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.090704  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.091112  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:31.093568  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:31.094683  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:31.101053 11341 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:31.101591 11341 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:31.102003 11341 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.102279 11277 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:31.102974  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 281238us
I20260709 13:18:31.103080 11277 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.104099 11124 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.104673  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.105762 11277 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:31.106406 11124 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:31.106930 11198 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:31.072535 11287 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.112902 11341 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.072919  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:31.115736  6671 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:31.115559  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:31.116958 11124 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:31.117177  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:31.117530 11124 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:31.118182 11124 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:31.118208  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.121855  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.123433  6665 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:31.124392  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.124408  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:31.125365  6665 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:31.125871 11124 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.126353  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.122610  6671 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:31.127002 11277 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:31.127702  6668 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:31.136700  6663 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:31.134210  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.139197  6517 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:31.153314  6666 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:18:31.154284  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 80158.2us
I20260709 13:18:31.137261 11277 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.157128 11334 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.143070  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:31.159916  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.160605 10777 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:31.161799 10504 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:31.165737 11277 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:31.174170 11320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.006s	sys 0.018s
W20260709 13:18:31.174979 11320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.007s	sys 0.018s
I20260709 13:18:31.187094  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.168547 11212 raft_consensus.cc:2764] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:18:31.205495  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:31.211488  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48344.1us
W20260709 13:18:31.216149  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.217460  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.219586 11297 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:31.201180 11341 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:31.223248  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 64555.5us
I20260709 13:18:31.226112 11196 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:31.226643 11196 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.227027 11196 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.227921 11024 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:18:31.234216 11331 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.234773  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.236582  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:31.237352  7233 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:31.201864 11334 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.239989 11334 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:31.241773  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.210100 10777 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.245316 10777 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:31.221728  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:31.252782  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49745.7us
I20260709 13:18:31.246279  6668 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
W20260709 13:18:31.263824  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53370.7us
I20260709 13:18:31.246835 11297 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.266326  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:31.274935  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 67055.6us
I20260709 13:18:31.283389  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:31.285631  6590 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:31.273335 11297 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.291010  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:31.286322 10777 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:31.295663  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.296494 11299 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:31.297230 11299 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.296393 11316 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:18:31.297782 11299 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.301811  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:31.300500  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:31.292204 11341 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:31.305891  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:31.293687 11124 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:31.310911  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:31.310706 11331 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:31.311856 11061 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:31.312309 11061 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:31.311964  6594 raft_consensus.cc:2410] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:31.300539 10777 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.319945  6590 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:31.322835  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:31.323408  6593 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.317227  6562 leader_election.cc:400] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15.
I20260709 13:18:31.324079  6562 leader_election.cc:403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:18:31.322474 11061 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.303951 11230 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:31.326845  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 105624us
I20260709 13:18:31.320739  6597 raft_consensus.cc:2376] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10. Re-sending same reply.
I20260709 13:18:31.318512 11224 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:31.316762 10777 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:31.317418 11341 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:31.337160  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:31.343338  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:31.348326  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.338819 11316 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:31.350890  6486 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:31.353061  6590 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.341835 11224 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:31.345057 11061 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.359160 11346 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:31.368418 11240 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:31.374899  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.376109 11240 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:31.374897 11224 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.379168  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:31.380059 11212 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:31.380452 11346 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.380491 11212 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:31.387599 11341 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.387889 11212 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.395643  6516 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:31.398006  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:31.398598  6516 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:31.399243 11198 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:31.399681 11198 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:31.400079 11240 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:31.394680  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.400086 11198 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.401662 11303 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:31.402684 11108 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:31.402928  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:31.403165 11108 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.403641 11108 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.404362  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:31.405313 11212 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.406839 10504 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:31.407244 10504 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:31.407163 11303 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:31.407662 10504 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.407847 11303 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:31.408812 11303 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.410332 11303 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.410889 11103 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:31.412036  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:31.412149 11242 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:31.412670 11108 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:31.415113  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73978.8us
I20260709 13:18:31.423148 10856 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:31.425237 10504 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.395383 11343 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.427785  6516 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:31.429584  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:31.430744  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.432125  6519 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:31.418248 11198 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.434680  6594 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:31.435590  6597 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:31.433830  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.437579 11346 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.503430 11198 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:31.504148 11341 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:31.438130  6588 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:31.504711 11341 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.505266 10504 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.506940 11341 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:31.438395 11268 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.440848  6596 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:31.507649  6596 pending_rounds.cc:85] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 8
I20260709 13:18:31.507893  6596 pending_rounds.cc:107] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:31.508102  6596 raft_consensus.cc:2889] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:31.447424 11240 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:31.509037 11268 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.449846 11226 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:31.422432 11277 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.452032  6561 leader_election.cc:400] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15.
W20260709 13:18:31.511556  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 59704.6us
I20260709 13:18:31.511605 11226 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:31.512115 11226 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:31.513003  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:31.513159 11226 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.479233  6593 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:31.476065  6590 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:31.492594  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.518677 11268 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:31.492498  6420 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 3 to 9, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:31.519146  6663 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:18:31.519572  6663 pending_rounds.cc:85] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 9
I20260709 13:18:31.519798  6663 pending_rounds.cc:107] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:18:31.494410  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:31.520906  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 87151.4us
I20260709 13:18:31.496048  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:31.521211 11351 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.522142  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.520021  6420 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 5 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.527465 11268 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:31.528061  6420 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 5 to 6, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 6 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:31.532075 11226 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:31.534108  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.545855  6420 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 5 to 6, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 6 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:31.551434 11230 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.519152  6667 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:31.497370  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.497699  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.460417  6592 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:31.514917  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:31.516166 11287 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:31.516711  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:31.516989 11277 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:31.517179 11299 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.494841  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:31.564910  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48943.3us
I20260709 13:18:31.520043  6663 raft_consensus.cc:2889] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:31.568625  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:31.572635  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50634.2us
I20260709 13:18:31.498597 10856 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:31.578521  6671 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
W20260709 13:18:31.578742  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 166851us
I20260709 13:18:31.579666 11242 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:31.580156  6665 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:31.581804 11299 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:31.582837 10504 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.583364 11108 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:31.583536  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 179265us
I20260709 13:18:31.564214 11287 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:31.591565  7200 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:31.591748  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77130.7us
I20260709 13:18:31.565672 11346 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:31.584478 11277 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:31.586831 11230 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:31.601713 11124 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:31.602197 11124 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:31.602569 11370 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.254s	user 0.002s	sys 0.002s
I20260709 13:18:31.609194  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.616377 10777 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.602649 11124 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.600885 11230 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.603040 11184 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:31.605904 11277 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:31.623831 11351 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:31.627863  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 280709us
I20260709 13:18:31.601145 11196 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.625363 11230 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:31.634493  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.638684 11343 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:31.638841  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:31.639495 11343 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.639901 11376 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:31.640079  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:31.640498 11376 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:31.641611  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.641538 11376 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.641989 10856 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.643553 10856 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.641736 11196 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.625654 11184 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:31.647008 11184 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.646884 11226 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:31.648859 11103 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:31.648924 11108 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:31.649559 11103 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:31.650856 11103 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.652949 11024 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:31.620054 11346 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.607597 11287 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:31.656329  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 87827.4us
I20260709 13:18:31.662611  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:31.662827  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62501.4us
I20260709 13:18:31.657169 10504 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 13:18:31.671859  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50942.6us
W20260709 13:18:31.671888  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 99743.3us
I20260709 13:18:31.623348 11297 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:31.673636  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.672878  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.673493 11343 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:31.667390 10777 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.684726  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:31.687924  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:31.688557  6662 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:31.673480 11316 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15.
I20260709 13:18:31.667889 11303 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:18:31.692894 10504 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:31.702283  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:31.683879 11196 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:18:31.706995  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 52273.1us
I20260709 13:18:31.709045  6522 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.701457 11226 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:31.710891 11226 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:31.707989  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:31.716540 11226 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.718600 11346 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:31.721686 11297 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:31.723197  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49165.7us
I20260709 13:18:31.724056 11196 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:31.724776 11196 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:31.685285 11277 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.744073 11380 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:18:31.734808 11061 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:31.735319 11226 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.708245 11230 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:31.753554  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.751492  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:31.740600  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:31.757000  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.747344 11346 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.753965 11061 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:31.761101 11375 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.103s	user 0.002s	sys 0.000s
I20260709 13:18:31.726423 11196 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.789830  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:31.780870  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 78701.2us
I20260709 13:18:31.758543 11242 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:31.761780 10504 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:18:31.794600 11373 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.122s	user 0.002s	sys 0.000s
I20260709 13:18:31.798974  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:31.799612  6663 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.765344 11376 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:31.802058  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:31.802834  6669 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:31.805640  6522 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:31.806419  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:31.805435  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 102857us
I20260709 13:18:31.776751 11379 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:31.813205 10777 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:31.813593 11379 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.813851 10777 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.815837 10777 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:31.816525 11184 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.819643 11230 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.823235  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:31.823354 11230 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.801369 10504 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:31.824554  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 98003.3us
I20260709 13:18:31.825939  6486 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:31.826689  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 78805us
I20260709 13:18:31.823968 11230 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.804787 11376 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.834921 11152 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.835804 11152 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.838315 11152 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.835057 11242 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:31.835008  7235 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:31.840310  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.851280  7233 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:31.851003 10504 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.842689 11299 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:18:31.852916 11299 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:31.845971 11061 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.861447  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.862646  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:31.862735 11242 raft_consensus.cc:2764] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:31.863837 11242 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:31.863996  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:31.864391 11242 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:31.864969 11242 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.867273  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.867933 11152 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.868119 11103 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:31.869596 11230 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.870273  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:31.870893  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 70545.4us
I20260709 13:18:31.871022  6519 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:31.868642  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.832417 11376 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.868283 11287 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:31.842199 11184 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:31.861510 11061 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.902245 11103 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:18:31.904343  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.904948  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:31.907510  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.908674 11124 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.917042 11242 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:31.923945 11379 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:31.924722  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 64754us
I20260709 13:18:31.925418  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:31.943055  7233 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:31.938941  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75008.4us
I20260709 13:18:31.953320  6516 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:31.953464  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:31.955261  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.956641  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.957844  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.957962 11024 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:31.900506 11297 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.963635  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.964669  7233 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:31.965915  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:31.966275  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 97726.1us
I20260709 13:18:31.966575  7236 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.967444  7200 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:31.969393 11184 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:31.973603  7236 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:31.976292 10504 raft_consensus.cc:2764] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:18:31.981040  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:31.981184  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:31.958803 11343 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.986500  6663 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:31.989071  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:31.991451 11240 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:18:31.985905 11381 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:31.992359 11240 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:31.992733 11240 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:31.992985 11103 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:18:31.993276 11152 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:31.993513 11240 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:31.994405 11196 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.987887 11379 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:18:31.989904 11303 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.002552  6669 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:31.999768 11240 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.006290 11124 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.004107  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.007297  7231 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.005863 11287 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:31.999773 11379 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:32.009100  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:31.983973 11226 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:32.012392 11379 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:32.015082 11346 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:32.018033  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.020709 11242 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.017455 11024 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:32.003144 11303 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.026961 11303 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.029989 11226 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:32.044598  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53460.2us
I20260709 13:18:32.013697 11379 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:32.047601  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 66628.3us
I20260709 13:18:32.052014  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:32.000422 11380 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:18:32.055190  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.056957  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:32.057853  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 52383.3us
I20260709 13:18:32.026484 11376 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.060962 11376 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:32.061708  7235 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.058513 11341 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.068924  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.016485 11381 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:18:32.047154 11297 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:32.069739  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 168066us
I20260709 13:18:32.031307 11226 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:32.068706  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.074956  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.076467  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.076735  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.076735 10856 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:32.077590  6591 raft_consensus.cc:2410] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:18:32.077487 10856 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.078194 11184 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.078774 11184 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.080295  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.080909  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.081652  7235 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:18:32.084173  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.085417  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:32.083868 11379 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.087299  6663 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:32.071481 11196 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.089386  7236 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 8: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.091496  6596 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:32.093206  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.093731  6669 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:32.097999  6663 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:18:32.098354  6663 pending_rounds.cc:85] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
I20260709 13:18:32.098547  6663 pending_rounds.cc:107] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:18:32.099251  6663 raft_consensus.cc:2889] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:32.103150 11152 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.103591 11152 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.104039 11152 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.105326  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.105463 11346 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:32.105917 11346 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:32.106678  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.106786 11346 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:32.083195 10856 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.085934 11376 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:32.110509  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.111727 11184 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:32.116499  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.112888  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:32.113502  6591 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:18:32.114049  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.125828  6597 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.114799 11381 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:32.115259 11380 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:32.141927 11380 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:32.142017 11376 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:32.142423 11380 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.126220  6589 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:32.142596 11376 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.145524 11376 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:32.115877  7236 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:32.147514  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.116417  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.117358  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.148223  7236 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.117571  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.149821  7230 raft_consensus.cc:2368] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:32.118875  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.119124  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.151916  7234 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:18:32.118280  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.153040  7237 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:32.105043  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:32.156895 11303 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:32.157547 11380 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.123293  7235 raft_consensus.cc:2368] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:32.124874  6592 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:32.125430  7229 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:32.126021  6667 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:32.132155  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46782.1us
I20260709 13:18:32.132931  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.169791  6597 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
W20260709 13:18:32.171497  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 63732.5us
I20260709 13:18:32.184471  7231 raft_consensus.cc:2410] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:18:32.137091  6669 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:32.111694 11346 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:32.149209  7236 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:32.150728  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.150797  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.192881  6596 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:18:32.185609  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.168224 11230 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.169370 11381 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:32.189136  7235 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:18:32.189352  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.187700  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.202229  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.202546  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.205665  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:32.206005  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.214035  6670 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:32.214488  7228 raft_consensus.cc:2368] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:32.215266  7229 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:32.215554  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:32.216403 10856 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:32.216250 11268 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:32.216715  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.216889 10856 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:32.218005  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.219027  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.219355  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.219606  6662 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:32.219684  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.221275  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.222154  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:32.217923 10856 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.222769  6668 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 7.
I20260709 13:18:32.224126  6671 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:32.215394  7237 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:32.230944 11303 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:32.232836 11404 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.233716  6519 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:32.233963  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.234442 11380 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.234750  6517 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:32.193912 11240 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:32.234826  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.235214  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.235697 11240 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:32.236181 11240 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:32.236621  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.237037 11240 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.240020 11403 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.240540 11403 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.241585 11381 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.242512  6522 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:32.243448  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:32.219359  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.245451  6664 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:32.215178  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 82326.1us
I20260709 13:18:32.246414  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.247008  6669 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:32.233417 11404 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:32.237620 11398 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.188s	user 0.000s	sys 0.003s
I20260709 13:18:32.251200  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:32.231612  6662 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:32.245924  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.252094  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.254405  6670 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:32.189654 11152 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.259291  6663 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:18:32.245020 11403 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:32.264200  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76590.8us
I20260709 13:18:32.252259 11287 consensus_queue.cc:1243] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:18:32.266081  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 49442.5us
I20260709 13:18:32.265404  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.269296 10777 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:32.269480  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.269843 10777 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:32.270378 11334 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.270810 11334 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.271157 11334 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.271098  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.272217  7237 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:32.274916  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 53726.9us
I20260709 13:18:32.275701  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.276890  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.277837 11341 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:32.278023  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.278609 11341 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:32.253412 11277 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:32.253744 11124 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.257820 11380 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:32.279408 11334 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.280197 11380 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:32.280438 11196 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:32.280081 10777 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.280709 11380 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:32.280992 11196 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:32.281746 11334 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:32.281888 11196 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.253630 11379 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:32.254908 11299 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:32.257483 11024 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.259318 11395 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:32.285336  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 67395.1us
I20260709 13:18:32.285813  7233 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.295078  6664 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:32.297274  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:32.300606  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.295017 11376 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.316320  7235 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:32.320472 11346 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:32.321347 11346 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:32.313496 11061 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:32.322757  6415 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 332) took 16876 ms (client timeout 15000 ms). Trace:
I20260709 13:18:32.323129 11346 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:32.326153  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:32.330951  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.331147  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.331584  6522 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.297937 11299 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:32.304877 11024 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:32.336727  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.340794  6666 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:32.348217  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.348685  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.349364  6590 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:32.307279 11395 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:32.307894 11379 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:32.325518 11124 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:32.359328  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.365334  6591 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:32.307485 11408 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:32.368045  6522 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:32.335675 11299 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:32.384380  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 65783.3us
I20260709 13:18:32.386830  6597 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:32.394356  6636 leader_election.cc:400] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:32.394809  6636 leader_election.cc:403] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:18:32.323184 11061 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.355892 11403 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:32.419531  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:32.342567 11376 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.379844 11277 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:32.411530  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.428300  6636 leader_election.cc:400] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:32.423933  7235 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:32.408565 11268 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:32.383425 11124 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:32.430419  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.406512 11240 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.434216  7235 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:18:32.437727  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 140013us
I20260709 13:18:32.441273  6593 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:32.416078 11024 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:32.445604 11124 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.393590 11343 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:18:32.450906 11061 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:32.433079 11341 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.442741 11240 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.468457 11303 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:32.323184  6415 rpcz_store.cc:269] 0709 13:18:15.421156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.426771 (+  5615us) service_pool.cc:224] Handling call
0709 13:18:32.297148 (+16870377us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:27.723281 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:27.724782 (+  1501us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:27.724869 (+    87us) write_op.cc:435] Acquired schema lock
0709 13:18:27.724895 (+    26us) tablet.cc:662] Decoding operations
0709 13:18:27.873782 (+148887us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:27.882840 (+  9058us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:27.892499 (+  9659us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:18:27.914684 (+ 22185us) tablet.cc:701] Row locks acquired
0709 13:18:27.914741 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:18:27.914935 (+   194us) write_op.cc:270] Start()
0709 13:18:27.915093 (+   158us) write_op.cc:276] Timestamp: P: 1783603107914861 usec, L: 0
0709 13:18:27.915135 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:18:27.915829 (+   694us) log.cc:844] Serialized 8169 byte log entry
0709 13:18:29.446021 (+1530192us) op_driver.cc:464] REPLICATION: finished
0709 13:18:29.864714 (+418693us) write_op.cc:301] APPLY: starting
0709 13:18:29.864818 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:30.271375 (+406557us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:30.271501 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:30.284972 (+ 13471us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b18003e1040. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:30.506453 (+221481us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:30.550839 (+ 44386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 13:18:30.550904 (+    65us) write_op.cc:312] APPLY: finished
0709 13:18:30.551387 (+   483us) log.cc:844] Serialized 188 byte log entry
0709 13:18:30.982238 (+430851us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b
0709 13:18:31.175662 (+193424us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:31.176305 (+   643us) write_op.cc:454] Released schema lock
0709 13:18:31.176443 (+   138us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:31.207512 (+ 31069us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:31.210308 (+  2796us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4053775,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":384922,"apply.run_cpu_time_us":54497,"apply.run_wall_time_us":1312293,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91865,"num_ops":21,"prepare.queue_time_us":366957,"prepare.run_cpu_time_us":40397,"prepare.run_wall_time_us":524970,"raft.queue_time_us":937079,"raft.run_cpu_time_us":115395,"raft.run_wall_time_us":615974,"replication_time_us":1457871,"spinlock_wait_cycles":47041280,"thread_start_us":1906596,"threads_started":6,"wal-append.queue_time_us":1022237,"wal-append.run_cpu_time_us":35327,"wal-append.run_wall_time_us":1484791}]]}
I20260709 13:18:32.471038 10777 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:32.487866  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68442.8us
I20260709 13:18:32.486732 11419 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:32.523133  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.523984  6520 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:32.512019 11395 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.525013 10777 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.527137 10777 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:32.516606 11303 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:32.492552  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.493229  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.494006  7236 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:32.495064  6522 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:18:32.530855  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.531584  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.531610  6520 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.533030 11242 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:32.533428 11242 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:32.534735  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.538669  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.538879  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.539314  6593 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:32.539489  6596 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:32.496083  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.540241  7237 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:32.540503  7235 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.540428 11410 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.496985  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.497812  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.541136 11410 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:32.490481  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62357.4us
I20260709 13:18:32.542011  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:32.542551  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:32.498411  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 59738.5us
I20260709 13:18:32.543380 11410 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.543560  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:32.544308  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.495004  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.542582  6665 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:32.531183  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.545373  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.546530  7229 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.546973  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:32.553671  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:32.559020 11410 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.559784 11410 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.563055  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.563279 11303 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.564399  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.564682 11303 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.565001  6596 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:32.565950  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.566896  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.562579 11410 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.567509  6519 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:32.567873 11381 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:32.562598  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.569309 11240 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.511564 11341 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.571049  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.574086  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.574697  6515 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:32.539600 11316 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.570261 11341 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.586891  6518 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:32.435329 11268 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.576660  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.589313  7237 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:32.588575  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.577029  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:32.577311  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.577616  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.580684  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.572338  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.566483  6596 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:32.587560  6595 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:18:32.591235  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.588538  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.606132  6520 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:32.611016  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.612133  6513 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:32.614495  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.615454  6522 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:32.605239  7229 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:32.617060  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.571614 11287 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:32.619973  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 77044.4us
I20260709 13:18:32.580193 11061 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:32.574826 11403 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:32.626204  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:32.624359  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47425.9us
I20260709 13:18:32.632633  6515 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:32.634524  6636 leader_election.cc:400] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:32.577322 11404 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.616845 11401 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:32.638770  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:32.580386 11103 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 13:18:32.643272  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65728.8us
I20260709 13:18:32.620157 11381 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:32.654364  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.616317 11196 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:32.631735 10504 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:32.663556  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 120062us
I20260709 13:18:32.594560 11316 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.626926 11287 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:32.632938 11061 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.636310 11379 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:32.631971 11403 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:32.664419  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.663672 10504 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.641539 11401 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:32.675520  6589 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 8.
I20260709 13:18:32.671021 11061 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.679548 11152 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:32.675449 10504 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.684533 11299 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.688158 11277 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:32.689174 11277 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:32.692637  6515 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:18:32.695036 11376 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:32.698020  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 71123.6us
I20260709 13:18:32.699331  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:32.700383 11268 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.701361 11316 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.702304 11401 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:32.703462 11401 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:32.668272 11287 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.708814  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.709671  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.711089 11230 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.711546 11230 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.711975 11230 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.671416 11395 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:18:32.688195  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.721662  6595 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.724258  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:32.725530  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.674698 11403 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:32.720649 11316 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:32.748811  6519 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:32.750161  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.751005  7233 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:32.728782  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:32.756248  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.757025  6597 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:32.758149  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.758826 11196 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:32.759305 11196 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:32.759661 11196 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.748307 11316 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.763398 11316 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.764014 10777 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:32.764552 10777 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:32.765578 10777 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.767597 11426 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:32.768201  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:32.768388 11426 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:32.769582 11426 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.772042  6591 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:32.773515  6597 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:32.773895  6591 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:18:32.775022 11184 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:32.691814 11380 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.793136  6562 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:32.727715 11376 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.809482  6517 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:32.810266  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 153194us
I20260709 13:18:32.754326 11152 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:18:32.754552 11277 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:18:32.807403 11240 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:32.812511 10777 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.790694 11184 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:18:32.835707 11240 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.842742 11240 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:32.851316  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 45889.7us
I20260709 13:18:32.817137 11424 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:32.858717 11184 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.818778 11379 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:32.862136 11277 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:32.838160 11427 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:18:32.854302 11343 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:32.867993  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.877279 11433 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:32.877435 11432 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:18:32.873034  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:32.854146  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 189751us
W20260709 13:18:32.857270  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 157990us
I20260709 13:18:32.864794 11299 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.899428 11430 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:18:32.893041 10504 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.902933 11108 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:32.905792 11380 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:32.906180 11415 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.912077  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:32.913371  6594 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: [12.8-14.9]
W20260709 13:18:32.923349  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.885422 11277 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:32.930080  6520 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:32.930410 11277 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.931521  6517 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:32.929296 11426 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:18:32.893811 11424 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:32.927167  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:32.922598  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56028.1us
I20260709 13:18:32.884172 11184 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:32.949980  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:32.956805  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.922991 11108 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:32.965708  6591 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:32.928879 11435 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:32.921895 11427 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:32.969185  7233 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:32.940635 11299 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.972227  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:32.974852  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 56892.2us
I20260709 13:18:32.994661  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:32.994732  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:32.996163  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:32.996594  7229 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:32.996867  6520 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:32.968597 11124 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:33.038537  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 127602us
I20260709 13:18:32.921989 11316 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.055364  6589 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:18:33.078369  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.080515  6670 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:18:33.091598  6589 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:33.111874  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:32.943871 11415 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.100214  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:33.116642  6670 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:33.121712  6595 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:33.135354  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:33.136013  6517 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:33.116734  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:33.127553  6663 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:33.039011 11230 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.101395 11103 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:33.190418  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:33.200090  6663 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:33.128120 11426 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:33.047910 11124 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.076900 11061 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.197921  6664 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:33.085259 11435 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.141847 11441 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.218400 11242 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.219866 11242 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:33.226375 11401 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.523s	user 0.002s	sys 0.005s
I20260709 13:18:33.238271  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:33.229898 11240 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.190534 11421 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:33.176863 11438 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:33.231571 11440 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:18:33.231986 11441 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:33.236270 11341 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.616s	user 0.005s	sys 0.006s
I20260709 13:18:33.236883 11124 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:33.243115  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 293201us
I20260709 13:18:33.261190  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:33.190609 11427 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.202831 11452 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.239434 10856 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.266086  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:33.269115 11417 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:33.269543 11417 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:33.269913 11417 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:33.270725 10777 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:33.084806 11424 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.276324 11316 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.277779 11415 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.278666  6517 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:18:33.279023  6517 pending_rounds.cc:85] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 10
I20260709 13:18:33.279256  6517 pending_rounds.cc:107] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:18:33.279495  6517 raft_consensus.cc:2889] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:33.279414 11427 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.179148 11108 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.281673 11427 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.264639 11341 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.282750 11240 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.282874  6665 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:33.284286 11341 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.259346 11421 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:33.287533 11421 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:33.288851 11443 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.095s	user 0.002s	sys 0.000s
W20260709 13:18:33.291301 11196 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.002s
I20260709 13:18:33.291694 11196 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.254989 11438 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.293668 11196 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:33.293856 11438 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.294256 11196 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:33.295233  7230 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:33.295609 11061 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:33.212610 11426 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.297112  7230 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:33.298038 11426 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.267078 11103 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:33.298758  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:33.299093 11103 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.299503 11103 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:33.301059 11395 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:33.301419 11395 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:33.259601 11287 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.553s	user 0.003s	sys 0.007s
I20260709 13:18:33.303011 11287 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.304065  6516 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:33.304693  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:33.305402  7230 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:33.305052 10856 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.267668  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:33.306336 11287 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.281703 11108 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:33.307072 10856 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.308851 11441 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.309233  6670 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:33.302209 11395 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.306505  6664 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:33.311419  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.314224  6592 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
W20260709 13:18:33.316326  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.318033  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:33.318296  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:33.318049  7237 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:33.318933  6517 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:33.312292 10504 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:33.320724  6514 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:33.323114  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:33.323799  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:33.327347  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:33.326543  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:33.328377  7230 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:33.317925 11287 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.329293  7228 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
W20260709 13:18:33.334596  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.324970 11395 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:33.336196  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 297984us
I20260709 13:18:33.337080  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:33.339740  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:33.319658 11196 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:33.320582 11441 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:18:33.335749 11184 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:33.336246 11316 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.351046 11316 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.333276 11415 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:33.260035 11435 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.354709  6663 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:33.356989  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.324842 11438 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:18:33.355770  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:33.357643  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:33.359946  6514 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:33.361349  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:33.365576 11196 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:33.367707  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 437644us
I20260709 13:18:33.369524 11299 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:33.372674 11442 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 13:18:33.335528 11287 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.375217 11287 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.376590 10856 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:18:33.379842 11024 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:33.380282 11024 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:33.342535 11401 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:33.380283  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:33.312173 11452 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:33.372562 11403 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.001s
I20260709 13:18:33.386991  6666 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:33.385286  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:33.357426  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:33.358278 11415 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.396191  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:33.393122 11403 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.398837 11403 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.399451  6520 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:33.400753 11403 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:33.401618 11403 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:33.402535 11403 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:33.406045  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48693.6us
I20260709 13:18:33.408840 11287 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.409339  6514 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:33.411772  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:33.414755  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.414690  6513 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:33.382090 11240 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:33.417508  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:33.385185 11424 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:33.422601  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:33.424837  6517 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:33.425952  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:33.394982 11426 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:33.431188  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:33.394840 11415 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:33.429551  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:33.433524  6670 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:33.435134  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:33.389580 11343 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:18:33.419548 11240 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:33.458562  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.428128 11316 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.381102 11024 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.431946 11303 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:33.414759 11287 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.483367  6670 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:33.492946  6666 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:33.463084 11240 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:33.433605 11410 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:18:33.479142 11438 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:33.541213  6665 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:33.433786 11408 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:18:33.478725 11401 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.547487  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:33.484356 11464 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:33.558514  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:33.446132 11449 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 13:18:33.539553 11196 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.537843 11230 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:33.577261  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 154767us
I20260709 13:18:33.491360 11417 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:33.587723  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 157718us
I20260709 13:18:33.493372 11341 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:18:33.564781 11426 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:33.556962 11438 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:33.559790 11403 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:33.591447  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:33.595137  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:33.597436  6592 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:33.594458  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:33.581799 11404 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:33.589323 11464 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:33.555547 11401 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.610705 11468 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:33.617931 11476 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:33.607771  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 106902us
I20260709 13:18:33.603583 11341 consensus_queue.cc:1243] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:18:33.588341 11196 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.614540 11403 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:33.603825 11404 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:33.606070 11435 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:33.649063  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:33.649426  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:33.649870  6668 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:33.622843 11452 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.651815  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:33.652168 11475 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:33.654765  6671 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:18:33.626837 11476 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.632699 11468 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:33.686803 11457 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
I20260709 13:18:33.689059 11467 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:18:33.690193 11467 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:33.690577 11467 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:33.691704 11467 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.693531 11467 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:33.694283 11459 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:18:33.695551 11417 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.687331 11103 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.697079  7233 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:33.653976 11438 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.756834 11417 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:33.759979 11417 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:33.761798 11463 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:18:33.762315 11417 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.760130 11466 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.137s
W20260709 13:18:33.763106 11460 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:18:33.763366 11417 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:33.763899 11417 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.702152 11458 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.196s
I20260709 13:18:33.766510 11417 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.665983 11447 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.200s
I20260709 13:18:33.667351 11303 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.667689 11475 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.797814  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 940) took 1128 ms. Trace:
I20260709 13:18:33.798333  6516 rpcz_store.cc:276] 0709 13:18:32.669073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.670786 (+  1713us) service_pool.cc:224] Handling call
0709 13:18:32.678524 (+  7738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.304421 (+625897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.797368 (+492947us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b540031fbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:33.797587 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438179200}
I20260709 13:18:33.654095 11426 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:33.815675  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 257200us
I20260709 13:18:33.818400  6668 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:33.797576 11287 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.791553 11303 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.832038  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:33.830386  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:33.837956 11401 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.837971  6513 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:33.844092 11303 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.816830 11447 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:33.847719 11447 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.849736 11447 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.850647 11196 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.851326 11287 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:33.851727 11287 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:33.852494 11287 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.863513  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:33.791885 11404 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.867307  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:33.868402  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:33.868989  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.870549  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.873247  6595 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:18:33.867926  6596 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:33.873703  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:33.878219  6594 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:33.872918 11401 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:18:33.862947 11240 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.795308 11468 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.880766 11103 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.889345  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:33.889971  6592 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:33.893793 11478 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.894556  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:33.894419 11478 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.895246  6595 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:33.897163 11478 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.858529 11415 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:33.919752  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 488605us
I20260709 13:18:33.922921  6590 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 13:18:33.906982 11303 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:33.891955 11403 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:33.936666 11429 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.059s	user 0.013s	sys 0.008s
I20260709 13:18:33.941519 11431 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:33.942340  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1045) took 1401 ms. Trace:
I20260709 13:18:33.946825 11476 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.921968 11478 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.908752 11408 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:33.918370 11483 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:33.952478  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75451.9us
I20260709 13:18:33.974200  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:33.974833  6516 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:33.977417 11483 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:33.978161 11478 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.978433 11483 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:33.980564 11478 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.965476 11415 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.917377 11240 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.969796  6593 rpcz_store.cc:276] 0709 13:18:32.540834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.540962 (+   128us) service_pool.cc:224] Handling call
0709 13:18:32.541225 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.655843 (+114618us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:32.672495 (+ 16652us) log.cc:844] Serialized 14 byte log entry
0709 13:18:32.881924 (+209429us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:32.881989 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:32.882045 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.942045 (+1060000us) spinlock_profiling.cc:243] Waited 91 ms on lock 0x7b5400210f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:33.942096 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.942249 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16502,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":19120,"raft.queue_time_us":9656,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4231,"spinlock_wait_cycles":234316288,"thread_start_us":183775,"threads_started":2,"wal-append.queue_time_us":151694}
I20260709 13:18:33.948880 11303 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:33.994670 11475 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:33.983695 11458 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.996152 11458 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:33.996693  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.997411 11469 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.353s	user 0.000s	sys 0.006s
I20260709 13:18:33.998934 11470 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:33.999902 11471 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:33.998997 11458 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:33.984035  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:33.986732  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.002104 11381 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:34.002471 11381 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.002812 11381 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:34.003180  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.003784  6668 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:33.969591 11431 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:33.985503 11242 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:34.000363 11471 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:18:34.002812  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 81383.8us
I20260709 13:18:33.999603 11470 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.006934  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:34.006470  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:34.028100  6488 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:34.005431 11431 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.018051  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.029480  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:34.019032 11403 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.021771  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:34.030381  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:34.033157  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.041394  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.042016  6517 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:34.039691  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:34.034960  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:34.050384  6671 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:34.034554  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.052491  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:34.053097  6517 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:34.053325  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.030454  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.055140  6516 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:34.055295  6562 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.056691 11438 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:34.057178 11438 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.057567 11438 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.058892 11108 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:34.059265 11418 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.059638 11475 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:34.060082 11475 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:34.059782 11418 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.060500 11475 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.064348  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.063313  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:34.081172  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:34.081799  6664 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.038880 11196 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.040715 11242 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.085181  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.047503 11458 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:33.981266 11408 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.062022 11418 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.117404  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.062300 11432 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:34.122453  6517 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:34.122408  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:34.050130 11426 raft_consensus.cc:2764] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:34.125583  6670 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:34.152858  6671 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:34.056996 11424 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:18:34.090337 11483 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:34.120246 11401 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:34.118630 11480 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:34.134902 11485 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:34.092794 11303 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.140347 11470 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.141448 11108 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.144441 11418 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.145735  6590 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:34.146353  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:34.203058  6593 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:34.157425  6664 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:34.157913  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.090859 11452 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:34.188150  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.165832 11432 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:34.195978 11439 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.000s	user 0.007s	sys 0.002s
I20260709 13:18:34.152688 11438 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.114924 11196 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.199501 11435 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.241501 11435 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.244822 11435 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.212574 11108 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.248816 11108 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:34.252321  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.253223 11446 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.253760 11446 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.255975 11446 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.260919  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:34.062456 11473 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 13:18:34.165679 11341 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:34.213231  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:34.180204 11494 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:34.270056  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 188254us
I20260709 13:18:34.212446 11418 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.214715  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:34.271696  6666 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:34.205668 11299 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.214910 11432 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:34.186530 11483 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.217353  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.198192 11485 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.124238 11458 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.197142 11486 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.124828 11103 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:18:34.204566  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.308979  6671 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:34.232313 11196 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:34.233520  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.235587 11438 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.239168  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.240688 11404 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.253240  6670 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:34.276808 11242 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:34.219264 11401 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.228637 11408 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.319497  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.200217 11480 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:34.320165  6666 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:34.317353  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:34.321121  6671 raft_consensus.cc:2376] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9. Re-sending same reply.
I20260709 13:18:34.319540  6521 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:34.324293 11418 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.324918 11401 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:34.315459  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54606.3us
I20260709 13:18:34.303638 11494 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.303995 11483 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.322098  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.305305 11485 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:34.308236 11458 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:34.334648  6664 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:18:34.336062 11103 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:34.304782 11299 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:34.338829 11196 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:18:34.343163 11470 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:34.345007 11449 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:34.262688 11381 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:34.331831 11501 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.366394 11501 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.334100 11480 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:34.349479  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.349974  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:34.346935  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 75276.8us
I20260709 13:18:34.354494  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1061) took 1577 ms. Trace:
I20260709 13:18:34.352679 11242 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.355814 11479 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:34.379227  6667 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:34.372282 11452 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:34.355784  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:34.316254 11486 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:34.356642 11497 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:34.357046 11494 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:34.357728 11471 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.359529 11299 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:34.278644 11341 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.363849  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:34.364117  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.302755 11424 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:34.379036  6597 rpcz_store.cc:276] 0709 13:18:32.777549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.777673 (+   124us) service_pool.cc:224] Handling call
0709 13:18:32.805102 (+ 27429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:32.805426 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:32.805451 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:32.806286 (+   835us) log.cc:844] Serialized 26 byte log entry
0709 13:18:32.897270 (+ 90984us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:32.897312 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.897375 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.195273 (+1297898us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:34.352382 (+157109us) raft_consensus.cc:1685] finished
0709 13:18:34.352404 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.354422 (+  2018us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4692,"spinlock_wait_cycles":8972416,"thread_start_us":73612,"threads_started":1,"wal-append.queue_time_us":232527}
W20260709 13:18:34.407255  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 51536.5us
I20260709 13:18:34.395649  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.408806  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:34.416718  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 109994us
I20260709 13:18:34.420050  6664 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:34.422734  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.392174 11501 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.432325  6519 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:34.432717  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:34.435921  6588 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:34.434463  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.439007  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:34.439354  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.440130  6515 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:34.445338  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.438781  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:34.447506  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52062.9us
I20260709 13:18:34.449481  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:34.452741  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:34.453145  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.452548  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:34.453653  6594 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:34.449131  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:34.454846  7229 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:34.457284  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.461117  7233 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:34.461301  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:34.461790  6666 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:34.462069  6595 raft_consensus.cc:2410] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:34.462236  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:34.464638  6589 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:34.462894  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:34.463021  6666 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:18:34.418193 11430 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:34.465077  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:34.465708  6592 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:34.463683  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2315) took 1252 ms. Trace:
I20260709 13:18:34.419224 11480 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:18:34.469452  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.461169  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:34.464298  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:34.474255  6519 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:34.422362 11108 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:34.479030 11108 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:34.480279 11108 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.481704 11108 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:34.484236  6590 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:34.496119  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.473094  6520 rpcz_store.cc:276] 0709 13:18:33.211224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.211419 (+   195us) service_pool.cc:224] Handling call
0709 13:18:33.215474 (+  4055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.399828 (+184354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.463475 (+1063647us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540014d7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:34.463630 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422705408}
I20260709 13:18:34.497630  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:34.496814  7237 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:34.498683  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.499701  7231 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:34.518772  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.473524 11446 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:34.400017 11242 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.520853  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.521543  6667 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:34.523124  6519 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:34.529866  7237 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:18:34.530287  6671 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:34.438645 11441 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:34.525884  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 199745us
I20260709 13:18:34.466117 11452 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:34.466578 11299 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:34.529103  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:34.538081  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.539057  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.543871  6519 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:34.405987 11479 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.546032  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.470860 11471 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.472667 11430 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.548640  7236 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:34.547066  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:34.502647 11480 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:34.447978 11424 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:34.531270 11429 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:18:34.577260  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.577766  7233 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:34.516294 11475 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.550086 11441 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:34.627421 11297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.003s	sys 0.040s
I20260709 13:18:34.671743  6519 raft_consensus.cc:2368] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:34.665257  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:34.665102 11483 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.600479  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.630057  7232 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:34.666288  6663 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:34.667872 11485 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:34.674105  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.668205 11424 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.550511 11196 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.668483 11108 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:34.628365  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.669494 11430 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.647154  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.684844  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.722344  6519 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:34.627233  7229 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:34.625231  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:34.670606 11490 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:34.724069  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:34.675009  7236 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 8.
I20260709 13:18:34.727739  7233 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.656328 11479 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.698167  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.730224  6520 raft_consensus.cc:2368] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11.
I20260709 13:18:34.731060  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.731344  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.669685 11242 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.674708 11475 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.662022 11341 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:34.672027 11297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.004s	sys 0.042s
I20260709 13:18:34.556070 11452 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:34.654417 11486 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:34.725595  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:34.731974  7232 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:34.732256  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.732903  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.730674  7234 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:34.736812 11242 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:34.735288 10856 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:34.742570 10856 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:34.743027 10856 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:34.743635 11441 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:34.744922  6513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2604) took 1319 ms (client timeout 1560 ms). Trace:
W20260709 13:18:34.745296  6513 rpcz_store.cc:269] 0709 13:18:33.425588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.425719 (+   131us) service_pool.cc:224] Handling call
0709 13:18:34.744723 (+1319004us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54003f75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:34.744839 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939234048}
I20260709 13:18:34.749233 11483 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.750816 11485 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:34.751189 11452 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:34.660981 11467 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.905s	user 0.006s	sys 0.005s
I20260709 13:18:34.698256 11103 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.552784 11381 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:34.547906 11495 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.281s
I20260709 13:18:34.738895  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.742035 11506 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.728978 11108 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:34.753743  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 196573us
I20260709 13:18:34.759495  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.732039 11446 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:34.761914  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.598922 11507 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.672453 11480 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:34.761890  7233 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:34.770579  7229 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:34.775615  7232 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:34.738032 11196 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.774724  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:34.819162  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44524.7us
I20260709 13:18:34.820075  6562 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.821955  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:34.822202  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.822892  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.821955  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.823475  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:34.823954  7233 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:34.742053 11242 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.826117 11408 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:34.839577 11480 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.842051  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.844269  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.844456  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.845314  7236 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:18:34.849321 11277 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:34.850198 11434 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.984s	user 0.005s	sys 0.005s
I20260709 13:18:34.778230 11495 consensus_queue.cc:1243] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:34.762476 11103 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.856627  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:34.775332 11467 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:34.854799  7228 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:34.858501  6588 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:34.781306 11503 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.766784 11506 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.827692 11441 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:34.869037  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.840909 11426 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:34.850942 11475 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:34.873711  7235 raft_consensus.cc:2410] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:18:34.877910  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 95551.9us
I20260709 13:18:34.862875 11467 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:34.783916 11507 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.872398 11441 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.894387  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:34.901269  6637 leader_election.cc:400] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:34.901634  6637 leader_election.cc:403] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:34.861667 11512 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:18:34.905079 11424 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:34.879230 11518 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:34.906036  7233 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:34.907449  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:34.883313 11467 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:34.910930  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:34.912585  7230 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:34.893780 11441 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:34.889428 11522 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:34.913065  7230 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:34.916923  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.911762 11473 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:34.923334 11465 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.483s	user 0.004s	sys 0.006s
I20260709 13:18:34.925428 11242 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:34.926874 11108 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:34.897688 11343 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:34.870852 11503 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.963171  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:34.963213 11473 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:34.965147  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:34.967389 11408 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:34.968055 11451 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:34.878301 11426 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.962559  7230 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.906134 11505 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:34.993433  7230 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.994647 11520 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.995210 11520 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.997375 11520 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:34.998644  7229 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:34.999697 11503 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.004420 11024 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.887689 11446 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.006713 11430 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:34.929613 11424 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:34.962798 11522 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.009452 11497 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:35.011407 11196 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:34.970867 11343 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:35.011909 11517 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:18:34.987241 11473 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:34.989894 11451 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:35.022405 11108 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:35.025701 10856 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.027155 11506 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.029450  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.008411 11505 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.967770 11475 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:35.020350 11522 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:35.045619 11458 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.000s
I20260709 13:18:35.073571  7237 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:35.033520 11196 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:34.928650 11518 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.036695 11430 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.045181 11103 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:35.063192 11108 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.081565  6662 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:35.042917 11507 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:35.067790 11505 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:35.085911 11464 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.086397 11464 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.091461 11277 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.091990 11277 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.096169  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.097093  6521 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:18:35.097224  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 67834.4us
I20260709 13:18:35.097918  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.098515 11528 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:35.099519 11528 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:35.100455  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:35.100750 11417 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:35.101398 11417 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.103418 11108 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:35.030468 11497 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.108873 11497 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.037622 11502 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:18:35.111980 11475 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.116427 11417 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:35.120347  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.119998  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:35.122962  6668 raft_consensus.cc:2376] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12. Re-sending same reply.
I20260709 13:18:35.123780  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:35.122960  7230 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:18:35.124421  7230 pending_rounds.cc:85] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 9
I20260709 13:18:35.124656  7230 pending_rounds.cc:107] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:18:35.124899  7230 raft_consensus.cc:2889] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.123927 11103 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:18:35.126981 11513 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:18:35.127390 11103 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:35.128324 11343 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.128257 11103 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.129887 10856 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.130496  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:35.130767 11529 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:35.131237  6668 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:35.131340 11529 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.133518 11529 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.134043 11525 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.134544 11525 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.136646 11525 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.080401 11458 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.144408  6516 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:35.145519 11451 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.146186  6670 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:35.146512 11458 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.143538 11426 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.147632 11441 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.148208  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:35.146821 11451 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:35.151126 11341 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:35.151641 11341 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.153558 11341 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.154043 11341 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:35.154165 11494 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.154639 11494 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.155014  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:35.083894 11408 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.155298 11467 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.156755 11436 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.157250 11436 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.157907 11501 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:35.159265 11436 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.149582 11441 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.174685  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.160804  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.163946  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.176146  6518 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:35.177086  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.177671  6518 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:35.178938  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:35.179855 11533 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:35.180359 11533 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.165760  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.180922 11533 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:35.181124  6513 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:35.094485 11277 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.159654  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.182629  6597 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:35.182854  6593 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:35.183758  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.184360  6590 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:35.183774  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.186444  6591 raft_consensus.cc:2368] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:35.165360  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.187737  6517 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:35.192410  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:35.194450  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.195047  7235 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:35.195376  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.195922  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.196873  7235 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:35.202313  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.204003  6522 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:35.203441  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.205325  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.206064  6520 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:35.180250  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.209292  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:35.191241 11506 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:35.231684  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 83526.5us
W20260709 13:18:35.231707  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 72098.2us
I20260709 13:18:35.232805  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.233134  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:35.246827  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:35.208202  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:35.275214  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.276007  6588 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:35.191625 11430 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.279773  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.270504  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.285202  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:35.285567  7230 raft_consensus.cc:2368] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:35.285701  7235 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:35.270562  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:35.264070  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 55937.5us
I20260709 13:18:35.279935  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.190905 11447 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:35.280988  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.296046  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.296638  6589 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:35.154505 11341 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.263026  7229 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:35.270081  7200 leader_election.cc:400] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11.
I20260709 13:18:35.299423  7200 leader_election.cc:403] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:18:35.300385  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.300894  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.282226  6588 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:35.301678  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.301873  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:35.270304  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 106091us
I20260709 13:18:35.302464  6520 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:35.302948  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:35.303151  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 70782us
I20260709 13:18:35.289570  6597 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:35.295284  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:35.301285  6594 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:35.305651  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.303998  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:35.306354  6594 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:35.263226 11410 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:35.307166  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.202314 11503 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 13:18:35.265678 11403 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:35.368607  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:35.369132  6594 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:35.356740  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:35.311853  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46021.6us
I20260709 13:18:35.369818  7230 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:35.370188  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.314761  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.371981  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.372296  6519 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:35.374022  6519 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:18:35.374462  6519 pending_rounds.cc:85] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 8
I20260709 13:18:35.373219  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:35.264223 11427 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:35.355876  6595 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:35.376169  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.377197  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.378161  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.360208  6596 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:35.319252  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:35.329516  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.361495  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 57553.7us
I20260709 13:18:35.379598  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:35.380558  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:35.380991  6596 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:35.302806 11124 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.384313 11124 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.386695 11525 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.387216 11525 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.390940  6595 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:35.269949 11441 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:35.268635 11424 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:35.265748 11513 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:35.362893  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.202131 11512 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:18:35.269869 11103 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.277910 11196 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:35.356017 11502 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:35.365186 11474 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.374768  6519 pending_rounds.cc:107] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:18:35.401921  6519 raft_consensus.cc:2889] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.402769  6597 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:35.362175  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.278113 11455 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.297435 11497 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:35.262393 11526 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.167s
I20260709 13:18:35.404942  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.270761 11467 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.406385  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.397156  6594 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:35.307608 11447 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.416833 11447 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.416960 11431 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:35.308027 11532 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:18:35.401473 11452 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:35.408608 11497 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:35.419786 11497 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:35.420181 11287 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:35.405706 11103 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.422259 11103 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:35.426898  6636 leader_election.cc:400] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:35.427217  6636 leader_election.cc:403] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:35.409080  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:35.413002  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 93833.6us
I20260709 13:18:35.428517  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.359745 11507 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:35.428845  6596 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:35.308809 11475 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:18:35.283989 11480 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 13:18:35.406773  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.391892 11403 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.386519 11124 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:35.432179  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.435470  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.478521  6515 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:35.382445 11522 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.484428 11522 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.475806  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:35.485456  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.485591  6590 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:35.486886  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54781.8us
I20260709 13:18:35.487847  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.407294 11525 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.437496  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.391752 11410 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:35.494930 11410 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.411733 10856 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:35.412076 11424 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:18:35.499046 11539 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
I20260709 13:18:35.498922 11424 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.433524 11494 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.438725  6596 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:35.491968  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:35.503782 11341 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.507807 11108 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.510176  7231 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:35.515878  7235 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:35.517050  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.517925  7235 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:35.518932  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:35.412426 11196 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:35.301281 11430 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.493255  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.425974 11408 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:35.520340 10856 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:35.536574 11108 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.412268 11474 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.536461 10856 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.538651 11474 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:35.479610 11495 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:35.539409 11495 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.541530 11495 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:35.493211 11475 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:35.545678 11505 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:35.548609  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 121831us
I20260709 13:18:35.429991 11441 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:35.436056 11303 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:35.491153 11403 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:35.557268  6519 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: [13.13-13.13]
W20260709 13:18:35.552335  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 59131.6us
I20260709 13:18:35.414667 11455 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.559949  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.562355  6665 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: [12.12-14.13]
I20260709 13:18:35.565397  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:35.569682  7230 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:35.571448  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:35.573350 11475 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:35.574779 11467 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:35.575162 11230 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:35.576022 11230 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:35.583369  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.521782 11522 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:35.484704 11533 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.556975 11024 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:35.605299  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.612960  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.613723  7235 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:35.552394 11525 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:35.556027 11529 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:35.617468  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.566488 11426 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:35.619714  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.624574  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:35.623459  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.608355 11435 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:18:35.615995 11477 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:35.615881 11316 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:35.559073 11408 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:35.632083  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.632037  6637 leader_election.cc:400] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:35.622586 11534 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:35.628902 11230 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:35.622540 11537 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:35.642937  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.619150 11502 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:18:35.645015  6596 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:35.645514  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.664417  6593 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:18:35.622944 11427 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:35.645566  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.628254 11525 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:35.636499  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 232911us
I20260709 13:18:35.671737  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:35.652012 11536 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:18:35.672721  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.674273  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.655890  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:35.660483  7229 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:35.675788  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:35.676374  7229 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:35.664258  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.677776  7235 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:35.667204  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.669018  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.670228  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.678764  6596 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:35.679603  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.650867  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:35.681469  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.682036  6596 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 8.
I20260709 13:18:35.682428  6518 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:35.683907  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.685948  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:35.686714 11495 raft_consensus.cc:2764] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:18:35.687650  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.688271 11495 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:18:35.663194 11451 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.003s
I20260709 13:18:35.690340  6563 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:35.691525  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.694032  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:35.663471 11230 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.663154 11316 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:35.765231  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:35.765789  6522 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:35.766543  6515 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:35.697297  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:35.640214 11518 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:35.867877  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 88151.3us
I20260709 13:18:35.695523  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:35.667135 11427 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:35.770871  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 73632.7us
I20260709 13:18:35.776731  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:34.944207  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:35.869174  6518 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:35.869097  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:35.646987 11341 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:18:35.673780 11449 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:35.882261  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 119146us
I20260709 13:18:35.774487 11230 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:35.892524 11495 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:35.905560  6518 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:35.905835  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:35.699404 11522 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.208s
I20260709 13:18:35.699627 11471 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.208s
I20260709 13:18:35.909198 11430 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:35.699939 11424 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:35.638685 11441 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:35.931129  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:35.953419  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 177418us
I20260709 13:18:35.701809 11547 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.695077 11529 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:35.700430 11480 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:35.942440  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:35.944249  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:36.016767  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72604.5us
I20260709 13:18:35.938546  7229 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:35.952877  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 47104.5us
I20260709 13:18:36.020432  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:35.955013  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.956291  6593 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:36.022996  6593 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:35.882212 11501 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:35.868127 11534 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:35.918046 11427 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:35.665095 11196 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.103529  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1391) took 1447 ms. Trace:
I20260709 13:18:35.943027 11522 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:18:36.064500  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44168.8us
I20260709 13:18:36.113821  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:36.101135  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:36.086356 11527 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:36.114845 11527 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.115231 11527 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:36.115505  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.937965 11495 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.116271  6589 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:36.116449 11495 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.117344 11526 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:36.104056  7233 rpcz_store.cc:276] 0709 13:18:34.656206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.090186 (+433980us) service_pool.cc:224] Handling call
0709 13:18:36.102892 (+1012706us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:18:36.103060 (+   168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.103225 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.103249 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.103289 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:36.103320 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.103371 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.103476 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160462080}
I20260709 13:18:36.123555  7233 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:36.107120  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:35.700603 11537 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.305s
W20260709 13:18:36.114421 11497 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.694s	user 0.007s	sys 0.003s
I20260709 13:18:36.125588 11497 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.127712 11497 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.128216 11497 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:36.128662 11497 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.130775  6669 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:36.132397  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:36.130470 11497 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:36.133886 11497 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:36.134362 11497 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.136476 11495 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.138456 11495 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:35.918320 11494 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.156s
I20260709 13:18:35.962083 11410 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.002486 11518 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:35.952665 11424 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:36.074249 11482 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:36.057986 11480 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:36.056362 11501 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:36.114065 11507 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.000s
W20260709 13:18:36.176712  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 480680us
W20260709 13:18:36.187588 11546 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 13:18:35.960152 11446 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.209s
I20260709 13:18:36.188666  7233 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
W20260709 13:18:36.189028 11477 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.000s
I20260709 13:18:36.189286 11518 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.189512 11477 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:36.189814 11518 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.190349  6592 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:36.102589 11525 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.194303 11408 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.003875 11433 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.211s
W20260709 13:18:36.186414 11431 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.002s
I20260709 13:18:36.168015 11502 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.176236 11424 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:36.208456  6664 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2187) took 2065 ms (client timeout 1659 ms). Trace:
I20260709 13:18:36.207952  6671 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:36.056670 11547 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.181234 11501 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:36.177857 11480 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:36.213490  6664 rpcz_store.cc:269] 0709 13:18:34.143341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.212975 (+ 69634us) service_pool.cc:224] Handling call
0709 13:18:36.208239 (+1995264us) spinlock_profiling.cc:243] Waited 69.4 ms on lock 0x7b540005b254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:36.208369 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159669632}
W20260709 13:18:36.198918 11403 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.606s	user 0.004s	sys 0.004s
I20260709 13:18:36.102178 11430 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
W20260709 13:18:36.291843  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.200395 11507 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11.
I20260709 13:18:36.303447  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:36.306952  7229 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:36.174322 11410 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.215679 11124 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.200635 11497 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.312978  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:36.308009 11473 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.236s	user 0.010s	sys 0.000s
I20260709 13:18:36.319103  6664 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:36.300539 11403 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.351742  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:36.350991  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.362326  6664 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:36.363250  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.363874  6664 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:36.364399  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 323010us
I20260709 13:18:36.364548  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:36.365078  6664 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:18:36.366035  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:36.374884  6664 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:36.200166 11316 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.392064  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.313489 11471 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:36.374334 11543 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.711s	user 0.001s	sys 0.000s
I20260709 13:18:36.404882  6664 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:36.315182 11507 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
W20260709 13:18:36.412791  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 244200us
I20260709 13:18:36.413416  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.415027  6514 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:36.411029  7198 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.749s (SENT) [suppressed 9 similar messages]
I20260709 13:18:36.314553 11124 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:36.405128 11024 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.782s	user 0.008s	sys 0.001s
I20260709 13:18:36.418887  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2329) took 2350 ms. Trace:
I20260709 13:18:36.423705  6516 rpcz_store.cc:276] 0709 13:18:34.063997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.064111 (+   114us) service_pool.cc:224] Handling call
0709 13:18:36.414589 (+2350478us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540003f554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:36.414745 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482027904}
I20260709 13:18:36.400866 11497 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.218273 11502 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.422466  6589 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:36.405354 11431 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.488544 11431 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.467847  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:36.430888 11024 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:18:36.430094 11230 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:36.470793  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 179695us
I20260709 13:18:36.412078 11471 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:36.482151 11507 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:36.477773 11527 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.487952 11403 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:36.433260 11559 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:36.480647  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 952) took 1223 ms. Trace:
I20260709 13:18:36.373715 11473 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.522351  6663 rpcz_store.cc:276] 0709 13:18:35.249606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.249726 (+   120us) service_pool.cc:224] Handling call
0709 13:18:35.278055 (+ 28329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.362415 (+ 84360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.362455 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.364667 (+  2212us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.558457 (+193790us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:35.558494 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.558562 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.472312 (+913750us) raft_consensus.cc:1685] finished
0709 13:18:36.472331 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.472477 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81562,"raft.queue_time_us":80578,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2075,"spinlock_wait_cycles":145664,"thread_start_us":193371,"threads_started":1,"wal-append.queue_time_us":257863}
I20260709 13:18:36.505259  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:36.487368 11558 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:18:36.480191 11482 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:36.527889  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:36.531888  6597 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:36.507644 11471 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:36.534260  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:36.513235 11507 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.471045 11497 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.502604 11431 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.566090  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:36.563973  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:36.569236  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:36.565320  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 199347us
I20260709 13:18:36.575137 11446 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:36.578271  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 166995us
I20260709 13:18:36.580533  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:36.533916 11433 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:36.587211  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.588096  6663 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:36.588580  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2581) took 1164 ms. Trace:
I20260709 13:18:36.588905  6663 rpcz_store.cc:276] 0709 13:18:35.423954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.586932 (+1162978us) service_pool.cc:224] Handling call
0709 13:18:36.588519 (+  1587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.589818  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.564221 11507 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:36.571946 11431 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.595871  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2183) took 2449 ms. Trace:
I20260709 13:18:36.596841  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 960) took 1175 ms. Trace:
I20260709 13:18:36.597461  6663 rpcz_store.cc:276] 0709 13:18:35.420917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.589562 (+1168645us) service_pool.cc:224] Handling call
0709 13:18:36.596774 (+  7212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.598649  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:36.602346  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:36.603710  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2579) took 1175 ms. Trace:
I20260709 13:18:36.576543 11446 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:36.596364  6666 rpcz_store.cc:276] 0709 13:18:34.142151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.465879 (+323728us) service_pool.cc:224] Handling call
0709 13:18:34.466140 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.470859 (+  4719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.470889 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.471313 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:18:34.683474 (+212161us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:36.468420 (+1784946us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.591111 (+122691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.591189 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.591227 (+    38us) raft_consensus.cc:1685] finished
0709 13:18:36.591243 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.591346 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1739481,"prepare.queue_time_us":1807883,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":6478,"raft.queue_time_us":5024,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":308791,"threads_started":2,"wal-append.queue_time_us":254808,"wal-append.run_cpu_time_us":1507,"wal-append.run_wall_time_us":1013467}
I20260709 13:18:36.587805 11547 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:36.568152 11403 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:18:36.604102  6663 rpcz_store.cc:276] 0709 13:18:35.423439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.598401 (+1174962us) service_pool.cc:224] Handling call
0709 13:18:36.599276 (+   875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.561707 11562 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:18:36.531919 11559 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:36.610986  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.003s	sys 0.012s
I20260709 13:18:36.614228  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2566) took 2372 ms. Trace:
I20260709 13:18:36.592906 11277 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:36.577169 11575 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 13:18:36.621069  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.004s	sys 0.012s
I20260709 13:18:36.624965  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:36.626799  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62924.4us
I20260709 13:18:36.629451  7232 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:36.600144 11410 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:36.619675 11343 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:36.606101 11433 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:36.610445 11446 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:36.608455 11563 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.621423  6667 rpcz_store.cc:276] 0709 13:18:34.241526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.522117 (+280591us) service_pool.cc:224] Handling call
0709 13:18:34.541326 (+ 19209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.541573 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:34.553470 (+ 11897us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.414923 (+1861453us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b18001b1f00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:18:36.415095 (+   172us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.611612 (+196517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.611685 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:36.611744 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.614066 (+  2322us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.614176 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1783201,"prepare.queue_time_us":1880839,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":186,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":26948,"spinlock_wait_cycles":50697088,"thread_start_us":196170,"threads_started":1}
I20260709 13:18:36.604348 11431 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.650416 11563 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.653378  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.604502 11570 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:18:36.655689  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.633906 11523 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:36.657464  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2202) took 1042 ms. Trace:
I20260709 13:18:36.661798  6666 rpcz_store.cc:276] 0709 13:18:35.614595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.635897 (+1021302us) service_pool.cc:224] Handling call
0709 13:18:36.657388 (+ 21491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.662891  6667 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:36.643340 11573 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:36.667063  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2576) took 1244 ms. Trace:
I20260709 13:18:36.667554  6667 rpcz_store.cc:276] 0709 13:18:35.422660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.655421 (+1232761us) service_pool.cc:224] Handling call
0709 13:18:36.666986 (+ 11565us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.644446 11024 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.644022 11433 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:36.647960 11410 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.624553 11277 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.651654 11564 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.672549  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:36.675207  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 103145us
I20260709 13:18:36.675514  6593 raft_consensus.cc:2368] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13.
I20260709 13:18:36.677706  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:36.678258  6666 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:36.681725 11410 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.682380 11586 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:36.683213 11586 consensus_queue.cc:1243] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:36.683259  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.684144  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 958) took 1275 ms. Trace:
W20260709 13:18:36.681427  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101889us
I20260709 13:18:36.684469  6667 rpcz_store.cc:276] 0709 13:18:35.408366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.679023 (+1270657us) service_pool.cc:224] Handling call
0709 13:18:36.684085 (+  5062us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.682247 11501 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.646510 11571 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:18:36.704442  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:36.684695 11560 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:18:36.712630  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:36.685497 11230 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.725317  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:36.726317  6593 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.717681  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:36.728024  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75128.4us
I20260709 13:18:36.676280 11518 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.731434  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:36.733726  7230 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:36.666450 11523 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.677299 11024 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.724495 11587 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.746964  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:36.748368  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:36.742100  6666 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:36.686784 11424 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.734179 11588 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.730823 11403 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:18:36.680190 11564 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.757682  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2575) took 1335 ms. Trace:
I20260709 13:18:36.758997 11573 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:36.759471  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:36.759780  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:36.754087  7230 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:36.774752  6515 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:36.775182  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:36.763149  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:36.764463  6666 rpcz_store.cc:276] 0709 13:18:35.422418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.662644 (+1240226us) service_pool.cc:224] Handling call
0709 13:18:36.757623 (+ 94979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.766067  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:36.780239  6521 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:36.781230  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:36.757504 11583 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:36.791399 11583 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:36.792217 11583 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.792817 11528 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:36.793200 11583 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:36.795231 11585 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:18:36.772183 11588 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.798343 11588 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:36.819921 11410 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:36.826254 11480 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.000s
I20260709 13:18:36.785337 11573 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.786648 11449 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:36.770815  6593 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:36.789127  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:36.790995 11559 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:36.798940 11528 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.761785 11587 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.799472  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1286) took 1343 ms. Trace:
I20260709 13:18:36.737923 11505 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.804476 11523 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:36.805730 11230 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:36.806064  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:36.806799 11564 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:36.808859 11424 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.810880 11024 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.830754 11588 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:36.832783 11563 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.837505 11464 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.838529 11431 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:36.941368 11426 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:36.941936 11426 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.835464 11446 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.839931 11501 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:36.840677  6667 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:36.737483 11433 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:36.781936  6519 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:36.861706 11467 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.947847  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2171) took 2898 ms. Trace:
I20260709 13:18:36.873202  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:36.874943  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2648) took 1680 ms. Trace:
W20260709 13:18:36.875465 11427 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.001s
I20260709 13:18:36.875830  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1276) took 1683 ms. Trace:
I20260709 13:18:36.877720  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1078) took 1231 ms. Trace:
I20260709 13:18:36.951833  6597 rpcz_store.cc:276] 0709 13:18:35.646099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.671520 (+ 25421us) service_pool.cc:224] Handling call
0709 13:18:36.877500 (+1205980us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b5400092ed4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:36.877654 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962042240}
I20260709 13:18:36.881687  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:36.881923  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:36.882299  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:36.953209  6516 raft_consensus.cc:2410] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:36.788302 11582 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:36.891857  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 215989us
I20260709 13:18:36.938059 11441 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:36.897833  6666 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:36.899714  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:36.962363  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:36.901048  6662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2165) took 2823 ms (client timeout 1978 ms). Trace:
I20260709 13:18:36.885831 11480 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:36.862566 11553 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.461s	user 0.003s	sys 0.000s
I20260709 13:18:36.963578  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:36.969575  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:36.907263  7235 rpcz_store.cc:276] 0709 13:18:35.439414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.696057 (+256643us) service_pool.cc:224] Handling call
0709 13:18:35.696352 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.697428 (+  1076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.697457 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.829070 (+131613us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.209349 (+380279us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:36.209807 (+   458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.210150 (+   343us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.782854 (+572704us) raft_consensus.cc:1685] finished
0709 13:18:36.782893 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.783019 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":159284,"thread_start_us":233516,"threads_started":1,"wal-append.queue_time_us":469232}
I20260709 13:18:36.970865  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:36.871601 11537 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:36.872082 11583 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:36.934298  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 128287us
I20260709 13:18:36.904135 11482 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:36.948237  6670 rpcz_store.cc:276] 0709 13:18:34.050184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.440093 (+389909us) service_pool.cc:224] Handling call
0709 13:18:34.460943 (+ 20850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.146659 (+685716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.947600 (+1800941us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b540011dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:36.947777 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1575181184}
I20260709 13:18:36.884274 11507 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.949318  6590 rpcz_store.cc:276] 0709 13:18:35.194762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.195200 (+   438us) service_pool.cc:224] Handling call
0709 13:18:35.262683 (+ 67483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.485937 (+223254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.874637 (+1388700us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54002fbcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:36.874873 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512205056}
I20260709 13:18:36.893246  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:36.955914  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.032684  7236 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:37.033722  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:36.950811  7231 rpcz_store.cc:276] 0709 13:18:35.192370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.202017 (+  9647us) service_pool.cc:224] Handling call
0709 13:18:35.270938 (+ 68921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.510507 (+239569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.875571 (+1365064us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540051f754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:36.875754 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549766016}
I20260709 13:18:36.964462  6520 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:36.964449  6662 rpcz_store.cc:269] 0709 13:18:34.048092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.048254 (+   162us) service_pool.cc:224] Handling call
0709 13:18:36.871153 (+2822899us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54001e0354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:36.871278 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280271616}
W20260709 13:18:36.967478  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.971401  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:37.035112  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63785.5us
I20260709 13:18:36.971896 11316 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:36.974287  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.037461  7235 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:36.975445  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:36.903878 11528 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.978813  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.028039  6664 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:37.047427  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.049153  6662 raft_consensus.cc:2376] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14. Re-sending same reply.
I20260709 13:18:36.959107 11501 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.943043 11471 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.943979 11588 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.036149  6636 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:37.052024  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 76641.3us
I20260709 13:18:37.030121 11595 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:36.980046 11582 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:37.056701  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:37.057166  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 78443.7us
I20260709 13:18:37.049989  7235 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:37.050997  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.060350  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.060451  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:36.978402 11408 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
W20260709 13:18:37.054071 11553 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:18:37.047427  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.032042  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:36.954357 11467 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.061131  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.069383  6662 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:37.062153  7231 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:37.063328  7230 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:18:36.959599 11427 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:36.978055 11441 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:37.052887 11410 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:37.056367 11523 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:37.028187 11583 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.071703  6519 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:36.980980 11537 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.077179  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.061504 10856 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:37.063642 11563 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:37.089735 11596 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:18:37.077028  6666 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:37.082609  6517 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: [12.12-14.13]
I20260709 13:18:37.084550 11593 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.085511  6670 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:37.096909  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2199) took 1487 ms. Trace:
I20260709 13:18:37.096252  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1077) took 1450 ms. Trace:
I20260709 13:18:37.097282  6667 rpcz_store.cc:276] 0709 13:18:35.610070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.698439 (+1088369us) service_pool.cc:224] Handling call
0709 13:18:37.091777 (+393338us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b54000e56d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.096835 (+  5058us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142923136}
I20260709 13:18:37.056849 11501 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.099092  6516 pending_rounds.cc:85] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 7
I20260709 13:18:37.099305  6518 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:37.099398  6516 pending_rounds.cc:107] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:18:37.097517  6592 rpcz_store.cc:276] 0709 13:18:35.645801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.649672 (+  3871us) service_pool.cc:224] Handling call
0709 13:18:37.092226 (+1442554us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b540034fb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.096202 (+  3976us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185382784}
I20260709 13:18:37.100529  6516 raft_consensus.cc:2889] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:37.102181  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.061008 11595 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.102962  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.107913  6518 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:37.103108 11579 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:37.064909 11024 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 13:18:37.104604  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44231.4us
I20260709 13:18:37.105590  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.093648 11564 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.111382  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.094770 11563 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.118021  6662 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:37.118324  6592 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:37.118856  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 962) took 1520 ms. Trace:
I20260709 13:18:37.119504  6662 rpcz_store.cc:276] 0709 13:18:35.599065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.096038 (+1496973us) service_pool.cc:224] Handling call
0709 13:18:37.118782 (+ 22744us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.113451  6518 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:37.109833  6670 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:37.121471  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2578) took 1698 ms. Trace:
I20260709 13:18:37.122908  6670 rpcz_store.cc:276] 0709 13:18:35.423188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.065867 (+1642679us) service_pool.cc:224] Handling call
0709 13:18:37.121395 (+ 55528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.097023 11570 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.124155  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:37.191459 11558 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:18:37.191919  6670 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:37.191745 11570 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.192567 11564 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.193126  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.194168  6670 raft_consensus.cc:2410] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:18:37.194609  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2200) took 1584 ms. Trace:
I20260709 13:18:37.178066 11024 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:37.195441 11024 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:37.195574 11588 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:37.112495 11523 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:37.179025 11427 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.179488  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.204319  6667 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:37.180656  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.181414  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:37.183743 11124 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.107014 11593 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:37.187170 11598 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:18:37.188268 11601 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:18:37.188158 11583 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:37.189241 11599 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:18:37.189858 11467 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.190871 11507 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.194849 11564 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:37.197175  6670 rpcz_store.cc:276] 0709 13:18:35.610467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.192773 (+1582306us) service_pool.cc:224] Handling call
0709 13:18:37.194549 (+  1776us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.197896  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:37.137208  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.228471  6666 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:37.228926  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2582) took 1631 ms. Trace:
I20260709 13:18:37.199586 11609 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:37.179499 11441 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.201758 11579 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:37.179527 11537 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.203137  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:37.118912  6592 pending_rounds.cc:85] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 8
I20260709 13:18:37.235224  6592 pending_rounds.cc:107] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:18:37.219820 11230 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:18:37.235683  6592 raft_consensus.cc:2889] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:37.235087  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.238201  6670 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:37.205562  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.239037  6667 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:37.214713  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.239804  6662 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:37.204715 11563 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:37.216869  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.220753 11572 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
I20260709 13:18:37.221374 11501 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:37.222502 11595 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.224596  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.235174 11609 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.225195  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1389) took 2569 ms. Trace:
I20260709 13:18:37.226102 11480 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.229194  6666 rpcz_store.cc:276] 0709 13:18:35.597539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.108423 (+1510884us) service_pool.cc:224] Handling call
0709 13:18:37.228877 (+120454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.196502 11024 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.260473 11426 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.264915 11024 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:37.266261 11520 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:37.266700 11520 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.267586 11520 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.268644 11520 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:37.234532 11471 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.213423 11570 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:37.272364 11561 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.792s	user 0.002s	sys 0.002s
I20260709 13:18:37.272994  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.273659  6518 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:37.273844  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2494) took 1974 ms. Trace:
I20260709 13:18:37.242225  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.274192  6588 rpcz_store.cc:276] 0709 13:18:35.299467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.357315 (+ 57848us) service_pool.cc:224] Handling call
0709 13:18:35.357607 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.363382 (+  5775us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:35.632115 (+268733us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b180045ce20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:18:35.689493 (+ 57378us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.212572 (+523079us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:36.220320 (+  7748us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.220639 (+   319us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:36.220719 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.220875 (+   156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.271639 (+1050764us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:37.273630 (+  1991us) raft_consensus.cc:1685] finished
0709 13:18:37.273651 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.273788 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41769,"prepare.queue_time_us":950093,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":136084,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":44232,"spinlock_wait_cycles":38623360,"thread_start_us":307226,"threads_started":2,"wal-append.queue_time_us":487470}
I20260709 13:18:37.275825  6662 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:37.242482  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 959) took 1822 ms. Trace:
I20260709 13:18:37.276800  6667 rpcz_store.cc:276] 0709 13:18:35.420581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.205204 (+1784623us) service_pool.cc:224] Handling call
0709 13:18:37.242444 (+ 37240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.243646 11593 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.245738 11579 consensus_queue.cc:1243] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:37.114234 11482 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.209306 11523 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.248473 11230 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:37.235595 11528 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.217401 10856 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.284894 11230 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.285342 11230 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.285547 11426 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.250999 11497 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:37.251801  6522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2339) took 1890 ms (client timeout 1576 ms). Trace:
I20260709 13:18:37.260048  7237 rpcz_store.cc:276] 0709 13:18:34.655618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.080780 (+425162us) service_pool.cc:224] Handling call
0709 13:18:35.158492 (+ 77712us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.174631 (+ 16139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.174665 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.175074 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.866163 (+1691089us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b18003fdea0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:36.978722 (+112559us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:36.978766 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.978838 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.224898 (+246060us) raft_consensus.cc:1685] finished
0709 13:18:37.224917 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.225118 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15748,"raft.queue_time_us":42652,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":60437,"spinlock_wait_cycles":7128960,"thread_start_us":112357,"threads_started":1,"wal-append.queue_time_us":1857818}
I20260709 13:18:37.232089 11592 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:37.277509  6670 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:37.284560 10856 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:37.249617 11603 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:18:37.294483  6670 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:18:37.288697  6522 rpcz_store.cc:269] 0709 13:18:35.361185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.364152 (+  2967us) service_pool.cc:224] Handling call
0709 13:18:37.251601 (+1887449us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400045954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.251711 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762693888}
I20260709 13:18:37.265002 11609 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:37.291092  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.296443  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2571) took 1875 ms. Trace:
I20260709 13:18:37.297307  6670 rpcz_store.cc:276] 0709 13:18:35.421473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.293745 (+1872272us) service_pool.cc:224] Handling call
0709 13:18:37.294086 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.295661 (+  1575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.296375 (+   714us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.292052 11592 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.302419  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 955) took 2019 ms. Trace:
I20260709 13:18:37.302843  6666 rpcz_store.cc:276] 0709 13:18:35.282689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.281702 (+1999013us) service_pool.cc:224] Handling call
0709 13:18:37.301954 (+ 20252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.302124 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.302150 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:37.302191 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:37.302222 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.302275 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.302368 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.304236  6667 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:37.305358  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 961) took 1884 ms. Trace:
I20260709 13:18:37.305677  6667 rpcz_store.cc:276] 0709 13:18:35.421209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.283681 (+1862472us) service_pool.cc:224] Handling call
0709 13:18:37.304081 (+ 20400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.305134 (+  1053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.305307 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.307528  6520 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:18:37.307593 11523 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:37.309782  6667 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:37.310375  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2201) took 1699 ms. Trace:
I20260709 13:18:37.310691  6667 rpcz_store.cc:276] 0709 13:18:35.610814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.306374 (+1695560us) service_pool.cc:224] Handling call
0709 13:18:37.309501 (+  3127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.310161 (+   660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.310326 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.312963  6670 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:37.313659  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2577) took 1890 ms. Trace:
I20260709 13:18:37.314030  6670 rpcz_store.cc:276] 0709 13:18:35.422907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.298276 (+1875369us) service_pool.cc:224] Handling call
0709 13:18:37.312573 (+ 14297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.313378 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.313598 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.315521 11592 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.319214  6667 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:37.320637  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2204) took 1027 ms. Trace:
I20260709 13:18:37.320998  6667 rpcz_store.cc:276] 0709 13:18:36.293237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.311374 (+1018137us) service_pool.cc:224] Handling call
0709 13:18:37.319002 (+  7628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.320340 (+  1338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.320587 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.326936  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.328810  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:37.332288  6667 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:37.333851  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2207) took 1039 ms. Trace:
I20260709 13:18:37.334209  6667 rpcz_store.cc:276] 0709 13:18:36.294043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.321747 (+1027704us) service_pool.cc:224] Handling call
0709 13:18:37.330988 (+  9241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.332627 (+  1639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.333798 (+  1171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.337616  6667 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:18:37.338490  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2208) took 1044 ms. Trace:
I20260709 13:18:37.343353  6522 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:37.343719  6522 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:37.332049 11341 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:37.344376  6522 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.348644  6522 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:37.333676 11426 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:37.338994  6667 rpcz_store.cc:276] 0709 13:18:36.294321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.334926 (+1040605us) service_pool.cc:224] Handling call
0709 13:18:37.337364 (+  2438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.338129 (+   765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.338424 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.346000 11455 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.352726  6594 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [9.8-10.9]
I20260709 13:18:37.353824 11426 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:37.354874 11426 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.386669  6522 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:37.391885  6667 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:37.394351  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2580) took 1970 ms. Trace:
I20260709 13:18:37.396535  6588 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:18:37.394826  6666 rpcz_store.cc:276] 0709 13:18:35.423706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.303765 (+1880059us) service_pool.cc:224] Handling call
0709 13:18:37.317020 (+ 13255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.317266 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:37.317622 (+   356us) log.cc:844] Serialized 14 byte log entry
0709 13:18:37.394002 (+ 76380us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:37.394057 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:37.394102 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.394174 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.394287 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1342,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":83,"thread_start_us":75877,"threads_started":1}
I20260709 13:18:37.386833 10856 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:37.395849 11571 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:18:37.404250  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 953) took 2153 ms. Trace:
I20260709 13:18:37.405737  6665 rpcz_store.cc:276] 0709 13:18:35.251068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.251647 (+   579us) service_pool.cc:224] Handling call
0709 13:18:35.269930 (+ 18283us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:35.562767 (+292837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.404000 (+1841233us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54000e6ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:37.404193 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671938944}
I20260709 13:18:37.407845  6666 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:37.409044  6666 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:37.410280  6666 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:18:37.410898  6666 pending_rounds.cc:85] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 9
I20260709 13:18:37.411429  6666 pending_rounds.cc:107] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:18:37.411741  6666 raft_consensus.cc:2889] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:37.411866  6669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 951) took 2165 ms (client timeout 1749 ms). Trace:
W20260709 13:18:37.412274  6669 rpcz_store.cc:269] 0709 13:18:35.246737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.266206 (+ 19469us) service_pool.cc:224] Handling call
0709 13:18:37.411701 (+2145495us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54003f5f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.411802 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899820800}
I20260709 13:18:37.414440 11533 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.416364 11563 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:37.418792 11610 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:18:37.421833 11473 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.423882 11446 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:37.427064 11577 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.740s	user 0.002s	sys 0.000s
I20260709 13:18:37.410173 11230 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.443703  7237 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:37.446174 11587 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:37.447075  6671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2190) took 2902 ms (client timeout 2365 ms). Trace:
W20260709 13:18:37.447506  6671 rpcz_store.cc:269] 0709 13:18:34.544248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.775211 (+230963us) service_pool.cc:224] Handling call
0709 13:18:37.446904 (+2671693us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540005a854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.447007 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3185725824}
I20260709 13:18:37.451591 11230 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.453471 11341 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:37.455439  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:37.458523  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1289) took 1326 ms. Trace:
I20260709 13:18:37.459765 11563 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.463443  7233 rpcz_store.cc:276] 0709 13:18:36.131957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.132189 (+   232us) service_pool.cc:224] Handling call
0709 13:18:37.458348 (+1326159us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b54006e14d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.458443 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76880640}
W20260709 13:18:37.462517 11567 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.873s	user 0.003s	sys 0.000s
I20260709 13:18:37.467357  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.467993  7233 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:37.468808  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:37.470754 11618 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:18:37.470845  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.471994  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.472101  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2205) took 1178 ms. Trace:
I20260709 13:18:37.473114  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.472585  6670 rpcz_store.cc:276] 0709 13:18:36.293511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.318862 (+1025351us) service_pool.cc:224] Handling call
0709 13:18:37.319188 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:37.319664 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.319705 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:37.324518 (+  4813us) log.cc:844] Serialized 52 byte log entry
0709 13:18:37.394846 (+ 70328us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:37.394902 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.394998 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.471878 (+ 76880us) raft_consensus.cc:1685] finished
0709 13:18:37.471898 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.472048 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3728,"thread_start_us":51137,"threads_started":1,"wal-append.queue_time_us":54912}
I20260709 13:18:37.477061  6591 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 13. Ops: []
I20260709 13:18:37.477705  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2647) took 2295 ms. Trace:
I20260709 13:18:37.477995  6591 rpcz_store.cc:276] 0709 13:18:35.194450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.476586 (+2282136us) service_pool.cc:224] Handling call
0709 13:18:37.476881 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.477485 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.477654 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:37.481168  7233 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:37.481501  7230 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:37.482394  7233 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:18:37.482987  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 207015us
I20260709 13:18:37.485515  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 956) took 2159 ms. Trace:
I20260709 13:18:37.488034  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2616) took 2324 ms. Trace:
I20260709 13:18:37.488417  6514 rpcz_store.cc:276] 0709 13:18:35.163570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.164374 (+   804us) service_pool.cc:224] Handling call
0709 13:18:36.414800 (+1250426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.416691 (+  1891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.487855 (+1071164us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540003f550. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.487976 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2710890368}
W20260709 13:18:37.492220  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.499346  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44011.5us
I20260709 13:18:37.500334  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:37.502810 11103 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:37.503280 11103 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:37.503648 11103 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:37.505162 11518 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.506875 11474 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:37.507251 11474 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:37.508852 11528 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:37.509071  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:37.509428 11528 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.509967 11528 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.510075  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.512840  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:37.518666  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 954) took 2266 ms. Trace:
I20260709 13:18:37.518731 11513 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:37.519181  6668 rpcz_store.cc:276] 0709 13:18:35.252007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.263819 (+ 11812us) service_pool.cc:224] Handling call
0709 13:18:35.264069 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.369055 (+104986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.369083 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.369121 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:35.369149 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.518419 (+2149270us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540011cbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:37.518465 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.518591 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241057664}
I20260709 13:18:37.519831  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.520799 11609 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:37.521176 11513 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:37.521608 11507 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:37.521616 11513 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.522100 11507 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.522770 11513 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.523296 11503 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:37.523116 11507 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.523916 11513 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:37.526160  6663 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:37.528033  6668 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:37.528100  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.528806  6663 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:37.511147 11563 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:18:37.529487 11520 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:37.529978 11520 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.530377 11563 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:37.530754 11563 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:37.531140 11343 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:37.530920 11520 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.531581 11343 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.531936  6637 leader_election.cc:400] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13.
I20260709 13:18:37.532230  6637 leader_election.cc:403] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:37.531617 11563 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.534531  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.535269  6669 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:37.536361  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:37.539146  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.539990  6666 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:37.541227  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.541941  6666 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:37.544888  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.545912  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:37.544941  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1271) took 1923 ms. Trace:
I20260709 13:18:37.548681  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.547756  7229 rpcz_store.cc:276] 0709 13:18:35.621576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.019413 (+397837us) service_pool.cc:224] Handling call
0709 13:18:36.019763 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.307435 (+287672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.544699 (+1237264us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54004de254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:37.544874 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659696000}
I20260709 13:18:37.549726  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.554100  6670 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:37.552309  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.556797  6666 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:37.516340 11584 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:37.563035  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.517194 11506 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:37.562934  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.508134 11474 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.563700 11512 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:37.565068 11334 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:37.552660 11452 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:18:37.566216  6663 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:37.566990 11506 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.545687 11426 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13.
I20260709 13:18:37.553256 11473 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:37.615185  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:37.616145  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.604450 11103 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.618999  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.604029 11512 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.620801  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.584170 11502 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:37.595243  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.596637  6669 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:37.625070  6522 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:37.626554  6595 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:37.596637  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:37.600461  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.626991  6595 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:37.626837  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.601411  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.606086  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.608469  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:37.611828  6671 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:37.551575 11582 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:18:37.627929  6668 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.631604 11582 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:37.631987 11582 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.551653 11518 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:18:37.633554  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:37.634166  6517 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:37.627987  6670 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.630792  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.629096  6666 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:37.636099  6516 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:37.636833  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:37.584349 11474 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.567507 11563 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:37.553320 11587 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:18:37.644937  6518 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:37.656128  6668 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:37.647935  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.657662  7236 raft_consensus.cc:2368] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
W20260709 13:18:37.660461  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 63917.5us
I20260709 13:18:37.632869 11582 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.661670  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.662245 11582 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.624486 11506 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.666923  6516 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:37.667524  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.658317  6595 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.664597 11343 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.669461  6595 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:37.672197 11503 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:37.673038  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.673578  6518 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:37.676729  6595 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:37.668713  6636 leader_election.cc:400] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:37.677852  6636 leader_election.cc:403] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:18:37.633740 11334 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:37.687315  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:37.688776  6670 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:37.690080  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2501) took 2311 ms. Trace:
I20260709 13:18:37.690690  6520 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
W20260709 13:18:37.691275 11417 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:18:37.692998 11103 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.693529 11343 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.690477  6595 rpcz_store.cc:276] 0709 13:18:35.365645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.404756 (+ 39111us) service_pool.cc:224] Handling call
0709 13:18:35.405054 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.626465 (+2221411us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b54000d9ba0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:18:37.677143 (+ 50678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.677315 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164116864}
I20260709 13:18:37.699529 11334 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.700510  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.701152 11587 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:37.694181 11528 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.702425  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:37.709165  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:37.709741  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.664194 11426 raft_consensus.cc:2764] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:18:37.655315 11518 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:18:37.716110  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:37.721488  6671 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:37.706888 11512 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.721990  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.724489  7229 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:37.723718  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:37.727248  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:37.734881  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:37.736282  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.721141 11103 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:37.740249  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:37.713289 11452 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:18:37.746089  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:37.756736  6671 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:37.760586  7236 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:37.762995 11452 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:37.766047 11452 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.766765 11103 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:37.767036 11452 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.768152 11452 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.767788 11103 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:37.770776 11625 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:18:37.772775  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.773540  7230 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:37.764770 11506 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.776902 11506 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:37.779731 11334 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:37.774346  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.787129  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.787744  7230 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:37.745497 11502 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:37.782987 11464 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:37.790064 11502 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.806262  7234 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:37.745266 11431 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:37.808110  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 62097.7us
I20260709 13:18:37.799613 11464 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:37.788503  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.807245 11502 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.814810  7229 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:37.810508 11426 raft_consensus.cc:2764] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:37.485824  6662 rpcz_store.cc:276] 0709 13:18:35.326169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.283821 (+1957652us) service_pool.cc:224] Handling call
0709 13:18:37.289562 (+  5741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.289768 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:37.290008 (+   240us) log.cc:844] Serialized 14 byte log entry
0709 13:18:37.433492 (+143484us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b1800050e00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:37.485266 (+ 51774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:37.485322 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:37.485362 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.485403 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.485475 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":102,"spinlock_wait_cycles":6055680,"thread_start_us":51542,"threads_started":1}
I20260709 13:18:37.815083 11343 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:37.816102  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:37.827482  7229 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:37.831419 11477 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:37.831881 11477 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:37.832696 11477 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:37.833875 11477 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:37.834976  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1403) took 3175 ms. Trace:
I20260709 13:18:37.838886  7228 rpcz_store.cc:276] 0709 13:18:34.659965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.099281 (+439316us) service_pool.cc:224] Handling call
0709 13:18:35.306265 (+206984us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:18:35.306412 (+   147us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:35.479379 (+172967us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:35.485068 (+  5689us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.658930 (+173862us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b18003733c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:36.066875 (+407945us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:36.138461 (+ 71586us) log.cc:844] Serialized 52 byte log entry
0709 13:18:36.168824 (+ 30363us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:36.168861 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.861876 (+693015us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:37.343031 (+481155us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b18003733c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:37.834647 (+491616us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54002e4ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:37.834695 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.834740 (+    45us) raft_consensus.cc:1685] finished
0709 13:18:37.834752 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.834918 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65089,"prepare.queue_time_us":10578,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":73868,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":102315,"spinlock_wait_cycles":492665088,"thread_start_us":273514,"threads_started":1,"wal-append.queue_time_us":811040}
I20260709 13:18:37.842698  7228 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:37.843896  7228 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:37.863459  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 621605us
I20260709 13:18:37.865458  6563 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:37.866887 11464 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.867745 11537 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:37.869865 11537 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:37.873554 11507 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:37.874915  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:37.830976 11528 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:37.878423  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 578867us
W20260709 13:18:37.881225  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.886394 11628 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:18:37.887081 11579 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.887754 11579 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.888975 11607 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:37.889479 11607 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:37.891677 11607 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.892107 11502 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:37.894043  6486 connection.cc:256] RPC call timeout handler was delayed by -1.5743471580001369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 13:18:37.886282 11408 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:37.895393  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.895553  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.896544  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.894471  6486 connection.cc:256] RPC call timeout handler was delayed by -1.5140998210001726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:18:37.898317  6486 connection.cc:256] RPC call timeout handler was delayed by -1.3554238000001533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
I20260709 13:18:37.883759 11587 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:37.899210  6486 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:37.917498  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:37.956481  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:37.894440 11502 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:37.920879 11417 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:37.962483  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:37.962813  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:37.963675  6519 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.936051 11579 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:37.979079  6665 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:37.978158  6664 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2198) took 2598 ms (client timeout 1803 ms). Trace:
W20260709 13:18:37.980281  6664 rpcz_store.cc:269] 0709 13:18:35.380208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.413082 (+1032874us) service_pool.cc:224] Handling call
0709 13:18:37.977835 (+1564753us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54000e65d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.978054 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405111040}
W20260709 13:18:37.970005  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 70906.5us
I20260709 13:18:37.954475 11438 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:37.987509  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 103524us
I20260709 13:18:37.984705  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:37.905390 11408 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:37.988251  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.992442  6519 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:37.994171  6515 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:37.925993 11316 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 13:18:37.995414  7232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1270) took 2371 ms (client timeout 2357 ms). Trace:
I20260709 13:18:37.963387 11417 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.002357  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:38.012343  6518 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.925065 11607 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 13:18:38.008445  7232 rpcz_store.cc:269] 0709 13:18:35.621337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.655641 (+ 34304us) service_pool.cc:224] Handling call
0709 13:18:37.992844 (+2337203us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54000ba754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:37.992975 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455974784}
I20260709 13:18:38.002161  6517 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:37.967206 11287 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.014297 11287 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:38.015957 11502 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.016580 11287 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:37.992746 11494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.808s	user 0.036s	sys 0.059s
I20260709 13:18:38.023115  7229 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:37.968467 11446 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:37.986006 11430 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:37.987573 11438 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:38.036589  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2661) took 1373 ms. Trace:
I20260709 13:18:38.037041  6590 rpcz_store.cc:276] 0709 13:18:36.663100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.080745 (+417645us) service_pool.cc:224] Handling call
0709 13:18:37.080951 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.081084 (+   133us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:37.094212 (+ 13128us) log.cc:844] Serialized 14 byte log entry
0709 13:18:37.833767 (+739555us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b1800486ac0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:38.036190 (+202423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:38.036244 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:38.036290 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.036418 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.036535 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21765,"prepare.queue_time_us":13130,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2855,"spinlock_wait_cycles":9287552,"thread_start_us":180406,"threads_started":1,"wal-append.queue_time_us":917439}
W20260709 13:18:38.040335  6561 leader_election.cc:336] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.004s (SENT)
I20260709 13:18:38.022529 11417 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:38.048518  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 63919.2us
I20260709 13:18:38.049255  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:38.049595  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.044175  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.050354  6589 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:38.045084  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.012615 11277 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:38.050338  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:38.046586  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.047369  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:38.029073 11494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.851s	user 0.036s	sys 0.063s
I20260709 13:18:38.044094  7229 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:38.059235  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.059557  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:38.054960  6590 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:38.061091  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.064018  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.065129  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.032976 11430 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:38.040303 11438 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:38.039109 11410 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:38.070327  6518 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:38.073745  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:38.074885  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.086876  6590 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:38.077639  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.087772  6597 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
W20260709 13:18:38.088376  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.088655  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.061510 11474 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:38.089463  6590 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:38.031811 11482 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:18:38.065115 11571 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:38.041857 11287 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:18:38.071259 11494 consensus_queue.cc:1243] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 14.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:18:38.095582  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.039278 11403 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:18:38.065290 11595 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:38.024600 11502 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.036458 11446 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.078661 11430 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:38.078791 11380 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:38.092986 11586 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:38.111100  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.092819 11474 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:38.099865 11571 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:38.119283  6597 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:38.153370  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:38.166919  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 78412.3us
I20260709 13:18:38.167645  6516 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:38.149138 11571 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:38.153937  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:38.153846  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:38.151528  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:38.170691  6518 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:38.183367  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.184553  6518 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:38.186722  6513 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:38.189416  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.190289  6591 raft_consensus.cc:2410] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:18:38.117573 11403 consensus_queue.cc:1243] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:38.192240 11648 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:18:38.192513 11630 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:18:38.192708 11648 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:38.193744 11648 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:38.192711 11633 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:18:38.110117 11562 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 13:18:38.199352 11635 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:18:38.200989 11649 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:38.203504  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:38.204005  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1091) took 1517 ms. Trace:
I20260709 13:18:38.204878  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:38.204358  6592 rpcz_store.cc:276] 0709 13:18:36.686284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.238433 (+552149us) service_pool.cc:224] Handling call
0709 13:18:37.240758 (+  2325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.862172 (+621414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.862216 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:37.862799 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.041341 (+178542us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:38.041369 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.041414 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:38.203798 (+162384us) raft_consensus.cc:1685] finished
0709 13:18:38.203818 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.203953 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":621023,"raft.queue_time_us":625719,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"thread_start_us":178160,"threads_started":1,"wal-append.queue_time_us":227756}
I20260709 13:18:38.112887 11526 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:18:38.118530 11586 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:38.207072  6519 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:38.124306 11474 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:38.123065 11380 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:38.233515  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.246464  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:38.247263  6513 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:38.209091 11403 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:38.249482  6597 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:38.251682  6515 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:38.254361 11482 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:18:38.260196 11408 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:38.240168 11586 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:38.263453 11446 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:38.292501 11641 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:18:38.294080 11430 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.294978  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1285) took 1067 ms. Trace:
I20260709 13:18:38.295348  7237 rpcz_store.cc:276] 0709 13:18:37.227638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.290792 (+ 63154us) service_pool.cc:224] Handling call
0709 13:18:38.294807 (+1004015us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400170a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:38.294920 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261217536}
I20260709 13:18:38.264050 11438 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.296613 11380 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:38.297559 11438 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:38.254451 11403 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:38.298272 11403 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:38.270696  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.273444 11474 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:38.298885  6597 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:38.300268 11474 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:38.275918 11527 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:38.304777 11639 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:38.306077  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1092) took 1607 ms. Trace:
I20260709 13:18:38.306411  6593 rpcz_store.cc:276] 0709 13:18:36.698791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.260388 (+561597us) service_pool.cc:224] Handling call
0709 13:18:37.263098 (+  2710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.882861 (+619763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.882913 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:37.883325 (+   412us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.031581 (+148256us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:38.031623 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.288494 (+256871us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b54000933d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:38.288542 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:38.305806 (+ 17264us) raft_consensus.cc:1685] finished
0709 13:18:38.305824 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.306020 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":567319,"raft.queue_time_us":567486,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":119679360,"thread_start_us":147901,"threads_started":1,"wal-append.queue_time_us":319096}
I20260709 13:18:38.307556  6519 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:38.308156  6597 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
W20260709 13:18:38.308530  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49456.6us
W20260709 13:18:38.309432  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.282711  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 968) took 1473 ms. Trace:
W20260709 13:18:38.363471  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.367872  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:38.362955  6665 rpcz_store.cc:276] 0709 13:18:36.809324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.406788 (+597464us) service_pool.cc:224] Handling call
0709 13:18:37.978813 (+572025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.979671 (+   858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.282479 (+302808us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54000e65d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:38.282612 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314673024}
I20260709 13:18:38.361176  6589 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
W20260709 13:18:38.376868  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 196156us
I20260709 13:18:38.310132  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:38.257354 11645 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.128s
W20260709 13:18:38.384267  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 203716us
I20260709 13:18:38.353134 11438 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:18:38.387727  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77697.4us
I20260709 13:18:38.391625  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:38.392170  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.390441  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.395529 11529 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:38.395864  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:38.395968 11529 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:38.411437 11529 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:38.411547  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:38.399327  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:38.398536  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:38.399788  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.385238 11430 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:38.415918 11607 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:38.417356 11607 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:38.417264 11586 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:38.418192  6597 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:38.418596 11467 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.419260  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1277) took 1445 ms. Trace:
I20260709 13:18:38.419122 11467 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:38.394320 11408 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:38.420970  6589 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:38.407044  6668 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:38.386742  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:38.421880  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.417899 11607 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.418551  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.405772  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:38.424072  7229 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:38.424556  7232 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:38.400897  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:38.425549  6636 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:38.425597  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:38.426323  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:38.426486  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:38.426553  6665 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:38.427155  6664 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:38.429589  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.431407  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:38.432283  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:38.433291  7237 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:38.434428 11565 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:38.436404  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:38.436691 11464 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:38.437243 11464 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:38.437600 10856 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:38.438441 11464 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.419623  7231 rpcz_store.cc:276] 0709 13:18:36.974161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.108915 (+134754us) service_pool.cc:224] Handling call
0709 13:18:37.109248 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.516720 (+407472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.516757 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:37.517214 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:18:37.926541 (+409327us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b1800364b40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:38.022243 (+ 95702us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:38.114084 (+ 91841us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.114474 (+   390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.418998 (+304524us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400646c54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:38.419044 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:38.419082 (+    38us) raft_consensus.cc:1685] finished
0709 13:18:38.419095 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.419209 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17658,"prepare.queue_time_us":16798,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":18865,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":3169,"spinlock_wait_cycles":905229440,"thread_start_us":171759,"threads_started":2,"wal-append.queue_time_us":523002}
I20260709 13:18:38.443446 11607 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:38.425299 11587 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:38.436102  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.437625  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:38.444844  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:38.437144  7232 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:38.430290 11430 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:18:38.420933 11507 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.454110  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.472579  6589 raft_consensus.cc:2368] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13.
I20260709 13:18:38.385085 11474 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:18:38.424676  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:38.475239  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50625.7us
I20260709 13:18:38.452842  6596 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:38.478201  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.437083 11502 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:38.489370  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:38.444507 11587 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.436728 11565 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:38.460315 11527 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:38.458638 11647 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:18:38.458161 11645 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:38.524084  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:38.495908  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:38.447778 11417 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.540072  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:38.471728 11512 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:18:38.538865 11527 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:38.445590 11651 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:18:38.524719 11587 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:38.529630 11537 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.000s
I20260709 13:18:38.527760 11565 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:38.459179 11607 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:18:38.494766 11547 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:38.494017 10856 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:18:38.524335 11230 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:18:38.564420 11527 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.520663 11316 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:18:38.521574 11606 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:18:38.572238  6588 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:38.554504 11477 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.009s
I20260709 13:18:38.537181 11645 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:38.605858  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 124289us
I20260709 13:18:38.559973 11537 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:38.607764  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:38.606606  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:38.608253  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:38.608793  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:38.617183  6664 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:38.571336 11467 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:38.618419  6513 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
W20260709 13:18:38.618868  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 232226us
I20260709 13:18:38.483741 11507 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.619740  6666 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 9: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:38.647276  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:38.651576  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:38.652994  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:38.657516  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 261702us
I20260709 13:18:38.658344  6597 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:38.658383  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:38.663290  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:38.666445  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.600216 11547 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:38.618186 11343 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:38.652904 11430 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:38.654486 11566 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:38.661742 11474 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:38.617802 11590 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:18:38.617106 11512 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:18:38.656855 11646 raft_consensus.cc:2764] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:18:38.666982 11586 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:38.673612 11651 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:38.683612  7229 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:38.708335  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:38.708424  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:38.792773  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 295133us
I20260709 13:18:38.705827  6518 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:38.793346  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.787665  6665 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:38.804028  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:38.674531  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 247297us
I20260709 13:18:38.812404  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:38.815344  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:38.817176  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:38.773909 11625 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:18:38.822475  7228 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:38.824450  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:38.826228  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:38.834352  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:38.786640 11607 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:38.777882 11502 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:38.768630 11474 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:38.684633 11534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.017s	sys 0.026s
I20260709 13:18:38.714901 11343 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:38.689185 11573 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.689961 11446 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.715076 11566 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.856083 11566 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:38.715191 11430 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.862154 11430 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:38.689711 11562 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.867336 11562 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.682538 11482 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:38.871146 11482 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:38.872383 11482 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:38.777685 11606 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:18:38.786868 11645 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.827256 11477 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:38.855356 11464 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:38.880458  6592 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.010s	sys 0.011s
W20260709 13:18:38.846141 11534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.018s	sys 0.026s
I20260709 13:18:38.870294 11562 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:38.848570 11474 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:38.895732  6518 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:38.846731 11607 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:38.884513 11652 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.329s	user 0.001s	sys 0.001s
I20260709 13:18:38.897420  6518 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:38.852680 11502 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:18:38.927752  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.907538  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.973493  6514 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:38.962500  6518 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.974339  6514 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:38.895867 11417 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:38.960391 11441 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:18:39.023972 11644 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.875s	user 0.003s	sys 0.007s
I20260709 13:18:39.025095  6514 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.026429  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1309) took 1456 ms. Trace:
I20260709 13:18:39.026918  6514 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:39.026885  7236 rpcz_store.cc:276] 0709 13:18:37.570313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.982241 (+411928us) service_pool.cc:224] Handling call
0709 13:18:37.982788 (+   547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.993339 (+ 10551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.993369 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.030729 (+ 37360us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.083244 (+ 52515us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:38.083295 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.084128 (+   833us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.023373 (+939245us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:39.026085 (+  2712us) raft_consensus.cc:1685] finished
0709 13:18:39.026107 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.026367 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10420,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8514,"spinlock_wait_cycles":39650560,"thread_start_us":18458,"threads_started":1,"wal-append.queue_time_us":59440}
I20260709 13:18:39.032867 11537 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:39.033398 11537 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:38.892035 11316 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 13:18:38.882237 11446 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.962214 11606 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:38.876638 11573 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:38.885250 11658 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.136s
W20260709 13:18:39.076583  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.083551  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:38.905851 11464 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:39.043593 11403 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.737s	user 0.009s	sys 0.002s
I20260709 13:18:38.895493 11477 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.093454  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:38.881906 11547 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:38.960867 11529 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.118000  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:39.112200  6514 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:18:39.074337 11303 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:39.121730  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:39.135779  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 307829us
I20260709 13:18:39.079308 11662 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:39.113072  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:39.124744  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 306078us
I20260709 13:18:39.093758 11660 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.151178  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:39.152169  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.128657  7228 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:39.034422 11537 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.071884 11507 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:39.093397 11417 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:39.144639  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:39.122694  6514 pending_rounds.cc:85] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 10
I20260709 13:18:39.188201  6514 pending_rounds.cc:107] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:18:39.136484 11522 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.189359  6514 raft_consensus.cc:2889] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:39.152345 11649 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:39.203915  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.128728  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 316446us
I20260709 13:18:39.115836 11403 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.177836 11670 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:39.218133  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:39.132525 11650 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:18:39.187312 11417 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:39.242553  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 146685us
W20260709 13:18:39.264055  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 119225us
I20260709 13:18:39.267324  6596 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:39.268006  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.268415  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.269338  6596 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:39.270234  6589 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:39.248368  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 55978.4us
W20260709 13:18:39.262681  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 118148us
I20260709 13:18:39.266434  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.280702  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:39.282194  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:39.285465  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.286470  6596 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:39.291143  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.129372 11607 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.136210 11502 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.318753  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.320044  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:39.320781  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55937.4us
W20260709 13:18:39.341279  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 223367us
I20260709 13:18:39.199928 11522 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.371495  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:39.372138  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 140996us
I20260709 13:18:39.180720 11660 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:39.323158 11408 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.903s	user 0.010s	sys 0.002s
W20260709 13:18:39.271200 11652 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:39.292095 11669 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.334136 11527 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.438946  6595 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:39.360621 11658 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:39.370906 11648 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:39.378175 11650 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:39.291812 11590 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.146s
I20260709 13:18:39.428153 11680 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.438493 11571 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.259s	user 0.011s	sys 0.002s
I20260709 13:18:39.420346 11678 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:39.459424  6597 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:39.460700  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:39.321038 11676 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.159s
W20260709 13:18:39.449510 11654 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:18:39.437459 11408 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:39.470381 11683 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:39.472129 11667 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.299s	user 0.000s	sys 0.000s
I20260709 13:18:39.447326 11507 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.422866 11494 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:18:39.422802 11651 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:18:39.262957 11670 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.506042  6591 raft_consensus.cc:1217] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Deduplicated request from leader. Original: 12.12->[13.13-13.13]   Dedup: 13.13->[]
I20260709 13:18:39.552111  6591 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: [13.13-13.13]
W20260709 13:18:39.503237  6521 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 970) took 1352 ms (client timeout 1549 ms). Trace:
I20260709 13:18:39.449239 11696 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:39.471458 11665 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
W20260709 13:18:39.553000  6521 rpcz_store.cc:269] 0709 13:18:38.150718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.152134 (+  1416us) service_pool.cc:224] Handling call
0709 13:18:39.460533 (+1308399us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b5800010598. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:18:39.503154 (+ 42621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5681024}
W20260709 13:18:39.506965 11674 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 13:18:39.466637 11677 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.468189 11659 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.395s	user 0.000s	sys 0.001s
I20260709 13:18:39.467955 11446 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.542547  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.532631 11529 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.434036 11417 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.474320 11571 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:39.595523  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 401939us
W20260709 13:18:39.622152  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.520514 11464 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.450480 11527 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:39.550909  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 271555us
I20260709 13:18:39.464352 11650 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.627787  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:39.474507 11663 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.390s	user 0.000s	sys 0.001s
W20260709 13:18:39.633354  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 260410us
I20260709 13:18:39.637095  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:39.644330  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 280503us
I20260709 13:18:39.637432  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.554207 11687 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.657073  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:39.452472 11669 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.660528  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:39.658223  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 333652us
I20260709 13:18:39.667176  6516 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:39.667135  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:39.643517 11656 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.418s	user 0.000s	sys 0.001s
I20260709 13:18:39.468572 11680 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.541538 11507 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.666563  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:39.538777 11698 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.137s
I20260709 13:18:39.670001  6488 leader_election.cc:400] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13.
I20260709 13:18:39.678198  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:39.678296  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:39.679222  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:39.680342  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.681620  6591 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [9.8-10.9]
W20260709 13:18:39.664671 11534 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.020s	sys 0.009s
I20260709 13:18:39.589874 11590 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:39.534422 11683 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:39.690088  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53062.9us
I20260709 13:18:39.691334  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.644582 11645 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:39.651131 11671 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:39.689380  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:39.687075 11534 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.020s	sys 0.009s
I20260709 13:18:39.685688 11522 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.689599  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2681) took 1491 ms. Trace:
I20260709 13:18:39.709380  6592 rpcz_store.cc:276] 0709 13:18:38.198231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.207445 (+  9214us) service_pool.cc:224] Handling call
0709 13:18:38.207711 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.241271 (+ 33560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.241300 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.241821 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.888407 (+646586us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:38.888442 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.888510 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.689398 (+800888us) raft_consensus.cc:1685] finished
0709 13:18:39.689418 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.689544 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25460,"raft.queue_time_us":33187,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9379,"thread_start_us":508431,"threads_started":1,"wal-append.queue_time_us":777556}
I20260709 13:18:39.658143 11403 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.645330 11651 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.714910 11651 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.716015 11669 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.716629 11669 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:39.716981 11669 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:39.717047 11651 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.694814 11494 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.717753 11494 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.717878 11669 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.719084  6521 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:39.719749 11669 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:39.657395 11658 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:18:39.720135 11410 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.618s	user 0.012s	sys 0.002s
I20260709 13:18:39.662004 11689 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.720577 11410 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:39.697155  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:39.671635 11417 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:39.695930 11477 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.691206 11568 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:39.626856 11677 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.725876  7233 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:39.737618  7233 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:39.727648 11682 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:18:39.720865 11689 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.699613 11679 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:39.697201 11645 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:39.734438 11566 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.878s	user 0.011s	sys 0.000s
I20260709 13:18:39.742439 11672 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.743438 11565 consensus_meta.cc:287] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.170s	user 0.012s	sys 0.000s
I20260709 13:18:39.743873 11565 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.720974 11658 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.745524 11683 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:39.748147 11660 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:39.732947 11606 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.003s
I20260709 13:18:39.672279 11687 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.735144 11527 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.736255 11670 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.736639 11403 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:39.737017 11695 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:39.738832  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1434) took 1313 ms. Trace:
I20260709 13:18:39.743050 11672 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.751899  7228 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:39.730849 11707 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:39.719794 11494 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.758881 11606 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:39.758659 11697 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:39.760520 11700 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:18:39.760095  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:39.770110  7231 rpcz_store.cc:276] 0709 13:18:38.425575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.443452 (+ 17877us) service_pool.cc:224] Handling call
0709 13:18:38.443779 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.447553 (+  3774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.447584 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.459389 (+ 11805us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.838075 (+378686us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:38.838116 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.838194 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.738576 (+900382us) raft_consensus.cc:1685] finished
0709 13:18:39.738600 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.738767 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3662,"raft.queue_time_us":3574,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":13752,"thread_start_us":310794,"threads_started":1,"wal-append.queue_time_us":528138}
I20260709 13:18:39.735174 11713 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.696883 11590 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.769675  7233 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.762174 11680 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.790160 11687 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:39.790414 11672 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.769018 11403 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.791177 11707 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:39.707043 11671 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.791524 11650 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.791903 11686 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:39.787494 11675 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:39.794745  7233 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:39.755510 11568 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:39.757315 11464 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:39.788426 11677 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:39.801862  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.786475 11713 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.784241 11683 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.758047 11566 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.803064 11534 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.810902 11403 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:39.814071 11679 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:39.783546 11695 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:39.831185  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 153026us
I20260709 13:18:39.846344  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.848146  7233 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:18:39.848853  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1418) took 2789 ms. Trace:
I20260709 13:18:39.849210  7233 rpcz_store.cc:276] 0709 13:18:37.059146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.483239 (+424093us) service_pool.cc:224] Handling call
0709 13:18:37.483468 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.725777 (+2242309us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b5400700e20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:18:39.848604 (+122827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.848787 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427822336}
I20260709 13:18:39.850733 11707 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:39.854769 11108 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
W20260709 13:18:39.855240 11430 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.992s	user 0.008s	sys 0.005s
I20260709 13:18:39.855477 11565 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:39.856467  6596 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:39.856906 11651 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:39.857235 11671 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:39.858441 11691 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.321s	user 0.002s	sys 0.000s
I20260709 13:18:39.859146 11695 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.860244 11689 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.807083 11645 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.861544 11590 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.837972 11571 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:39.823009 11467 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:18:39.863550 11587 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.297s	user 0.009s	sys 0.004s
I20260709 13:18:39.863808  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2373) took 1157 ms. Trace:
I20260709 13:18:39.828883 11568 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:39.861580 11343 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.979s	user 0.008s	sys 0.004s
I20260709 13:18:39.867736  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:39.873302  6591 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:39.837180 11677 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.878048  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 180947us
I20260709 13:18:39.893177  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.841940 11679 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:39.839762 11683 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.876969 11430 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.881981  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2520) took 1463 ms. Trace:
I20260709 13:18:39.882764  6589 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:39.867561 11537 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:39.872700 11686 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.887156  6522 rpcz_store.cc:276] 0709 13:18:38.706727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.707599 (+   872us) service_pool.cc:224] Handling call
0709 13:18:38.707945 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:38.875122 (+167177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.875162 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:38.875898 (+   736us) log.cc:844] Serialized 52 byte log entry
0709 13:18:39.136491 (+260593us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:39.136984 (+   493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.863403 (+726419us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001e9454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:39.863454 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.863503 (+    49us) raft_consensus.cc:1685] finished
0709 13:18:39.863517 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.863749 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":454,"raft.run_cpu_time_us":56,"raft.run_wall_time_us":56,"spinlock_wait_cycles":383315712,"thread_start_us":258998,"threads_started":1,"wal-append.queue_time_us":606333}
W20260709 13:18:39.876011 11528 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.009s	sys 0.011s
I20260709 13:18:39.893827 11566 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.875116 11287 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:18:39.897616  6588 rpcz_store.cc:276] 0709 13:18:38.418236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.418342 (+   106us) service_pool.cc:224] Handling call
0709 13:18:39.881730 (+1463388us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400210a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:39.881907 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342340864}
I20260709 13:18:39.897017 11464 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:39.891954  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:39.896039  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:39.892783 11474 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.928s	user 0.011s	sys 0.001s
I20260709 13:18:39.909610  6513 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:18:39.914089  6513 pending_rounds.cc:85] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 11
I20260709 13:18:39.914340  6513 pending_rounds.cc:107] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:18:39.914721  6513 raft_consensus.cc:2889] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:39.842048 11534 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.914677  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.889025 11467 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:39.912182  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.886874 11651 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:18:39.915926  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:39.908813 11528 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.009s	sys 0.011s
I20260709 13:18:39.895344 11671 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:39.916699  6515 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:39.915087  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.927918  6522 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:39.909283 11571 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.908916 11675 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:39.910552 11645 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:39.916743  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.940670  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:39.919409  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 252330us
I20260709 13:18:39.918401 11658 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:39.891531 11587 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.894757 11343 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.897447 11677 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:39.946237  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:39.924800 11697 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:39.938493 11287 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:39.907047 11702 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:18:39.936215 11671 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.958902  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.946151 11695 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.916684 11474 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.976337  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:39.977733  6665 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:39.983398  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:39.978750  6666 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:39.979485  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:39.980026  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:39.985198  6596 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:39.982772  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:39.985232  6662 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:39.981357  6664 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:39.979049  6670 raft_consensus.cc:2368] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11.
I20260709 13:18:39.990717 11713 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:39.992199 11568 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:39.956873 11607 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:39.955539 11683 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:39.975181 11724 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:18:39.996402  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:39.941530 11571 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.962196 11671 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:39.999210  6666 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:39.913931 11686 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:39.963431 11687 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.013550  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.965699 11719 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:40.010875  6517 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:18:40.011596  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:39.979732 11660 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.007738  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:40.009732  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:39.994979 11697 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.982487 11658 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.028584  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:40.056181  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.057103  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.057762  6522 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:40.059293  6666 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
W20260709 13:18:40.059796  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 140618us
I20260709 13:18:39.994781 11708 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.008013 11568 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.998577 11710 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.943694 11675 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.067848  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:39.960829 11467 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.067534  7233 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:39.945735 11679 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.080454  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.083392  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:40.084641  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.066793  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:40.072620  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.075692  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.086318  6663 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:40.066417  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:40.077371  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.087918  6662 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:39.998284 11695 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.078006  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.067495  7232 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:18:40.086149  6666 raft_consensus.cc:2376] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14. Re-sending same reply.
I20260709 13:18:40.089555  7232 pending_rounds.cc:85] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 11
I20260709 13:18:40.089830  7232 pending_rounds.cc:107] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:18:40.090152  7232 raft_consensus.cc:2889] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:40.090574  7232 pending_rounds.cc:107] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:18:40.090904  7232 raft_consensus.cc:2889] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:40.093035  6667 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:18:40.094573  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.095674  7233 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:40.095543  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:40.096916  6663 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.097539  6671 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:40.098835  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:40.099254  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.099968  6667 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:40.099946 11668 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:40.100605 11668 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.101025 11668 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.026808 11687 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.074568 11719 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:40.112468  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.104806  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:40.107690  7231 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:40.105480  6671 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
W20260709 13:18:40.110908  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44569.5us
I20260709 13:18:40.113786  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.053696 11660 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.121033 11631 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:40.121582 11631 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.123692 11631 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:40.125396 11732 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:40.096306 11467 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.075351 11547 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:18:40.102424 11464 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.132685  6665 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:40.080413 11717 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:40.114985 11526 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:40.116014  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.133242  6663 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:40.155998  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.157151  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.157887  6663 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:40.062702 11658 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.116773  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:40.166131 11591 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:18:40.165691 11464 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.166527 11571 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.153942 11526 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:40.168464 11464 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:40.168030 11526 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.169324 11526 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.171077  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:40.068805 11697 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.077759 11710 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.172535  6669 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:18:40.173494 11697 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.174284 11710 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.174850  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.192224  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:40.193589  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.119046 11723 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:40.182792  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.184950  6596 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:40.186064  6664 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.120411 11477 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:40.077728 11568 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.191372  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:40.192271  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:40.182662 11721 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:18:40.194972  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.208949  6669 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:40.180642  6594 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:40.186769 11658 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:40.196059  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.174866 11717 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.224869  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.225719  6522 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:40.152643 11534 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.217159  6662 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:40.198402  6663 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:40.222035  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:40.223515  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.227164  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.211035 11676 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:40.194761 11343 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.233398  6664 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:40.231202  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.237272  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:40.238379 11695 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.224916 11710 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:40.228029 11477 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:40.238379  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:40.253253 11477 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.253887  6596 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.228186 11717 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:40.200819 11587 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.248415  6663 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:40.246448  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:40.247352 11722 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:18:40.244917 11697 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:18:40.251240  6662 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:40.251925 11675 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.257912  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.232491 11474 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.259654  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:40.260846  6663 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:40.261098  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.234592 11723 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:40.263209  6522 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
W20260709 13:18:40.264303 11685 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.633s	user 0.002s	sys 0.001s
I20260709 13:18:40.273980  6596 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:40.277655  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2529) took 1751 ms. Trace:
I20260709 13:18:40.278527  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.279467  6663 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:40.278214  6593 rpcz_store.cc:276] 0709 13:18:38.541127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.541253 (+   126us) service_pool.cc:224] Handling call
0709 13:18:38.568241 (+ 26988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.887344 (+319103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.887387 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.888338 (+   951us) log.cc:844] Serialized 26 byte log entry
0709 13:18:39.206898 (+318560us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:39.206950 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.207047 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.260957 (+1053910us) spinlock_profiling.cc:243] Waited 6.86 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:40.277389 (+ 16432us) raft_consensus.cc:1685] finished
0709 13:18:40.277414 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.277601 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85839,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4694,"spinlock_wait_cycles":15774592,"thread_start_us":636844,"threads_started":2,"wal-append.queue_time_us":632080}
W20260709 13:18:40.280228 11725 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:18:40.280869 11693 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:40.281289 11693 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.281787 11693 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.281322  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.282548  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.229135 11708 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:18:40.283243  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.283881  6593 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:40.228312 11732 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:18:40.284956  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.256182 11676 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:40.287428  6593 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:40.276163 11695 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:40.290388  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:40.293563  7233 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.294754 11631 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:40.296101 11631 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:40.296489 11631 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:40.293414 11695 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.297631 11695 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.297533 11631 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:40.298411  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.298803 11467 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:40.299676 11669 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
I20260709 13:18:40.299368 11467 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.301965  6663 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:40.304003  6663 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:40.302042 11467 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.305599 11631 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.311205 11660 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.311810 11669 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:40.313170 11669 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:40.313581 11669 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:40.256505 11671 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:40.319825  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.321367  7230 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:40.289556  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.322696 11687 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.314599 11669 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.325706 11695 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.326257 11677 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:40.327283 11403 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.343011 11287 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:40.344444 11403 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.332794  6522 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:40.277659 11723 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.345019 11573 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.346495  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 985) took 1439 ms. Trace:
I20260709 13:18:40.347893  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2369) took 1644 ms. Trace:
I20260709 13:18:40.292948  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.356158  6593 raft_consensus.cc:2410] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:40.341234  7233 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:40.342170 11669 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.348589  6519 rpcz_store.cc:276] 0709 13:18:38.703012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.703158 (+   146us) service_pool.cc:224] Handling call
0709 13:18:38.703562 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.102466 (+398904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:39.102511 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:39.103117 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:18:39.252464 (+149347us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:39.252734 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.857901 (+605167us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:40.347316 (+489415us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b5400216bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.347399 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.347485 (+    86us) raft_consensus.cc:1685] finished
0709 13:18:40.347511 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.347781 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":891,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":19038,"spinlock_wait_cycles":951615104,"thread_start_us":74092,"threads_started":1,"wal-append.queue_time_us":310653}
I20260709 13:18:40.347270  6517 rpcz_store.cc:276] 0709 13:18:38.907436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.975929 (+ 68493us) service_pool.cc:224] Handling call
0709 13:18:38.977303 (+  1374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:40.023749 (+1046446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.345956 (+322207us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001e8554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.346371 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2302217472}
I20260709 13:18:40.366045  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.367837  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.372005  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.360553  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.375150  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.378280  6519 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:40.379034  6517 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:40.285006 11547 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:18:40.381196  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.382532  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.383879  6517 raft_consensus.cc:1217] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Deduplicated request from leader. Original: 12.12->[13.13-13.13]   Dedup: 13.13->[]
I20260709 13:18:40.384407  6517 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:18:40.384991  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 987) took 1256 ms. Trace:
I20260709 13:18:40.385358  6517 rpcz_store.cc:276] 0709 13:18:39.128757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.383485 (+1254728us) service_pool.cc:224] Handling call
0709 13:18:40.383763 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:40.384782 (+  1019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.384945 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:40.339917 11607 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:40.386853  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.389114  7233 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:40.406184 11668 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.408193  6596 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:40.409317 11668 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.410322 11568 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.377411 11565 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 13:18:40.416728  6561 leader_election.cc:336] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.365s (SENT)
I20260709 13:18:40.417270  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:40.418466 11607 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:40.421540  6522 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:40.425093  6670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2222) took 1817 ms (client timeout 2365 ms). Trace:
W20260709 13:18:40.425506  6670 rpcz_store.cc:269] 0709 13:18:38.607634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.608013 (+   379us) service_pool.cc:224] Handling call
0709 13:18:40.424908 (+1816895us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540005a854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:40.425033 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3098646272}
I20260709 13:18:40.427433  6519 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:40.427726  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.428552  6591 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:18:40.428773  6519 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:40.429855  6519 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.432507 11686 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.439612  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2378) took 1731 ms. Trace:
I20260709 13:18:40.440068  6516 rpcz_store.cc:276] 0709 13:18:38.708321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.825040 (+116719us) service_pool.cc:224] Handling call
0709 13:18:38.836513 (+ 11473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.667738 (+831225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.439379 (+771641us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b540007e4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.439546 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909875968}
I20260709 13:18:40.443614 11719 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:40.444779 11719 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:40.445858 11661 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.346s	user 0.001s	sys 0.002s
I20260709 13:18:40.461760  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2677) took 2358 ms. Trace:
I20260709 13:18:40.467213  6519 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:40.478809  6518 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.556879  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.557762  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.558779  6593 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:40.462426  6590 rpcz_store.cc:276] 0709 13:18:38.103283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.103452 (+   169us) service_pool.cc:224] Handling call
0709 13:18:38.103862 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:38.106288 (+  2426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.106334 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:38.112377 (+  6043us) log.cc:844] Serialized 52 byte log entry
0709 13:18:38.385664 (+273287us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:38.556663 (+170999us) log.cc:844] Serialized 14 byte log entry
0709 13:18:39.935651 (+1378988us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b18004858c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:40.240705 (+305054us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b18004858c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:40.461366 (+220661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.461466 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.461518 (+    52us) raft_consensus.cc:1685] finished
0709 13:18:40.461536 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.461692 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1902,"prepare.queue_time_us":170713,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":3252,"raft.queue_time_us":96275,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":7908,"spinlock_wait_cycles":86048000,"thread_start_us":415695,"threads_started":2,"wal-append.queue_time_us":371869,"wal-append.run_cpu_time_us":13509,"wal-append.run_wall_time_us":1928950}
I20260709 13:18:40.578434  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:40.579312  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.606127  6590 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:40.604775  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:40.607501  6588 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:40.605629  6517 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:40.609494  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 988) took 1479 ms. Trace:
I20260709 13:18:40.609911  6517 rpcz_store.cc:276] 0709 13:18:39.129962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.386175 (+1256213us) service_pool.cc:224] Handling call
0709 13:18:40.605193 (+219018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.609193 (+  4000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.609426 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:40.616364  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:40.624423  6518 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:18:40.625151  6518 pending_rounds.cc:85] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 8
I20260709 13:18:40.625494  6518 pending_rounds.cc:107] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:18:40.625831  6518 raft_consensus.cc:2889] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:40.626724  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 984) took 1731 ms. Trace:
I20260709 13:18:40.627195  6518 rpcz_store.cc:276] 0709 13:18:38.894892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.895213 (+   321us) service_pool.cc:224] Handling call
0709 13:18:38.895590 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.438440 (+1542850us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b58006a25d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fdd0aa 00007f04b905f447 00007f04b9077fe7 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4
0709 13:18:40.626399 (+187959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.626660 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031550208}
W20260709 13:18:40.632398 11595 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:18:40.614777 10856 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:18:40.634235 10856 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:18:40.601836 11661 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.155s	user 0.003s	sys 0.000s
I20260709 13:18:40.463976 11230 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:40.640269  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.634722 10856 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:40.603497 11645 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:18:40.644516 11645 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:40.644968 11645 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:40.644357 10856 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.645793 11645 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.641112  6517 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:40.605989 11103 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.646724 10856 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:40.651862 11695 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.599763 11410 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:18:40.577811 11607 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:18:40.556859 11676 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.605827 11606 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:18:40.656570 11671 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.658676  6588 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:40.661119  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2642) took 1562 ms. Trace:
I20260709 13:18:40.679737  6514 rpcz_store.cc:276] 0709 13:18:39.098574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.340826 (+242252us) service_pool.cc:224] Handling call
0709 13:18:39.421260 (+ 80434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.689301 (+268041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:39.689331 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:39.689368 (+    37us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:39.689395 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.660818 (+971423us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400355a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.660886 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.661049 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615991040}
I20260709 13:18:40.674693  6669 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:40.604959 11631 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:40.677469  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1430) took 2253 ms. Trace:
I20260709 13:18:40.677639  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.683018  7229 rpcz_store.cc:276] 0709 13:18:38.424550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.424752 (+   202us) service_pool.cc:224] Handling call
0709 13:18:38.424998 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:38.794476 (+369478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.677229 (+1882753us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54006a3bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.677412 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593724160}
W20260709 13:18:40.683950  7232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.016s	sys 0.017s
W20260709 13:18:40.684238  7232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.016s	sys 0.017s
I20260709 13:18:40.685144  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.685559  6516 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:40.685729  7237 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:40.686476  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:40.684193  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.675907 11103 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.692350 11430 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.693216 11103 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.693555  7236 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:40.694154  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1441) took 1016 ms. Trace:
I20260709 13:18:40.694450  7236 rpcz_store.cc:276] 0709 13:18:39.677893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.678648 (+   755us) service_pool.cc:224] Handling call
0709 13:18:39.678998 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.693405 (+1014407us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b540025cc20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:18:40.693929 (+   524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.694104 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35175296}
I20260709 13:18:40.695518  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:40.650401 11717 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.699566  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:40.705231  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:40.705885  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.706880  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:40.707605  6590 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:40.678681 11664 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:40.711299  6518 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:40.703238  7229 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:40.642943 11230 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.683607 11645 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.714589 11664 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:40.699259 11693 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.722321  7233 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:40.721925 11664 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.723120 11664 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.723263 11693 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.718875  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.700534 11631 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.731520  7236 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.732596 11631 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:40.735592  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:40.738334  6590 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:40.738991  7236 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:18:40.743287  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.736140  7232 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:40.793800  7234 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.011s	sys 0.014s
W20260709 13:18:40.808688  7234 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.013s	sys 0.016s
I20260709 13:18:40.745258 11647 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:40.793524 11477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.014s	sys 0.011s
W20260709 13:18:40.829164  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 271352us
W20260709 13:18:40.808630  7231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.006s	sys 0.020s
W20260709 13:18:40.831926  7231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.007s	sys 0.020s
I20260709 13:18:40.836838  7232 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:18:40.813239 11477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.015s	sys 0.011s
I20260709 13:18:40.844295 11717 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:40.854424  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.857049  6666 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [11.8-15.9]
I20260709 13:18:40.810983 11647 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:40.864167  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:40.864960  6518 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.877844  6518 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:18:40.828912 11380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.000s	sys 0.026s
I20260709 13:18:40.842639 11693 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:40.888522 11507 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:40.885094 11380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.000s	sys 0.027s
I20260709 13:18:40.869266 11573 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:40.879730 11717 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.904212  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2368) took 2201 ms. Trace:
I20260709 13:18:40.904600  6513 rpcz_store.cc:276] 0709 13:18:38.702583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.702778 (+   195us) service_pool.cc:224] Handling call
0709 13:18:38.703219 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.915228 (+1212009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.903978 (+988750us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540019afd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.904140 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2773271936}
I20260709 13:18:40.907624  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2370) took 2203 ms. Trace:
I20260709 13:18:40.907997  6515 rpcz_store.cc:276] 0709 13:18:38.704054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.704194 (+   140us) service_pool.cc:224] Handling call
0709 13:18:38.704589 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.925556 (+1220967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.906551 (+980995us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400111054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.907569 (+  1018us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2787141376}
I20260709 13:18:40.928797  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2367) took 2924 ms. Trace:
I20260709 13:18:40.929407  6520 rpcz_store.cc:276] 0709 13:18:38.004525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.004681 (+   156us) service_pool.cc:224] Handling call
0709 13:18:38.005010 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.032198 (+ 27188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.032227 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:39.355797 (+1323570us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b180048d4e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a
0709 13:18:40.442319 (+1086522us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b2c000014c8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:18:40.442972 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 13:18:40.807713 (+364741us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:40.807774 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.807868 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.928500 (+120632us) raft_consensus.cc:1685] finished
0709 13:18:40.928529 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.928716 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26837,"raft.queue_time_us":27266,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7400,"spinlock_wait_cycles":5770880,"thread_start_us":364094,"threads_started":1,"wal-append.queue_time_us":472522}
I20260709 13:18:40.906263 11717 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.945125 11717 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.896418 11693 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.904139 11573 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.949323 11573 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.949681  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.951090  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:40.951766  7236 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.952231 11707 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.952739 11707 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.955205 11403 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.955487  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2643) took 1850 ms. Trace:
I20260709 13:18:40.955662 11403 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.956041  6521 rpcz_store.cc:276] 0709 13:18:39.105553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.634890 (+529337us) service_pool.cc:224] Handling call
0709 13:18:39.654092 (+ 19202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.719464 (+ 65372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.955173 (+1235709us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b540014c8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:40.955416 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148707072}
I20260709 13:18:40.957531 11651 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.957705 11403 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.955926 11586 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.958017 11651 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.960366 11482 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:40.958427 11586 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:40.962219 11651 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.963492 11586 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:40.958763 11417 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.963804 11707 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.964259 11417 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.965236  6591 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:18:40.966439 11417 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.958956 11696 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.959216  6594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1103) took 1681 ms (client timeout 1518 ms). Trace:
I20260709 13:18:40.959775  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:40.967133  6594 rpcz_store.cc:269] 0709 13:18:39.277554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.280487 (+  2933us) service_pool.cc:224] Handling call
0709 13:18:40.959015 (+1678528us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54002f7bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:40.959149 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2067650816}
I20260709 13:18:40.967140 11696 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.968142  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:40.968927  7230 raft_consensus.cc:2410] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:40.969513 11696 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.970474  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.971634  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.977200 11380 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:40.955533 11649 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:40.979544 11649 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:40.980868  7230 raft_consensus.cc:2410] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:18:40.981741  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.982088 11649 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:40.983196  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:40.986898  6486 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:40.988588  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.960132 11537 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:40.992709  7230 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:40.991994 11537 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:40.996737  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:40.997373  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1097) took 1731 ms. Trace:
I20260709 13:18:40.997413  7230 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:40.998070  6595 rpcz_store.cc:276] 0709 13:18:39.265722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.265855 (+   133us) service_pool.cc:224] Handling call
0709 13:18:40.997105 (+1731250us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003a2354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:40.997293 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337785216}
I20260709 13:18:41.005328  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1439) took 1909 ms. Trace:
I20260709 13:18:41.006155  7236 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:41.005709  7234 rpcz_store.cc:276] 0709 13:18:39.096352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.145251 (+ 48899us) service_pool.cc:224] Handling call
0709 13:18:39.193270 (+ 48019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.193504 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:39.199075 (+  5571us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.229786 (+1030711us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b18004435c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287
0709 13:18:40.809641 (+579855us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.005063 (+195422us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:41.005106 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:41.005146 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.005193 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.005279 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10452,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":12685,"raft.queue_time_us":177940,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":11092,"spinlock_wait_cycles":58266496,"thread_start_us":836617,"threads_started":3}
I20260709 13:18:41.007675  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.010998  7236 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 8.
I20260709 13:18:41.035265  7236 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:41.001045 11380 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:41.007444 11707 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:41.052385 11566 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.035228 11537 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.053002 11566 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.054009 11537 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:41.057087  6561 leader_election.cc:336] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.874s (SENT)
I20260709 13:18:41.057718  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:41.041968  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.073303  6522 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:41.052440 11693 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:41.063021 11702 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:41.072760 11380 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:41.073015 11707 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:41.041661  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.079432 11586 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:41.094892  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:41.039363  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.101677  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.111227  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.112668  7236 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:41.083187 11566 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.114919  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.112468  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.115939  7237 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:41.118335  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:41.127357  6669 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:41.134501  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.135254  6591 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:41.135367  6665 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:41.137542  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.138232  6664 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:41.138459  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:41.138557  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.140614  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.143333  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.144243  7236 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:41.101857 11702 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.145651  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.146870  6671 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:41.101485 11586 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:41.148500  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.148818  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.147251  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.149560  6671 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:41.145666  6665 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:41.150028  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.149060  6670 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:41.115440 11441 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:41.152417  7237 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:41.155956  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.156042  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.157269  7237 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:41.151846  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.152637  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.159372  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.160053  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.160192  6671 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:41.153831  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.160641  6662 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:41.161871  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.164120  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.161609  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.167030  6665 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:41.119575 11417 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:18:41.123687 11664 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:18:41.173157  7200 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:41.173475  6670 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:41.162539 11108 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:41.162400 11441 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:41.159736 11507 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:18:41.179184  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:41.279814 11441 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:41.241269  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.284478 11410 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:41.285395 11573 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:41.286020 11410 consensus_queue.cc:1243] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:41.239689  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:41.255075  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.257710  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.258227  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.259421  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.175282 11103 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:41.282333 11430 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
W20260709 13:18:41.289970  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.248796  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:41.193678 11586 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.133152 11380 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.297102  6591 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:41.297919  6594 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:41.290656  6515 raft_consensus.cc:2410] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:18:41.317764  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 138530us
I20260709 13:18:41.197000 11669 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:18:41.319512 11669 consensus_queue.cc:1243] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:41.179127 11695 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:41.157598 11702 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.322984 11702 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:41.169867 11707 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.324299 11430 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:41.324925 11430 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:41.327227 11651 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:41.327890  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 79467.2us
I20260709 13:18:41.329202 11651 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:41.330093  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:41.314843 11507 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:41.330519 11651 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:41.293190 11482 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:18:41.331365  6595 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:18:41.341295  6595 pending_rounds.cc:85] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 5
I20260709 13:18:41.342361  6595 pending_rounds.cc:107] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:18:41.342676  6595 raft_consensus.cc:2889] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:41.347564  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.350296 11695 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:41.326484 11430 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.376456 11430 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:41.336670 11651 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:41.380733 11651 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.396041 11651 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:41.400616  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1100) took 2133 ms. Trace:
I20260709 13:18:41.401196  6589 rpcz_store.cc:276] 0709 13:18:39.266554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.275822 (+  9268us) service_pool.cc:224] Handling call
0709 13:18:39.882517 (+606695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.909290 (+ 26773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.400084 (+1490794us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b5400210a50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:41.400369 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1394378112}
I20260709 13:18:41.412935 11103 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:41.414765 11507 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:41.426316  6515 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:41.429971  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.431663  6519 raft_consensus.cc:2410] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:18:41.434335  6487 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.658s (SENT) [suppressed 12 similar messages]
W20260709 13:18:41.420779  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 90880.4us
I20260709 13:18:41.439005  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:41.440958  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.443956 11403 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:41.444621 11403 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:41.444859  6592 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:41.444931  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.447270 11573 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:41.447593  6589 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:41.447836 11573 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.448693 11573 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:41.416803 11482 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:41.451160  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.453403 11417 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.457935 11417 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.459712 11717 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:41.460165 11717 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.460542 11717 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:41.461108 11417 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.467540  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.468385  7232 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:41.473400  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:41.473649 11770 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.004s
I20260709 13:18:41.443140  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.478418  6589 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:41.494314 11533 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:41.495012 11533 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:41.496430 11533 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.498548 11533 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:41.500047  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.500725  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.504231  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.504508  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:41.505492  6515 raft_consensus.cc:2368] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:41.446174 11403 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.511214 11579 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.512072 11579 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.516943  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.517820  6518 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:41.519388  6515 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:41.522555 11579 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.527835  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:41.529116  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.527860  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.529793  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.529906  6589 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:41.530648  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.532281 11575 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:41.532681 11575 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:41.532819  6591 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:41.533890  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.534885  6520 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:41.531436  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:41.524343  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81420.6us
I20260709 13:18:41.539193  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:41.541514 11657 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:41.541941 11657 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.542312 11657 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:41.534097 11575 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.549860 11575 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:41.552767  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.554165  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:41.554868  6592 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:41.557652  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:41.579098  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.577500 11230 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.579762 11647 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.567651  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1454) took 1138 ms. Trace:
I20260709 13:18:41.598248 11647 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:41.584612 11684 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 2.071s	user 0.000s	sys 0.007s
I20260709 13:18:41.600479 11316 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:41.595309  7229 rpcz_store.cc:276] 0709 13:18:40.429478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.683964 (+254486us) service_pool.cc:224] Handling call
0709 13:18:41.567273 (+883309us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b5400646ed4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:41.567453 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36603264}
W20260709 13:18:41.584188  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.608477  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:41.608880  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.585548 11714 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:18:41.616092 11316 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:41.616631 11316 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:41.617098 11316 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:41.620581  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2596) took 1639 ms. Trace:
I20260709 13:18:41.622298 11707 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:41.622828 11707 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:41.623381 11707 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:41.624850 11502 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:41.629004 11669 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:41.629679 11669 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:41.623389  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 923666us
I20260709 13:18:41.631750  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.632205 11669 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:41.633066  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:41.633466  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.635054  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2230) took 1561 ms. Trace:
I20260709 13:18:41.621007  6668 rpcz_store.cc:276] 0709 13:18:39.980906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.981076 (+   170us) service_pool.cc:224] Handling call
0709 13:18:39.981403 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:40.076377 (+ 94974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.076412 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:40.083292 (+  6880us) log.cc:844] Serialized 52 byte log entry
0709 13:18:40.613188 (+529896us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:40.615559 (+  2371us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.619568 (+1004009us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b18000516a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:41.619941 (+   373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.620060 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.620099 (+    39us) raft_consensus.cc:1685] finished
0709 13:18:41.620196 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.620358 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14533,"prepare.queue_time_us":1168,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":26912,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":14183,"spinlock_wait_cycles":15384448,"thread_start_us":587429,"threads_started":2,"wal-append.queue_time_us":686625}
I20260709 13:18:41.635468  6667 rpcz_store.cc:276] 0709 13:18:40.073186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.100844 (+ 27658us) service_pool.cc:224] Handling call
0709 13:18:40.101134 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:40.101450 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:40.457514 (+356064us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.623762 (+1166248us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:41.624576 (+   814us) log.cc:844] Serialized 26 byte log entry
0709 13:18:41.624816 (+   240us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:41.624853 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.624951 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.634717 (+  9766us) raft_consensus.cc:1685] finished
0709 13:18:41.634736 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.634876 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":920019,"prepare.queue_time_us":106282,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":29,"raft.queue_time_us":53094,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":30291,"thread_start_us":601349,"threads_started":2,"wal-append.queue_time_us":360669}
I20260709 13:18:41.638588  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:41.640916 11277 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:41.641390 11277 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.641773 11277 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:41.642462  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.643157 11343 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.643766 11343 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.645732 11679 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:41.646332 11587 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:41.664911  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.672673  6592 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
W20260709 13:18:41.687964  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.498498 11717 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.646332 11464 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:41.665042 11587 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.704914  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:41.714565  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:41.715499  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 80865.2us
I20260709 13:18:41.582866 11657 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.719036  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.709735 11464 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.715867 11587 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:41.732367 11669 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.736845 11657 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:41.738833  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:41.739524  6592 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:41.725419 11464 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:41.742961 11277 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.743489  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:41.745179 11277 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:41.745882 11380 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:41.749892  6420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 327) took 45204 ms (client timeout 15000 ms). Trace:
I20260709 13:18:41.754428  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.756788 11718 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:41.757041 11719 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:41.757493 11718 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.757897 11719 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.759366  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.760013 11718 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:41.760198 11775 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:18:41.760668 11719 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:41.761152  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.763054 11741 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:41.763437 11587 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.763993 11741 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:41.753067  6416 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 7 to 10, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 10 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:41.755038 11732 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:41.767455  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.767539  6592 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:41.770972  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.771193  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.772059 11566 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:41.771854 11732 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.772519 11566 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:41.768839  6518 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:41.773730 11566 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.772121  6595 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:41.776227  6416 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 11, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 11 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:41.780220 11595 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:41.780813 11595 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:41.781044 11573 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.782301  6521 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:41.783301  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.733032 11669 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.788419 11669 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:41.790597 11732 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:41.792191  6595 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:41.793311  6519 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:41.794883  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.795432  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:41.795995 11573 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.797467  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.797894 11502 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:41.798336 11502 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.798728 11502 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:41.804029 11713 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:41.804772 11277 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:41.805534 11277 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.820389 11277 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.813369 11566 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:41.821395  6591 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:18:41.821775  6591 pending_rounds.cc:85] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 6
I20260709 13:18:41.822000  6591 pending_rounds.cc:107] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:18:41.822234  6591 raft_consensus.cc:2889] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:41.823774  6522 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:41.825282 11660 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:41.811733 11537 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.757s	user 0.020s	sys 0.010s
I20260709 13:18:41.826439  6590 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:41.826886 11660 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.827510 11660 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.829856 11660 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.804239  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.833359  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.833977 11537 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:41.781945 11595 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:41.835978 11717 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:41.838186  7228 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:18:41.756381  6420 rpcz_store.cc:269] 0709 13:17:56.545867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.546121 (+   254us) service_pool.cc:224] Handling call
0709 13:18:41.749647 (+45203526us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:31.760368 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:31.870428 (+110060us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:31.870490 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:18:31.870536 (+    46us) tablet.cc:662] Decoding operations
0709 13:18:31.899292 (+ 28756us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:31.938536 (+ 39244us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:31.941812 (+  3276us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:18:31.998920 (+ 57108us) tablet.cc:701] Row locks acquired
0709 13:18:31.998981 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:18:31.999170 (+   189us) write_op.cc:270] Start()
0709 13:18:31.999347 (+   177us) write_op.cc:276] Timestamp: P: 1783603111999104 usec, L: 0
0709 13:18:31.999385 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:18:32.004137 (+  4752us) log.cc:844] Serialized 1741 byte log entry
0709 13:18:32.140616 (+136479us) op_driver.cc:464] REPLICATION: finished
0709 13:18:32.372623 (+232007us) write_op.cc:301] APPLY: starting
0709 13:18:32.399304 (+ 26681us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:32.450384 (+ 51080us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:32.450530 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:32.712404 (+261874us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:32.712681 (+   277us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:18:32.712736 (+    55us) write_op.cc:312] APPLY: finished
0709 13:18:32.713033 (+   297us) log.cc:844] Serialized 49 byte log entry
0709 13:18:32.713224 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:32.713329 (+   105us) write_op.cc:454] Released schema lock
0709 13:18:32.713375 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:32.713754 (+   379us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:32.713782 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":310257,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":192427,"apply.run_cpu_time_us":9432,"apply.run_wall_time_us":356705,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20839,"num_ops":4,"prepare.queue_time_us":190273,"prepare.run_cpu_time_us":5021,"prepare.run_wall_time_us":254093,"replication_time_us":141074,"spinlock_wait_cycles":2304,"thread_start_us":236345,"threads_started":2}]]}
I20260709 13:18:41.839883 11533 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:41.839046  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.842108  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:41.853183 11537 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:41.853487  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:41.853806 11537 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.854164  6591 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:41.854417 11537 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:41.842803  6671 raft_consensus.cc:2376] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13. Re-sending same reply.
I20260709 13:18:41.843464  7228 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:41.843621  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.859074 11343 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:41.859336  6670 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:41.844475  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:41.863065  7229 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:41.845007  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.864229  6665 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:41.846526  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:41.847518  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.824265  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:41.866880  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 87336.3us
I20260709 13:18:41.869200  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:41.869575  6662 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:41.870045  6664 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:41.872195  6591 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:41.873677  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:41.867198  6416 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 9 to 12, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 12 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.864454 11430 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:18:41.864964 11587 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:41.886119  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.811524 11403 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:18:41.912896  6663 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:41.911108 11718 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:41.925949  6416 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 5 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:41.939692  6670 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
W20260709 13:18:41.940977  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 116807us
I20260709 13:18:41.949203  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:41.952486 11695 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:41.953286 11695 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:41.955265  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56364.7us
I20260709 13:18:41.938722 11474 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:41.960100  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:41.962869 11587 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:41.944355 11586 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:41.970588  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:41.971360  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:41.971367 11586 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.972788 11464 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.974004 11586 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:41.976493  6666 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:41.978042  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:41.955956 11595 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:18:41.979031 11778 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:18:41.980006 11595 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.980948  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:41.971065 11573 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:41.985265 11573 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:41.986094 11573 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:41.986694 11587 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:41.987788 11587 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:41.989418 11587 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:41.991990  6416 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 6 to 9, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:41.997418  6591 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:18:41.999454  6591 pending_rounds.cc:85] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 7
I20260709 13:18:41.999914  6591 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:18:42.000355  6591 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.000818 11695 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:42.003264 11714 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:42.006839 11780 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:18:41.997221 11502 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:41.983121  7200 connection.cc:256] RPC call timeout handler was delayed by -1.8667576660000123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
W20260709 13:18:42.027829  7200 connection.cc:256] RPC call timeout handler was delayed by -1.8388576659999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 13:18:41.997992  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.957782  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:42.004585  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:42.006942 11464 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:42.008890  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.031533  6591 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:42.031947  6662 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
W20260709 13:18:42.033313  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 52730.3us
I20260709 13:18:42.033504  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:42.034927  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 102994us
I20260709 13:18:42.036322  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:42.037774  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.030439 11714 consensus_queue.cc:1243] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:42.033293 11474 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:42.047029  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:42.040149 11579 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:42.055218 11502 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.060698 11732 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:42.061232 11732 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.061633 11732 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:42.061795  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:42.063546  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59088.2us
I20260709 13:18:42.064409 11343 raft_consensus.cc:2764] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:42.066475  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.067049 11316 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.069798  6590 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:42.053841  6521 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.070086  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:42.053710  6416 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 6 to 7, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 7 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:42.049314  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:42.070096  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.042496 11695 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:42.064462  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.043617 11718 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.073163  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:42.075277  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:42.079144  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.053740 11671 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:42.069355 11316 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.053535 11649 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:42.087466  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:42.060103 11579 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:42.107774  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.108261 11579 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:42.109527  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.110118  6670 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:42.112447 11718 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.114825 11695 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.116757  6520 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:42.117749 11695 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:42.117751  6592 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.094710 11649 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:42.122365 11649 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.079866 11403 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.047886 11537 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.123188 11403 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.124917 11537 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.125463 11403 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.093968 11702 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.126382 11474 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.127319 11702 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:42.129230 11707 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 13:18:42.129607 11702 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:42.089586  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44693.6us
I20260709 13:18:42.129693 11707 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.130926  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.133314  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:42.071833 11713 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:42.088091 11595 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.134004  6597 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:42.134694 11669 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:42.135499 11669 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:42.137425  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.138808  6515 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:42.095463  6416 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 5 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:42.096407  6521 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:42.096777 11467 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.093708 11575 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:42.102783 11671 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.106549  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:42.105469  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.107077  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.095727 11657 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:42.140695  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47820.1us
I20260709 13:18:42.152283  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:42.145453  6592 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:42.146914  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.153345  6521 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:42.147246  6666 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: [11.10-12.11]
I20260709 13:18:42.088997 11533 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:42.159914  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.151976  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:42.176452  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.177201  6670 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:42.183049 11713 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:42.183676  6664 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:42.184896  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.185266  6662 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:42.186196  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.187057  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.187366  6671 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:42.187558  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:42.187665  6665 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:42.186290  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:42.188410  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.188675  6662 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:42.188724  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:42.189486  6664 raft_consensus.cc:2376] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 16. Re-sending same reply.
I20260709 13:18:42.189972  6662 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:42.190457  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.191195  6664 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:42.191318  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:42.192409  6664 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:42.192901  6662 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:42.188633  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.193639  6665 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:42.194875  6665 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:42.196198  6665 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:42.199290  6597 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:42.199558  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:42.194492  6662 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
W20260709 13:18:42.204222  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:42.206035  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59069.5us
I20260709 13:18:42.209101  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:42.186175  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.210602  6416 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 6 to 7, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 7 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:42.224200  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57692.3us
W20260709 13:18:42.235574  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.161984 11533 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.239634  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.244530  6595 pending_rounds.cc:85] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 6
I20260709 13:18:42.244951  6595 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:18:42.245250  6595 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.151405 11695 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:18:42.229530 11679 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:42.148233 11660 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:18:42.261566  6416 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 3 to 10. New cstate: current_term: 10 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:42.265468  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:42.242197 11533 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.243676 11343 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:42.268525  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:42.273319  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:42.277626  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 213242us
I20260709 13:18:42.162739 11467 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:42.282775  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 96688.5us
I20260709 13:18:42.279814  6416 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 7 to 10, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 10 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:42.250255 11657 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:18:42.263698 11679 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:42.241967 11573 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.256903 11717 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:18:42.262714 11651 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:18:42.324105 11651 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:42.324513 11651 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:42.325438 11651 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:42.327006  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68107.2us
I20260709 13:18:42.327770 11732 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:42.328572  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.329057  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:42.329421  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:42.329991 11732 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.313361 11573 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.313886 11595 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:42.314525 11467 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.314800 11657 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:42.314936 11697 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.317741 11707 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.271704 11586 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.334662 11695 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:42.337833  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.342537  6416 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 7 to 9, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 9 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:42.362749  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.311277 11679 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:42.346422 11718 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.351454 11316 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.352085 11657 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:42.386327  6416 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:42.393628  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 160155us
I20260709 13:18:42.396621  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:42.379724 11573 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:42.378610 11651 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.378328 11467 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:42.402863  6416 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:42.420959  7230 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:18:42.423653  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 286318us
I20260709 13:18:42.427196  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:42.407871 11573 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.362164 11474 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:18:42.433547  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.435521  7232 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:18:42.439744  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 131283us
I20260709 13:18:42.409274 11718 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:42.445500  6416 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.388520 11697 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:42.438822 11573 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:42.466406  6416 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:42.447760 11669 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:42.481319  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:42.484309  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:42.405409  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:42.488595  6594 raft_consensus.cc:606] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.489019  6594 raft_consensus.cc:614] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:42.434966 11467 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.442929 11316 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.487986  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:42.497669  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 73080.7us
I20260709 13:18:42.501313  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:42.506191  7236 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:42.510926  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:42.512698  7200 leader_election.cc:400] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:42.513108  7200 leader_election.cc:403] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:18:42.513908  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:42.514999 11800 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:42.515918 11800 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:42.516337 11800 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.516716 11800 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:42.517110 11467 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:42.518988  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46102.7us
W20260709 13:18:42.520803  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 190532us
W20260709 13:18:42.522286  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 216222us
I20260709 13:18:42.520687  6416 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 4 to 6, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 6 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.529098 11474 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:42.529590 11467 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.530324 11467 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:42.530989 11474 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:42.532661 11474 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:42.533066 11467 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:42.533555 11474 consensus_queue.cc:1243] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:42.534055 11477 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.534732 11477 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:42.537529 11477 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:42.540663  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.507959 11671 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.468686 11657 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.547503  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:42.555374  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:42.560436  7231 raft_consensus.cc:2410] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:18:42.564473 11533 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.544633 11467 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:42.572181  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45503.4us
I20260709 13:18:42.554940 11502 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:42.570680  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:42.577005  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a321aae654f82803540bffcf7968f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:42.565737 11697 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:42.590691 11467 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:42.592348 11807 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:18:42.593166 11467 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:42.593978 11657 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:42.596050 11533 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.582739 11403 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.577811  6594 raft_consensus.cc:606] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.581645  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:42.598585 11403 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.600729 11403 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.599032  7231 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:42.557755 11695 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:18:42.582132  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:42.606904  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a72647203b143708b26072192a9c102"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:42.607542  6594 raft_consensus.cc:606] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.607883  6594 raft_consensus.cc:614] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:42.611271  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:42.579553 11595 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:42.601272 11403 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:42.613632  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:42.645242  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:42.611305 11586 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.651579  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:42.655081 11799 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:18:42.655650 11791 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.444s	user 0.004s	sys 0.000s
I20260709 13:18:42.645992 11474 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.656106  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:42.615748 11467 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:42.665779  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 54619.9us
I20260709 13:18:42.667411 11820 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:42.647529 11403 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:18:42.660871 11806 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:18:42.671774  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 61424.1us
I20260709 13:18:42.674055  6593 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:42.672556  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:42.722601  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:42.728121  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.678136 11316 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.674317 11502 raft_consensus.cc:2764] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:42.746049  7236 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:42.701170 11403 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:42.675900 11575 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:42.669766 11820 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:42.751710  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 177699us
W20260709 13:18:42.753204  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 101691us
I20260709 13:18:42.664513 11474 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.764571  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:42.673561 11467 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:42.766556  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 60864.2us
W20260709 13:18:42.759618 11649 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.000s
I20260709 13:18:42.751199 11430 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:42.751509 11820 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:42.772917 11316 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.773586 11316 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.779036  6588 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:42.775532  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.755790 11679 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.758559 11575 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:42.795761  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 123258us
I20260709 13:18:42.770172 11649 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:42.810858 11797 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:18:42.811570 11796 raft_consensus.cc:2764] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:18:42.812157 11649 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:42.812901 11813 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 13:18:42.813067  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49663.4us
I20260709 13:18:42.813987  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.772080 11595 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:42.814895 11708 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:42.818882  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 57450.8us
I20260709 13:18:42.821532  6588 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:18:42.822206 11741 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:42.823909 11818 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:42.824486 11818 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.824841 11818 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.779376 11573 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.844594  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:42.842967 11579 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.733s	user 0.007s	sys 0.006s
I20260709 13:18:42.823029 11277 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:18:42.851763 11713 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:42.865303  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:42.868579  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:42.869206  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.871407  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.872038  6521 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:42.877480  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.878199  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:42.879048  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.880133  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.850795 11708 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:18:42.883404  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:42.884567  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:42.855350 11579 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.874987 11277 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:18:42.888387 11679 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.889298 11316 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.885923 11718 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:42.891593  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.892546 11343 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:42.894541  7231 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:42.884436 11671 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:42.896838 11832 raft_consensus.cc:2764] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:18:42.897357 11467 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.898694 11669 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:42.875222 11380 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.878836 11741 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:42.901760 11741 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:42.886206  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.905766  6522 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:42.876142 11587 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:42.879021 11732 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:42.913190  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:42.899904 11380 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.883760 11695 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:42.916831  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.916782  6521 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:42.917079 11732 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:42.936340  6518 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:42.936542 11732 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:42.906967 11575 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:42.943095 11575 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:42.943472 11575 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:42.944383 11502 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.944291 11575 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.944949  6664 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:42.936960 11713 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:42.945091 11502 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.945381  6664 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:42.946678  6664 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:42.951360 11533 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:42.956743 11801 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.372s	user 0.002s	sys 0.000s
W20260709 13:18:42.958900 11808 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.375s	user 0.003s	sys 0.000s
I20260709 13:18:42.926846  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:42.960038 11573 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:42.960801  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 87295.4us
I20260709 13:18:42.961675  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:42.963476  6594 raft_consensus.cc:606] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.963898  6594 raft_consensus.cc:614] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:42.967118 11441 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:42.967561 11441 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:42.967952 11441 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:42.970489 11796 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:42.971131 11796 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:42.977581  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.979712  6515 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:42.924130  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:42.987167  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63145.3us
I20260709 13:18:42.990052  6637 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:42.991488  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:42.943985 11651 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 13:18:42.992830  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:42.998883  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:42.999532  6663 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:42.999590  6665 raft_consensus.cc:1217] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[9.5-15.8]   Dedup: 9.5->[11.6-15.8]
I20260709 13:18:43.000993  6665 pending_rounds.cc:85] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 5
I20260709 13:18:43.001224  6665 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:18:43.001477  6665 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.992275 11697 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:43.002194 11533 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:43.002646 11533 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.003106 11533 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.995533 11464 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:18:42.999148 11671 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:43.006497 11702 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:43.006976 11702 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:43.007385 11702 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.994707 11316 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:18:43.013217 11579 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.016075 11502 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.970412 11732 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.000063 11474 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.984299 11714 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.017683 11714 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.018502 11732 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:42.973124 11343 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:42.902745 11741 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.020171 11714 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.022070 11796 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:43.003262  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.023219  6670 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:42.915815 11587 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:43.024252 11587 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.027041  6590 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:43.032119  6670 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:18:43.032119  6487 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.518s (SENT)
I20260709 13:18:43.033882 11695 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:43.045324  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:43.046782  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:43.040125 11800 consensus_meta.cc:287] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.002s
I20260709 13:18:43.041939  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.034914  6521 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
W20260709 13:18:43.053340  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 233756us
I20260709 13:18:43.035123  6663 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:43.054828  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 256780us
I20260709 13:18:43.038208 11464 consensus_queue.cc:1243] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:43.054984  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:43.061148  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.062397  6594 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:43.064821  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:43.065532  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:43.055027  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:43.059042  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:43.074791  6588 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:18:43.078207  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46123.5us
I20260709 13:18:43.073376 11671 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:43.077418  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:43.041311 11579 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:18:43.085047  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 95064.5us
I20260709 13:18:43.086104  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.086773  6515 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:43.088476  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.089404  6515 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:43.093214  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:18:43.093843  6521 raft_consensus.cc:606] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:43.096836  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:43.098234  6487 leader_election.cc:336] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.731s (SENT)
I20260709 13:18:43.080108 11474 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:43.102164  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.085796 11573 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:43.064122 11800 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:43.128429  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 85596.8us
I20260709 13:18:43.129673  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.072484 11403 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.091089 11464 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:43.153296  6670 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:43.153385  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.092927 11467 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:43.163210  6664 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:43.165501 11696 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:43.166167 11696 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.166527 11696 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:43.166953  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.167598  6669 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:43.168670  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.169751  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.172906 11573 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:43.175292 11403 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.176335  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.177497  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.128607 11671 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:43.105079 11316 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:43.189070  7232 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:18:43.178479  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.165545 11826 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:43.173969 11573 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.184779  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.179430 11671 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.191416  6664 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:18:43.198328  6664 pending_rounds.cc:85] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 6
W20260709 13:18:43.192823  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 107575us
I20260709 13:18:43.104882 11835 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:43.191428  6666 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:43.200829  6664 pending_rounds.cc:107] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:18:43.202100  6664 raft_consensus.cc:2889] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:43.221056  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 162693us
I20260709 13:18:43.156301 11679 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:43.164000 11464 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:43.247370  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 149162us
W20260709 13:18:43.249684  6487 leader_election.cc:336] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.658s (SENT)
I20260709 13:18:43.250177  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.203552 11579 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:43.266685  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.266834  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.267674  7231 raft_consensus.cc:2410] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:18:43.187490 11316 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:43.269461  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 172719us
I20260709 13:18:43.272619  6670 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:43.194582  6669 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:43.275520  7229 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:18:43.290607  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101221us
I20260709 13:18:43.292367  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.295245  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.266156 11464 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:43.265588 11821 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:43.241881 11679 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.295929  7230 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:43.202800 11835 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:43.226437 11838 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:43.307368 11837 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:18:43.308197 11800 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.310278 11702 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.311206  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.335243  7231 raft_consensus.cc:2376] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 14. Re-sending same reply.
I20260709 13:18:43.285010 11380 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.249742 11595 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:18:43.315023  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.163333 11820 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.155305 11695 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.230412 11533 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.320016  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:43.332362  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.288573 11807 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:43.335235  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.323948 11821 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:43.342068  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.329792 11679 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:43.358070  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 108421us
I20260709 13:18:43.328943 11718 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:43.360126  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.220990 11441 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.330905 11835 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:43.372687  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52764.9us
I20260709 13:18:43.365445  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:43.373723  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:43.299353 11587 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:43.368263  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 168370us
I20260709 13:18:43.368686  7228 raft_consensus.cc:2410] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:43.371639  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.373903  6588 raft_consensus.cc:606] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:18:43.375921  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:43.376255  6588 raft_consensus.cc:614] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:43.299702 11832 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:18:43.346590 11579 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:43.375771  7230 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:43.377420  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.377983  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.332386 11818 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.383047  6594 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:43.374823  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.379418  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:43.380591  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.387965  6521 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:43.395746  6590 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:43.396440  6592 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:43.369046 11812 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:43.397759  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:43.399362  6517 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:43.336952 11838 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.381445 11467 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:43.378474 11848 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:43.425786  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.387487 11566 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:43.406072  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.409534  6488 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:43.409648  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.410918 11836 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.411377  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:43.413110 11846 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:18:43.414328 11841 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:18:43.415709 11533 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.416194  7235 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:18:43.426836  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.443303  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.441998  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.438457  6667 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:43.494716  7230 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:43.426528  7232 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:43.440840  6666 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:43.384121 11573 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:43.401610 11741 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:43.489315  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:43.438767 11852 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:43.428579 11719 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:18:43.498504  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 55317.6us
I20260709 13:18:43.436905 11818 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.428761 11441 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.439707 11820 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.486380 11566 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.399830 11793 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.095s
W20260709 13:18:43.506737  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 109037us
I20260709 13:18:43.487989 11467 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:43.427826 11702 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.500705  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:43.492923 11850 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.001s
I20260709 13:18:43.504120  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:43.527050  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 148721us
I20260709 13:18:43.527443  7229 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:43.494460  7228 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:43.500160  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:43.376967 11718 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:43.511296  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.428401 11696 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.381848 11316 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.520882  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:43.535934  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 159437us
I20260709 13:18:43.427109 11579 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:18:43.525657  7230 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:43.526891  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.546447  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.500901 11852 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:43.508092 11566 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:43.520221 11719 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:43.518747 11502 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:43.547820  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 172022us
I20260709 13:18:43.550779  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.552122  6521 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:43.555806  7228 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:43.526829 11826 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:43.573729  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.533847 11587 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:43.535382  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.588927  7236 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:43.429437 11821 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:43.589843  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.587442  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.593709  7232 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:43.590013  6521 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:18:43.562378  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.591933  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.595053  6663 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:43.494071 11836 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.597791  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.598407  6521 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:43.588152  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.523743 11573 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:43.611424  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 201943us
I20260709 13:18:43.615653  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.619462  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.620445  6522 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:43.623641  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:43.624656 11848 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:43.624930 11849 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.213s	user 0.000s	sys 0.003s
I20260709 13:18:43.626226  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.626889  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.627832  6517 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:43.629009  7228 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:43.629369  7228 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:43.637105  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.555258 11702 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:18:43.651284  7235 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:43.652632  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:43.592875 11502 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:43.661455  6517 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:43.629781 11316 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:43.672998  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:43.661916  7228 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.685299  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.651777 11807 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:43.661594 11843 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:18:43.693472  6671 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:18:43.674562 11840 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
I20260709 13:18:43.694794  7228 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:43.694841  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:43.701310  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:43.682598  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:43.702945  6521 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.703985  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.705736  6592 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:18:43.706418  6517 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:43.709031  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:43.709167  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.709064  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.709858  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.690348 11821 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:43.724042  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:43.724452  7232 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:43.718672  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:43.719173  6597 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:43.727192  6513 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:43.728106  6521 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:43.741829 11502 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.704602 11587 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.701790 11713 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:43.766284  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 128583us
I20260709 13:18:43.777581  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.777822  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:43.773097 11587 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:43.791366  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:43.792846  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 195604us
I20260709 13:18:43.685281 11852 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:43.773500 11840 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:18:43.680117 11467 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:43.787921 11865 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:18:43.783735 11862 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:18:43.787247 11860 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:18:43.807261  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:43.848785  6597 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:43.766752 11874 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:43.788365 11863 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:43.767184 11854 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:43.847572  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.804988  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:43.850821  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.755968 11793 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:18:43.791389 11838 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:43.855093  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 258852us
I20260709 13:18:43.859537  6589 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:43.790014 11836 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:43.875275  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.862773  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:43.777624 11713 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.744194 11719 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.885244  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.856918 11696 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.885890  6595 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:43.859633 11441 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.722234 11826 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.857079 11718 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.846122 11679 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.895062  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:43.904520  6518 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:43.859937 11852 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:43.897226  6589 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:43.919118 11861 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:18:43.919462 11856 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:43.920012 11856 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.918435  6513 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
W20260709 13:18:43.925252  6487 leader_election.cc:336] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.849s (SENT)
I20260709 13:18:43.931036  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:43.857303 11874 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.860499 11854 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.923049  7229 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:43.934116  7231 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:43.856458 11859 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
W20260709 13:18:43.923507 11842 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.616s	user 0.000s	sys 0.004s
I20260709 13:18:43.939867  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:43.924310  6513 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:43.941023  6594 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:43.939925  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.942723  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:43.943406  7229 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:43.952718  7229 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:43.911224 11573 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.956453 11855 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:43.957266 11874 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.903882 11441 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:43.957623 11679 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.922758 11647 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.959969  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:43.922602 11856 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:43.961946  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 157055us
I20260709 13:18:43.962000  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:43.971158  6588 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: [9.5-13.7]
W20260709 13:18:43.958765 11835 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.003s
I20260709 13:18:43.968838 11867 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:43.979732 11867 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.980195 11867 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:43.960913 11885 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:43.980722 11885 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:43.981086 11885 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:43.982841 11876 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:18:43.983076  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:43.983368  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:43.985411 11871 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:43.986922 11881 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:43.987428 11881 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.987275 11573 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:43.987725  6588 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:43.987833 11881 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.988988 11854 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.985836 11566 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:43.993847 11566 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:43.976034 11809 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:43.997097  6597 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:43.977259  6591 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:18:44.006986  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 81760us
I20260709 13:18:44.007277  6596 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:43.975121  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:44.011731  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.011830  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:44.013079  6513 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:44.013978 11403 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:44.015779 11403 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:44.017290  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.019032 11573 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:44.020506  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:44.020911  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.024076  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.025089 11573 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:44.053159  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 78149.3us
I20260709 13:18:44.053916  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.054075  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.025525 11713 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:44.033351 11844 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.660s	user 0.000s	sys 0.005s
I20260709 13:18:44.036577  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.039834 11474 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:44.040011 11464 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.000s
I20260709 13:18:44.044302 11441 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:43.999172 11575 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.998860 11647 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.009698 11708 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:18:44.001475 11835 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.052250  6518 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
W20260709 13:18:44.054538  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44810.1us
W20260709 13:18:44.058159  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 195434us
I20260709 13:18:44.050863 11587 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.068962  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:44.047226 11847 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.701s	user 0.003s	sys 0.000s
I20260709 13:18:44.071218  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:44.055030 11573 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:44.063174 11854 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:44.056612 11889 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:44.057286 11871 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:44.082502  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:44.082815  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.084630  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:44.057010 11809 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:44.089912  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:43.969874 11855 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.062968 11474 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:44.092856  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2639) took 1123 ms. Trace:
W20260709 13:18:44.150574  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50489us
I20260709 13:18:44.075456 11713 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:44.064417 11403 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:18:44.074908 11573 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:44.177536  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.066262 11464 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.148838  6662 rpcz_store.cc:276] 0709 13:18:42.969780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.989016 (+ 19236us) service_pool.cc:224] Handling call
0709 13:18:43.016328 (+ 27312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.043647 (+ 27319us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:43.043692 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.061037 (+ 17345us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.159362 (+ 98325us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:43.160850 (+  1488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.160942 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.050560 (+889618us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b240021b678. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5e1a9c4 00007f04b5e18acb 00007f04b5fd3dc9 00007f04b5fd3a55 00007f04b5fd2fb0 00007f04b8c49488 00007f04b8c49be8 00007f04b8c21398 00007f04b8c1c76a 00007f04b8c1bbb6
0709 13:18:44.088054 (+ 37494us) raft_consensus.cc:1685] finished
0709 13:18:44.088086 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.092785 (+  4699us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42140,"raft.queue_time_us":28944,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3115,"spinlock_wait_cycles":72541056,"thread_start_us":68585,"threads_started":1,"wal-append.queue_time_us":141808}
I20260709 13:18:44.178210  6667 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:44.182080  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:44.159842  7228 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:44.183220  7228 pending_rounds.cc:85] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 8
I20260709 13:18:44.183491  7228 pending_rounds.cc:107] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:44.183734  7228 raft_consensus.cc:2889] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:44.184749  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 66168.4us
I20260709 13:18:44.092598 11708 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:44.089126 11679 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:44.144284 11869 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:44.192966  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 109182us
I20260709 13:18:44.088261 11890 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:44.098860 11441 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:44.099208 11874 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:44.101027 11277 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:44.182539 11847 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:18:44.174954 11713 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:44.204068  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:44.092254 11587 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:18:44.186959 11889 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:44.099787 11856 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 13:18:44.200884 11343 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:44.199149 11868 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.191288 11718 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:18:44.214084 11874 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.215453 11277 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:44.213462 11854 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:44.213268 11441 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:44.206383 11467 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:18:44.238004 11821 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.003s
I20260709 13:18:44.229774 11277 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:44.238061 11886 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:18:44.213939 11502 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:18:44.241070  7236 raft_consensus.cc:2410] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:44.225931 11679 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:44.226531 11872 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.230113 11874 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.249162 11890 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:44.252962 11696 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:44.253417 11464 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.238643 11441 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:44.249497 11587 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:44.254850 11579 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.265311 11872 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.293578 11579 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.295095 11872 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.295680 11579 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.258957 11835 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.259625 11647 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:44.266773 11893 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:18:44.267179 11889 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.268153  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2642) took 1295 ms. Trace:
I20260709 13:18:44.268504 11867 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.302754  6665 rpcz_store.cc:276] 0709 13:18:42.972476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.972588 (+   112us) service_pool.cc:224] Handling call
0709 13:18:42.995093 (+ 22505us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:18:43.041478 (+ 46385us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:43.079928 (+ 38450us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.157542 (+ 77614us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:43.158148 (+   606us) log.cc:844] Serialized 78 byte log entry
0709 13:18:43.158271 (+   123us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:43.158532 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.158887 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.158955 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.046569 (+887614us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:44.267930 (+221361us) raft_consensus.cc:1685] finished
0709 13:18:44.267955 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.268097 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46160,"prepare.queue_time_us":50051,"prepare.run_cpu_time_us":9129,"prepare.run_wall_time_us":46468,"raft.queue_time_us":31050,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":7862,"spinlock_wait_cycles":114481408,"thread_start_us":52013,"threads_started":1,"wal-append.queue_time_us":175733}
I20260709 13:18:44.304630 11867 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.263684 11679 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:44.306080 11679 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.308521 11573 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.270749 11595 raft_consensus.cc:993] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: : Instructing follower 3ceee54e49d041be9f7f48a2052709c5 to start an election
W20260709 13:18:44.272354  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 182036us
I20260709 13:18:44.311209  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:44.227564 11868 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.313493 11868 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.316834 11903 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:44.317417 11903 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:44.291095 11587 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:44.318393 11903 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.319360 11587 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.320617 11587 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:44.323278 11595 raft_consensus.cc:1081] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Signalling peer 3ceee54e49d041be9f7f48a2052709c5 to start an election
I20260709 13:18:44.326835 11277 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.328828 11277 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:44.329613  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.330192  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.330569  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:44.330981  6521 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:44.331619  6518 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:44.331990  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:44.295809 11854 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.333155  6515 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:44.273762 11836 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.279240 11890 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.278518  7233 raft_consensus.cc:2410] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:44.280061  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:44.280766 11695 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.281913 11855 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.265970 11696 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:44.335587 11695 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.232314 11713 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:44.336548 11864 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.540s	user 0.002s	sys 0.000s
I20260709 13:18:44.336329 11696 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.282502 11830 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:44.337718 11695 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.241896 11474 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.283353 11838 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:44.284473 11669 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:44.338805 11474 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:44.339160 11669 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.286064 11812 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.341552 11812 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.342237 11713 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.350744  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.351766  6664 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:44.276659 11316 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.002s
I20260709 13:18:44.290151 11874 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:44.298518 11807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.002s	sys 0.005s
I20260709 13:18:44.332000 11708 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:44.334229 11836 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.343760 11812 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.288563 11885 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.344202 11669 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.344128  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.345726  6517 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:44.346454 11879 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.543s	user 0.000s	sys 0.003s
I20260709 13:18:44.283486 11832 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:44.346674  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.372836  6518 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:18:44.373026  6589 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:44.343786  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.289702  6667 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:44.354372  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.330263  6513 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:44.367046  6592 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:44.366305  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.369095  6521 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:44.369199  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:44.366582  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.376364  6662 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:44.337739 11696 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:44.377658  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.378271  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:44.379690  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 184988us
I20260709 13:18:44.383884  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:44.359805 11807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.003s	sys 0.005s
I20260709 13:18:44.336598 11890 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.357175 11343 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:44.361274 11708 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:44.392089  6519 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:44.396970  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.397845  6514 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:44.398514 11848 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:44.399009 11848 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.399396 11848 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:44.403311  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 123337us
I20260709 13:18:44.404273 11890 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:44.404629  7200 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:44.373590 11826 raft_consensus.cc:2764] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:44.406077 11826 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:44.373915 11832 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:44.406544  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:44.365128 11836 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.338807 11838 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.398557  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.357125 11881 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.411429  6595 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:44.390584 11464 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:44.412375  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:44.411166 11880 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.414877 11881 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.415907  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.428684  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:44.416354  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.418586 11836 raft_consensus.cc:2764] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:44.436167  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:44.419667 11874 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.407776 11741 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:18:44.420114  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.420709 11832 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.422930 11838 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:44.424582  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.424918  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.389307 11807 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:44.425285  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.429174  6595 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:44.424628 11855 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:18:44.428447 11880 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.434688  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.423946 11573 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.439488  6662 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:44.443264  6589 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:44.464197  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.466325  6663 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:44.473742 11343 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.474165 11807 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.474984 11885 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:44.476382  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 56334.1us
I20260709 13:18:44.478374 11826 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:44.468210  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.479591 11441 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.489614 11441 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.480058  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:44.465238 11741 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:44.480999 11832 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:44.492710  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.493896  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.485491  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.494930  6667 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:44.485986 11719 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:44.496129 11719 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:44.494410  6592 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:44.496477 11719 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:44.497284 11741 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:44.497416 11719 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.497946 11741 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.499147  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.486400  6663 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:44.478266  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.509361  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:44.522763  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.524084  6596 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:18:44.540153  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.542640 11464 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:44.420734 11848 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.504307 11908 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:44.554786  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:44.543246 11464 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.539889 11826 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:44.544907 11836 raft_consensus.cc:2764] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:18:44.553717 11719 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:44.562515 11838 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:44.572641  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.592839 11474 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:44.592851 11909 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:44.565924  6590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:55992
I20260709 13:18:44.594957  6590 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:44.595427  6590 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.576709  6670 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:44.547920 11660 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.599684 11660 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.601511  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.602262  6595 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:44.539456 11903 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:44.583877  6666 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:44.584084  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1327) took 1269 ms. Trace:
W20260709 13:18:44.585425  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.586346 11586 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.582090  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.609916  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.610781  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.615231  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:44.621454  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:44.625607  6663 raft_consensus.cc:2368] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12.
I20260709 13:18:44.620759  7233 rpcz_store.cc:276] 0709 13:18:43.314670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.314776 (+   106us) service_pool.cc:224] Handling call
0709 13:18:44.583815 (+1269039us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b5400575dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:44.583983 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2157464448}
I20260709 13:18:44.627588  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1343) took 1080 ms. Trace:
I20260709 13:18:44.631124  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:44.628158  7236 rpcz_store.cc:276] 0709 13:18:43.547401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.598563 (+ 51162us) service_pool.cc:224] Handling call
0709 13:18:44.619357 (+1020794us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b54005a3554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:44.627505 (+  8148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386086144}
I20260709 13:18:44.633055  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:44.635720  6663 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:44.642109  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.642728  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.643561  6663 raft_consensus.cc:2410] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 10 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:18:44.644378 11854 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:44.641004  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:44.646221  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.506683 11708 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.648128  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.648845  6516 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:44.649576  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:44.652953 11868 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:44.629714 11586 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.654054 11868 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:18:44.622818 11876 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:18:44.639315  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.602546 11660 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:44.660634  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.661237  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.663111  6666 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:44.664875  7228 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:44.618685 11881 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:44.667013  6513 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:44.664081  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.671738  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:44.563982 11908 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.627697 11838 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.675000  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.675606  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.677364  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.676548  7230 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:44.683012  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.684880  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.687930  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.691969  7233 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:18:44.640440 11909 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:44.655368 11586 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.655531 11669 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:44.697690  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.702562  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.695926  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.695434  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.703575  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.695982  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.662772 11826 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:44.706324  7228 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:44.705842  7234 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:44.662662 11910 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:44.665021 11695 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:44.664780 11854 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:44.672717 11835 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:44.712163  6665 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:44.713549  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:44.682691 11859 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:44.721428  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.721362  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.695973 11852 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:44.690340 11800 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:44.701954 11474 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.726181  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.730749  6595 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:44.702684 11909 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.732056  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1352) took 1238 ms. Trace:
I20260709 13:18:44.734422  6590 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.655681 11903 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.740079  7228 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:44.741096  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.741544 11464 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:44.711102 11669 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.749102 11903 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.750418 11669 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.742453 11474 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.743455  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.744625  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.745651  6590 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:44.706550 11881 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.730866 11910 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.742166  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.771111  7234 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.771430  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.694466 11838 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.682852 11575 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:18:44.726187 11695 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.752060 11800 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.711095 11828 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:18:44.755893  6668 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: [11.8-16.10]
I20260709 13:18:44.753739  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:44.761507  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.808753  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.769723 11474 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.811879  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.768368 11867 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:44.731837 11852 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:44.762539  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:44.772719  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 80761.4us
I20260709 13:18:44.819100  6665 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:44.819335  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.773873  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.802309  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:44.803953  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:44.735874 11835 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:44.737607  7231 rpcz_store.cc:276] 0709 13:18:43.485066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.629709 (+144643us) service_pool.cc:224] Handling call
0709 13:18:43.933952 (+304243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.934456 (+   504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.723656 (+789200us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54007a3d50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:44.723829 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645396864}
I20260709 13:18:44.753587 11909 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:44.754747  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:44.811878  7233 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.812808  6667 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:44.763547  7235 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.813702  7237 raft_consensus.cc:2410] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:44.820500  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:44.822801  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 49006.8us
I20260709 13:18:44.824242  6662 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:44.826593  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.833304  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:44.831936  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:44.834903  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.837630  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.837879  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:44.815582 11575 consensus_queue.cc:1243] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:44.821230 11852 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:44.848274  7231 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:44.848639  7231 pending_rounds.cc:85] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 8
I20260709 13:18:44.817613 11848 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:44.843726  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:44.847260  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:44.849045  7231 pending_rounds.cc:107] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:44.854035  7231 raft_consensus.cc:2889] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:44.854185 11575 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:44.825271 11828 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:44.829941 11502 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:44.860077  7233 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:44.862059  7233 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:44.862875 11828 raft_consensus.cc:2764] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:18:44.863014 11927 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:44.865466 11502 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:44.867165  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.811781 11800 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:44.834437 11464 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.871048  6662 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:44.868131  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:44.759280 11854 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:44.842339 11595 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:44.881105  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 155684us
I20260709 13:18:44.896917  7228 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:44.907568  7233 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:44.874931 11807 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:44.889952 11920 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:18:44.874643 11474 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:45.002328  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 164772us
I20260709 13:18:45.041465  7233 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:18:44.879626 11925 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:18:45.043322  7198 leader_election.cc:400] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12.
I20260709 13:18:44.852931 11868 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:18:44.983920 11840 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:18:45.055398  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 161844us
I20260709 13:18:44.893695 11714 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:18:45.041038 11832 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.550s	user 0.001s	sys 0.008s
I20260709 13:18:45.050994 11474 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.046306 11807 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.073159  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:44.880364 11464 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:45.058017 11847 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:45.060439  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:45.090317  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.113907  6522 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
W20260709 13:18:45.063623 11918 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
W20260709 13:18:45.073820 11679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.027s	sys 0.035s
I20260709 13:18:45.064458 11908 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:45.121474  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.051604 11575 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:18:45.126363  6513 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 9: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:45.139211  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:45.143086 11923 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.291s	user 0.000s	sys 0.000s
I20260709 13:18:45.144796 11464 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:45.122589 11679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.028s	sys 0.035s
I20260709 13:18:45.149009 11910 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.149461 11930 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:45.085067 11807 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.897246 11835 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.161422  7234 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:45.083287 11474 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:45.112871 11714 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:45.117399 11881 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.171721  7230 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:18:45.151895 11832 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:45.174775 11933 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:18:45.188794  7233 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:45.175740 11838 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.183179 11826 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.210806  7235 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:45.183668 11695 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:45.203570  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:45.223865  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:45.226145  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 165788us
I20260709 13:18:45.220117  7234 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:45.221704  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:45.238583  6514 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:45.235965 11931 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:18:45.220607 11826 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.244958 11826 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:45.245823 11595 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:45.246883  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.247475  6513 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:45.247264 11714 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:45.248238 11719 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.000s
I20260709 13:18:45.251683 11651 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:45.280308 11651 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:45.281318 11651 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.235551 11832 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:18:45.247300  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.284596  6522 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:45.229146 11695 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:45.285609 11695 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.286036 11695 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:45.256166  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:45.288241  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:45.289418  7200 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:45.290143 11949 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:45.290567 11949 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.290791 11830 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:45.291065 11949 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.291613 11830 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:45.259675  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:45.231127 11929 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:18:45.218170 11942 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.264535 11940 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:45.266261 11927 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:45.271072 11741 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:45.276490 11935 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
I20260709 13:18:45.277645 11586 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:45.279284 11595 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:45.290530  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:45.328325  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:45.383114  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54890.3us
I20260709 13:18:45.384297  7231 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:45.377422  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:45.384799  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:45.297843 11852 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:45.427953  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 206323us
I20260709 13:18:45.460353  7234 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:45.354426 11838 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.428936  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2727) took 1001 ms. Trace:
W20260709 13:18:45.461442  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76717.2us
I20260709 13:18:45.327167 11832 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:45.425052  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.475003  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:45.476347  6665 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
W20260709 13:18:45.433254 11909 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.000s
I20260709 13:18:45.477391  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:45.479543  6518 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:45.479894  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 189342us
I20260709 13:18:45.357450 11800 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:45.478693  7234 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:45.348098 11830 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:45.382172 11586 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:45.465435  6596 rpcz_store.cc:276] 0709 13:18:44.427907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.548954 (+121047us) service_pool.cc:224] Handling call
0709 13:18:44.564011 (+ 15057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:44.611692 (+ 47681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:44.611741 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:44.627249 (+ 15508us) log.cc:844] Serialized 26 byte log entry
0709 13:18:44.828278 (+201029us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:44.828311 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.828371 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:45.428648 (+600277us) raft_consensus.cc:1685] finished
0709 13:18:45.428674 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.428852 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2251,"raft.queue_time_us":17779,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":123013,"threads_started":1,"wal-append.queue_time_us":361434}
I20260709 13:18:45.483085  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:45.488648  6671 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: [11.8-16.10]
I20260709 13:18:45.420039 11595 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:45.486462  6522 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:45.414134 11927 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.387725 11867 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:45.486538  6669 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:45.419643 11859 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:45.487700  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81512.3us
I20260709 13:18:45.492000  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:45.493352  6520 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:45.492381  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:45.493726  6520 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:45.494139  6667 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:45.392621  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 168790us
I20260709 13:18:45.495235  6488 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:45.491319  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:45.500039  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.475216 11947 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:45.470763 11951 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:45.505486  6518 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:45.501709  6520 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:45.415343 11946 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:18:45.507745  6520 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:45.510236  6517 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:45.355890 11940 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.446837 11695 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.524338  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:45.499470 11719 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:45.436299 11474 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.489684 11930 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:45.435913 11949 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.487711 11821 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:18:45.475422 11838 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.498932 11595 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.478171 11832 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.383191 11942 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.528461  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:45.511173 11951 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:45.510321 11867 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:45.499080 11830 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.532580 11474 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.542088 11930 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.502743 11800 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:18:45.545902 11940 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.507735 11947 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.499672 11927 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.553776 11807 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.554459 11741 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.526872 11954 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:45.555716 11719 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:45.557955  6520 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
W20260709 13:18:45.562430  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.509802 11714 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:45.487990 11909 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:45.578441  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54174.5us
I20260709 13:18:45.649812 11881 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:18:45.642933  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.645112  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 116747us
I20260709 13:18:45.646026 11960 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:45.678493  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.648056 11952 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:45.674808  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:45.578323  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48373.7us
I20260709 13:18:45.680114  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:45.675376  6516 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:45.682745  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:45.669641 11800 consensus_queue.cc:1243] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:45.687208  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:45.672531 11880 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.683184 11940 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:18:45.684798 11910 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:45.701591 11949 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:45.701969 11719 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:45.704413 11595 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.669045 11951 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.707425  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:45.670002 11741 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.682509 11952 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:45.752496  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48567.7us
I20260709 13:18:45.753775  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.754514  6517 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:45.755105  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.763036  6522 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:45.766009  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:45.767198  6522 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:45.770270  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1499) took 1064 ms. Trace:
I20260709 13:18:45.770854  7232 rpcz_store.cc:276] 0709 13:18:44.705723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.167749 (+462026us) service_pool.cc:224] Handling call
0709 13:18:45.168075 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.416400 (+248325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.416446 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.416502 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:45.416546 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.769957 (+353411us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54006cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:45.770027 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.770195 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570591488}
I20260709 13:18:45.641798 11586 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:45.700162 11947 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.773070  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:45.776249  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:45.763411 11960 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.760193 11952 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.777635  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.777529  7232 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:45.780738  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 98342.6us
I20260709 13:18:45.682310 11881 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.787358  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.788975  7232 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:45.804109 11719 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.805367 11956 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.804946 11719 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:45.805913 11956 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.807693 11945 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:18:45.808337 11719 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:45.813429 11909 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.817512 11910 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:45.818254 11910 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.818987 11807 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.772653 11474 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.688231 11867 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:45.866825  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.877527  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.882637  7232 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:45.883603  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:45.784976 11960 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.828219 11474 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.878943 11830 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.890434 11930 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.899273 11838 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.900851 11832 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.902206  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1350) took 1241 ms. Trace:
I20260709 13:18:45.902254 11910 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.902767  7228 rpcz_store.cc:276] 0709 13:18:44.660387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.811638 (+151251us) service_pool.cc:224] Handling call
0709 13:18:45.902039 (+1090401us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b54000ba754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:45.902147 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191376768}
W20260709 13:18:45.904690 11964 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:45.908077 11960 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.910439 11942 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.888441 11956 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.878703  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:45.912083 11881 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.913131  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.913553 11741 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.915333  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1365) took 1255 ms. Trace:
I20260709 13:18:45.915707  7229 rpcz_store.cc:276] 0709 13:18:44.659876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.672511 (+ 12635us) service_pool.cc:224] Handling call
0709 13:18:45.915149 (+1242638us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b5400571554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:45.915278 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878273408}
I20260709 13:18:45.914374  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.923568 11832 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:45.923789 11947 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.925088 11832 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.925674 11474 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.925437 11947 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.927352 11832 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.927953 11947 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.928146  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.929697  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.933601  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.934739  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.935527  6522 raft_consensus.cc:2410] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:45.935600 11741 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:45.936331  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.936242 11741 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.935329  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.924099 11881 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:45.937809  7232 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:45.942951 11881 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.945068 11881 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.946132  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2596) took 1597 ms. Trace:
W20260709 13:18:45.947067  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.949676  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:45.952183  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:45.948236  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.951803  7232 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:45.952975  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:45.953579  6522 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:45.956107  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.956816  6522 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:45.946501  6597 rpcz_store.cc:276] 0709 13:18:44.348445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.348560 (+   115us) service_pool.cc:224] Handling call
0709 13:18:44.348806 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.348978 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:44.505595 (+156617us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.824382 (+318787us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.897517 (+1073135us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b18003f6f40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:45.945657 (+ 48140us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b18003f6f40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:45.945799 (+   142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.945860 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:45.945900 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.945961 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.946055 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54204,"prepare.queue_time_us":57262,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1706,"raft.queue_time_us":2450,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":9754,"spinlock_wait_cycles":25707520,"thread_start_us":427654,"threads_started":2,"wal-append.queue_time_us":383996}
I20260709 13:18:45.960948  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:45.963855  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.964273 11579 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:45.965812 11579 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:45.966312  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1132) took 1305 ms. Trace:
I20260709 13:18:45.966951 11890 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:45.967383 11894 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:45.967891 11579 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.974013  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:45.969027  6590 rpcz_store.cc:276] 0709 13:18:44.660794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.820632 (+159838us) service_pool.cc:224] Handling call
0709 13:18:44.820968 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:44.841144 (+ 20176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:44.841174 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:44.841618 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:18:45.198581 (+356963us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:45.200311 (+  1730us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.273053 (+ 72742us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b4800061ed8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f 00007f04be098f37 00007f04be0ad55c 00007f04be0ad508 00007f04be0ad488 00007f04be0ad443 00007f04be0ac6ec 00007f04bcfd6a96
0709 13:18:45.965916 (+692863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.965995 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:45.966038 (+    43us) raft_consensus.cc:1685] finished
0709 13:18:45.966138 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.966256 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":777709,"prepare.queue_time_us":1731,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":783848,"raft.run_cpu_time_us":16102,"raft.run_wall_time_us":431467,"spinlock_wait_cycles":9058688,"thread_start_us":306052,"threads_started":1,"wal-append.queue_time_us":394689}
I20260709 13:18:45.974840  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:45.975610  6591 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:45.976807  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:45.977548  6595 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.971771  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:45.978780  6596 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:45.978924  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:45.946945 11956 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:45.972303  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:45.983284  6597 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:45.959307 11946 raft_consensus.cc:2764] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:18:45.986687  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:45.987324  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2599) took 1623 ms. Trace:
I20260709 13:18:45.962603 11889 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:45.982502 11696 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:45.991861 11669 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:45.981961 11890 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:45.992267 11669 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:45.993075 11669 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.993955 11890 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:45.994216 11669 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.995163 11956 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:45.997277 11956 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:45.974298 11854 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:45.999284 11889 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.001302 11889 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.002785 11903 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.003314 11903 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.005147 11586 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:46.007382  6670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1020) took 1655 ms (client timeout 1511 ms). Trace:
I20260709 13:18:46.005256 11903 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:46.007771  6670 rpcz_store.cc:269] 0709 13:18:44.351529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.351659 (+   130us) service_pool.cc:224] Handling call
0709 13:18:46.007145 (+1655486us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54000e65d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:46.007276 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462871296}
I20260709 13:18:46.019371  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:46.020044  6593 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:45.981894 11894 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.022454 11894 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:46.023800  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44956.5us
I20260709 13:18:46.025092  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:46.027806  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 72496.8us
I20260709 13:18:46.029028  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:46.032879  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:45.989058  6594 rpcz_store.cc:276] 0709 13:18:44.364107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.372527 (+  8420us) service_pool.cc:224] Handling call
0709 13:18:44.388365 (+ 15838us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:44.388718 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:44.388766 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:44.407478 (+ 18712us) log.cc:844] Serialized 52 byte log entry
0709 13:18:45.707112 (+1299634us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b1800428ce0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:45.762723 (+ 55611us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:45.886489 (+123766us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.972613 (+ 86124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.972720 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:45.972775 (+    55us) raft_consensus.cc:1685] finished
0709 13:18:45.987096 (+ 14321us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.987248 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87738,"prepare.queue_time_us":127648,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":91633,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":20797,"spinlock_wait_cycles":14826880,"thread_start_us":116633,"threads_started":2,"wal-append.queue_time_us":1367505}
I20260709 13:18:46.033653 11502 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:46.043388  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.044050  6590 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:46.044886  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.045569  7236 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:46.045895  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.033831 11951 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:18:46.047811  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:46.048511  6517 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:46.051534  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:45.998359 11854 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.056602  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:46.027464 11533 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:46.061458 11932 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.983s	user 0.004s	sys 0.002s
I20260709 13:18:46.067229  6595 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:46.070989  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.071609 11975 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:46.073547  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:46.075341 11907 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:46.080966 11579 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:46.045930  7236 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:46.054594  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:46.092293  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:46.105232  6637 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:46.107003  6637 leader_election.cc:403] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:46.109866  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.057125  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.110772  6595 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
W20260709 13:18:46.084147  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 57619.2us
I20260709 13:18:46.030827 11679 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:46.087901  6590 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
W20260709 13:18:46.112757  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58256.9us
I20260709 13:18:46.088876  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:46.114763  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:46.114903  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:46.114732  7200 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:46.119515  6669 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:46.120905  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:46.122509  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:46.123373  7232 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:18:46.123725  7232 pending_rounds.cc:85] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 10
I20260709 13:18:46.123929  7232 pending_rounds.cc:107] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:18:46.124130  7232 raft_consensus.cc:2889] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:46.125988  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.127243  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:46.128043  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.114367  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:46.090029 11669 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:46.132607  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.133255  6590 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:46.133893  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:46.134864  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:46.109131 11951 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:46.135984  7236 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.098672 11821 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:18:46.285271  7236 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:46.119611 11660 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:46.112282 11856 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:46.289976  7233 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:46.139755  6666 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:46.138729  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:46.279359  7232 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:46.292228  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.233551  6593 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:46.295034  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:46.297552  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 158859us
W20260709 13:18:46.194969 11932 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:46.327901 11977 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:46.328778 11977 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:46.331246 11977 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:46.336598  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.337730  6588 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:46.339710  6664 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:46.342058  6517 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:46.285044 11859 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:46.343230  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48043us
I20260709 13:18:46.343502 11859 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.344973 11586 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:46.345917 11859 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:46.344988  6518 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:46.315968 11855 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:46.347164  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:46.347306 11855 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:46.348294 11855 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.349311  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:46.350045  6670 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:46.314790 11909 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:46.177204 11441 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.150s
I20260709 13:18:46.128177 11696 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.203s
I20260709 13:18:46.345748 11870 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:46.134260 11826 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:46.322435  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.128895 11828 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:46.137991 11903 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:46.134434 11832 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:46.138690 11947 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:46.356907  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.311883 11856 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.361660  6595 raft_consensus.cc:2410] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:46.361660  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:46.125849 11881 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:46.288558 11343 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.355855 11910 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.202492 11474 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.225s
I20260709 13:18:46.200150 11741 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.126557 11579 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.135399 11669 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:46.362811  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.295630 11894 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.196871 11679 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.164s
I20260709 13:18:46.352656 11909 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.126362 11533 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.234s
I20260709 13:18:46.369202 11947 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:46.354399 11502 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:46.290040 11951 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.397733  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.379726  6588 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
W20260709 13:18:46.384248  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.384593  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.387984  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:46.372908  6637 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:46.400668  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.372215 11856 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:46.389688  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.371754 11881 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:46.401985  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:46.422040  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.399322  7236 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:46.383453 11669 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:46.423748  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1344) took 1765 ms. Trace:
I20260709 13:18:46.402158  6588 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 11: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:46.381184 11832 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:46.422569  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.424595  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:46.424185  7236 rpcz_store.cc:276] 0709 13:18:44.658534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.669428 (+ 10894us) service_pool.cc:224] Handling call
0709 13:18:44.695357 (+ 25929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.045482 (+1350125us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54004dea20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:18:46.423496 (+378014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.423697 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340342528}
I20260709 13:18:46.394017 11947 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:46.428175  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:46.429297  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:46.393436  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 91230.5us
I20260709 13:18:46.400163 11441 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:46.433828  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:46.410197  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:46.436021  6518 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:46.440887  7229 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:46.398876 11951 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:46.443701  6562 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:46.399335 11903 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:46.448413  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:46.449644  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:46.450203  7229 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:46.380622 11343 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.451329  7229 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:46.452898 11343 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.453480 11343 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.453940 11343 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.448413 11867 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:18:46.454967 11533 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:46.455041  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.455853 11832 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.445989 11702 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:18:46.456790 11940 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:46.431315 11836 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:18:46.465584 11660 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:46.406208 11881 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:46.439905 11502 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:46.440536 11441 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:46.395052 11890 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:18:46.470240  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:46.422845 11671 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:46.476243 11867 consensus_queue.cc:1243] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:46.425942 11900 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:46.479844  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.482999  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:46.486855 11940 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.431452 11277 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:46.487648  6596 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 12. Ops: []
I20260709 13:18:46.489674  6563 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:46.476389 11973 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:18:46.440258 11741 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:46.512801  6596 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:46.390031 11848 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:18:46.439754 11719 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:46.389492 11894 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.380045 11910 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.482350 11533 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:46.457587 11976 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:46.510113  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:46.466722 11903 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.506841 11277 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:46.523284 11533 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:46.524644 11343 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.507686 11940 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:46.494277 11874 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:18:46.526990  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:46.505748 11838 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:46.511162 11671 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:46.511255 11741 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.501814 11660 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:18:46.503774 11977 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:18:46.544332  6593 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:46.524987 11894 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.509460 11859 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:18:46.548422  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:46.506445 11900 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.529542 11702 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:46.541309 11952 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.713s	user 0.011s	sys 0.000s
I20260709 13:18:46.535094 11277 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:46.516213 11696 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:18:46.562173  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:46.562214  6518 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:46.580574  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:46.592182  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 151648us
W20260709 13:18:46.593083  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 149493us
I20260709 13:18:46.554939 11741 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:46.553122 11880 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:46.974037 11881 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.978348 11855 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:46.979159  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 400581us
I20260709 13:18:46.980521 11940 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:46.980974  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:46.981971 11848 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:46.981213 11881 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.670851  6514 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:46.750509  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2670) took 1274 ms. Trace:
I20260709 13:18:46.987454  6665 rpcz_store.cc:276] 0709 13:18:45.475699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.478742 (+  3043us) service_pool.cc:224] Handling call
0709 13:18:45.479036 (+   294us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:45.667793 (+188757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.667834 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:45.675727 (+  7893us) log.cc:844] Serialized 52 byte log entry
0709 13:18:45.759398 (+ 83671us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:45.759437 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.749820 (+990383us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54000e51d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:46.749913 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:46.749978 (+    65us) raft_consensus.cc:1685] finished
0709 13:18:46.749997 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.750438 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5004,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7576,"spinlock_wait_cycles":425906560,"thread_start_us":55483,"threads_started":1,"wal-append.queue_time_us":120227,"wal-append.run_cpu_time_us":1891,"wal-append.run_wall_time_us":1063516}
I20260709 13:18:46.611050  6517 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.637087  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:46.710909  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:46.688055  6592 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:46.986049  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:46.580137 11909 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.982828  6563 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:47.000067  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.001307  6518 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:18:47.002213  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.997325  6516 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
W20260709 13:18:46.687291 11983 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:18:46.688782 11982 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:18:46.702296 11978 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:18:46.584748 11889 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:46.672755 11852 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:47.006094  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:47.007740 11343 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.037145  6596 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:47.009972 11880 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:46.982738 11848 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:18:47.012327 11900 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.983095 11960 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:47.014988 11979 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.995661 11828 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:47.016820 11832 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.612782 11859 consensus_queue.cc:1243] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:18:47.039759 11856 consensus_meta.cc:287] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.000s
W20260709 13:18:47.034173  6637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 2.167s (SENT) [suppressed 6 similar messages]
I20260709 13:18:47.057480  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:47.035360  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.041307  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.037506 11343 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.037014 11852 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:47.043249 11708 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:47.039345 11719 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:18:47.065208 11852 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.066264 11719 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.003706 11718 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.066802 11719 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.067391 11852 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:47.068466 11856 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.069599  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:47.060515 11880 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:47.050928 11828 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:47.050414  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44385.6us
I20260709 13:18:47.071310 11695 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.071954 11880 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:46.726066 11533 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.072881 11828 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:46.612457 11952 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.074281 11996 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.074884 11996 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.076056 11952 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:46.644747 11867 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.076993 11996 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.077111 11867 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:47.078076  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.726336 11669 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.080122 11669 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:47.080984  6514 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:47.064607 11343 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:47.064934 11947 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.624s	user 0.004s	sys 0.004s
I20260709 13:18:47.066928 11718 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.052209 11910 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.034637 11940 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:46.749693 11277 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.078130  6515 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:47.056514 11859 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.040889 11909 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.082355  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.077580 11996 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.059818  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.088059  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.112282  7232 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
W20260709 13:18:47.114115  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.114764 11910 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.083806 11441 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.115275 11910 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.115769 11996 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.116655 11885 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:47.117770 11885 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:47.118292 11885 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:47.119164 11885 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.120175  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1372) took 1770 ms. Trace:
I20260709 13:18:47.111346 11828 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.120589  7234 rpcz_store.cc:276] 0709 13:18:45.350222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.480973 (+130751us) service_pool.cc:224] Handling call
0709 13:18:45.481247 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.546485 (+ 65238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.546518 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.546564 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:45.546600 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.119942 (+1573342us) spinlock_profiling.cc:243] Waited 65 ms on lock 0x7b54006cc354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:47.119992 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.120118 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149521664}
I20260709 13:18:47.105007  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:47.122290  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.122473 11718 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.122857  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.122942  7234 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:18:47.085278 11927 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.044492 11960 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.123723 11870 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:47.124401  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2447) took 1650 ms. Trace:
I20260709 13:18:47.125134 11960 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.084369 11984 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.125555  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.125804 11984 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.099156  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.099888 11867 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.092038 11903 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:47.155047 11903 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.099589 11947 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.099174 11533 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.057191 11832 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.100690 11741 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.104615  6668 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:47.107076  7233 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:47.117864 11996 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.121089 11828 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.123131 11403 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:47.082329 11981 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.503s	user 0.007s	sys 0.004s
I20260709 13:18:47.126356  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.097867  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.162935  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2281) took 1048 ms. Trace:
I20260709 13:18:47.163630  6518 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:47.097312 11277 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:47.178267  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.182180  6519 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
W20260709 13:18:47.128608  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 55665.8us
I20260709 13:18:47.130012  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.124756  6513 rpcz_store.cc:276] 0709 13:18:45.473752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.473872 (+   120us) service_pool.cc:224] Handling call
0709 13:18:45.474124 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.652110 (+177986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.652149 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.652203 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:45.652249 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.124168 (+1471919us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400354b54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:47.124211 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.124321 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408771328}
I20260709 13:18:47.133792  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.134447  6517 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:47.134663  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.135943  6516 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:47.190869  7232 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:47.191939  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.157164 11579 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.152387  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.095052 11854 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.085484 11719 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.155464 11903 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:47.120368 11885 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:47.117609 11910 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.163564  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2686) took 1409 ms. Trace:
I20260709 13:18:47.155555 11952 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:47.570987  6520 rpcz_store.cc:276] 0709 13:18:45.753847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.126072 (+1372225us) service_pool.cc:224] Handling call
0709 13:18:47.163508 (+ 37436us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:47.572479  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.039059 11889 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.162811 11403 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:47.128520 11695 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:47.175550  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2449) took 1699 ms. Trace:
I20260709 13:18:47.172818 11828 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.163439  6668 rpcz_store.cc:276] 0709 13:18:46.112786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.112920 (+   134us) service_pool.cc:224] Handling call
0709 13:18:47.069469 (+956549us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ceee 0000000000375939 00007f04bdfd01e0 00007f04be0990c4 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:18:47.161341 (+ 91872us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2513536}
I20260709 13:18:47.175951 11947 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:47.178218 11951 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.694s	user 0.005s	sys 0.006s
I20260709 13:18:47.163551  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.170897  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.576573  7233 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:47.576301 11951 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:47.576795  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 413289us
I20260709 13:18:47.577425  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:47.578320 11951 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:47.583230  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.584667  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.585381  6667 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
W20260709 13:18:47.169224  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 135057us
I20260709 13:18:47.587778  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.589331 11660 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.136678  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.054239 11979 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:47.219484 12000 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:18:47.111214 11859 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.376204 11984 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:47.128377 11909 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.380172  6669 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:47.380867  6519 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:18:47.598516  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:47.598834  6519 pending_rounds.cc:85] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 9
I20260709 13:18:47.599160  6519 pending_rounds.cc:107] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:18:47.599395  6519 raft_consensus.cc:2889] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:47.384315  6522 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:47.340078 11533 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:47.501540  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1514) took 1063 ms. Trace:
I20260709 13:18:47.450347 11741 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.499821 11832 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:47.517541  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.529750  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.221014  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:47.608165  6588 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
W20260709 13:18:47.367753  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 184276us
I20260709 13:18:47.246439 11868 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:47.541292 11870 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:47.501091 11719 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:47.546947  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 417003us
I20260709 13:18:47.614972  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.615101  6522 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:47.235327  7236 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:47.371855  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 191991us
I20260709 13:18:47.371196  7234 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 9.
I20260709 13:18:47.370884  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:47.616274  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:47.376631  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1354) took 2288 ms. Trace:
I20260709 13:18:47.230809 11996 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.149s
I20260709 13:18:47.598584 11942 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.574536 11889 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:47.602589 11867 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.575073 11695 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.631054 11867 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.181082  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2620) took 1050 ms. Trace:
I20260709 13:18:47.578517  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.634016  7233 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:47.563862  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1370) took 2214 ms. Trace:
I20260709 13:18:47.580399  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.578832 11907 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.581326  6662 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:47.574666 11403 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:47.582669  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.634999  7237 rpcz_store.cc:276] 0709 13:18:45.349672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.443978 (+ 94306us) service_pool.cc:224] Handling call
0709 13:18:45.444311 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.666001 (+221690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.666031 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.666074 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:45.666103 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.563530 (+1897427us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540028cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:47.563591 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.563786 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509401984}
I20260709 13:18:47.582962  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.638983  6671 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:47.620208  6592 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:18:47.575042  6521 rpcz_store.cc:276] 0709 13:18:45.476719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.476853 (+   134us) service_pool.cc:224] Handling call
0709 13:18:45.477142 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.672793 (+195651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.672831 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.672875 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:45.672905 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.175317 (+1502412us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400216bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:47.175366 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.175493 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449531264}
I20260709 13:18:47.591393  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.592047  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.640674  6664 raft_consensus.cc:2368] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12.
I20260709 13:18:47.641162  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:47.582652  6520 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:47.592826  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:47.595903  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 459307us
I20260709 13:18:47.602890  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1374) took 1135 ms. Trace:
I20260709 13:18:47.642235  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.639891  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:47.602885  7228 rpcz_store.cc:276] 0709 13:18:46.438402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.588606 (+150204us) service_pool.cc:224] Handling call
0709 13:18:46.589030 (+   424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:47.129469 (+540439us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:47.129513 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:47.130250 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 13:18:47.461978 (+331728us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:47.462042 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.462144 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:47.501251 (+ 39107us) raft_consensus.cc:1685] finished
0709 13:18:47.501272 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.501467 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38914,"raft.queue_time_us":397816,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":8141,"thread_start_us":392443,"threads_started":2,"wal-append.queue_time_us":328843}
I20260709 13:18:47.612282 11732 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.595189 11953 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:47.647806  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.647737  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:47.617666  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:47.647902  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:47.648847  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.649256  7228 raft_consensus.cc:2368] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11.
I20260709 13:18:47.597319 11885 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:47.597539 11708 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:47.596212 11660 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.654044  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.654703  7234 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:47.654964  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.630471 11859 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:47.632187  6597 rpcz_store.cc:276] 0709 13:18:46.130961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.674201 (+543240us) service_pool.cc:224] Handling call
0709 13:18:46.674505 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.674730 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:47.003404 (+328674us) log.cc:844] Serialized 14 byte log entry
0709 13:18:47.180656 (+177252us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.180721 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:47.180780 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.180884 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.181018 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":386750,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":38723,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":109,"thread_start_us":502014,"threads_started":2,"wal-append.queue_time_us":164244}
I20260709 13:18:47.578781  6666 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:47.617409  7235 rpcz_store.cc:276] 0709 13:18:45.088238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.213119 (+124881us) service_pool.cc:224] Handling call
0709 13:18:45.224805 (+ 11686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:45.225209 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.225238 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:45.225666 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:18:46.568535 (+1342869us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b18003c9700. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:46.696197 (+127662us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:46.696251 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.696357 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:47.376329 (+679972us) raft_consensus.cc:1685] finished
0709 13:18:47.376353 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.376556 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":620,"raft.queue_time_us":82,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4012,"spinlock_wait_cycles":51592576,"thread_start_us":127436,"threads_started":1,"wal-append.queue_time_us":1919779}
I20260709 13:18:47.658659  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.641336  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:47.659309  7228 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:47.659582  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1368) took 2998 ms. Trace:
I20260709 13:18:47.659973  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.611485 11533 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.561111  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:47.661906  7228 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:47.644645  6667 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:47.641997  7229 rpcz_store.cc:276] 0709 13:18:46.467807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.640071 (+172264us) service_pool.cc:224] Handling call
0709 13:18:46.670913 (+ 30842us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:46.986431 (+315518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:46.986475 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.007110 (+ 20635us) log.cc:844] Serialized 26 byte log entry
0709 13:18:47.039393 (+ 32283us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:47.039432 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.039502 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:47.602625 (+563123us) raft_consensus.cc:1685] finished
0709 13:18:47.602680 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.602797 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":315089,"raft.queue_time_us":345411,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":31320,"threads_started":1,"wal-append.queue_time_us":146945}
I20260709 13:18:47.648720  6668 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:47.627650  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.626874  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:47.662993  6591 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:47.649580  7237 raft_consensus.cc:2410] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:18:47.649926  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.645645 11854 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:47.619318 12007 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:47.626654 11575 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:47.651721  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.673393  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.627243 11945 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:47.652710  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:47.656154  6562 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:47.657966  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.676862  6663 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:47.676347  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:47.602592 11474 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.678184  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:47.660683  6662 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:47.641404  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.659934  7230 rpcz_store.cc:276] 0709 13:18:44.660683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.846528 (+185845us) service_pool.cc:224] Handling call
0709 13:18:45.170034 (+323506us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:45.177116 (+  7082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.659390 (+2482274us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b540027d8d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:47.659514 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735799552}
I20260709 13:18:47.663684  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.606238 11984 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.681208  6670 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:47.664335  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.638890 11942 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.674252  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:47.664398  7232 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:47.665133  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.638211 11870 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:18:47.665939  7229 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:47.666378  6597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:47.685635  6597 raft_consensus.cc:606] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:18:47.686023  6597 raft_consensus.cc:614] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:47.667243 11660 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.667730  6665 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:47.645671 11852 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.645750 11669 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.672230  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:47.627858 11719 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.651743 11999 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:47.673990  7235 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:47.657598 11940 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:47.708709  6591 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:47.652194 11732 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.670826 11951 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:47.651125 11966 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:18:47.713311  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:47.713910  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 61267.7us
I20260709 13:18:47.675398 11575 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:47.711746  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:47.738761  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 67170.7us
I20260709 13:18:47.686271 11579 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:47.738911  6520 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:47.740492  7234 pending_rounds.cc:85] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 8
I20260709 13:18:47.740854  7234 pending_rounds.cc:107] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:47.685333 11953 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:47.734263 11821 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:47.737331  6667 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:47.741355  7234 raft_consensus.cc:2889] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:47.741940  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.711138  6591 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:18:47.742061 11953 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.744117  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:47.738042  7198 leader_election.cc:400] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12.
I20260709 13:18:47.681309 11984 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.705072 11660 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:47.745467  7198 leader_election.cc:403] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:47.745805  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.712442 11854 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:47.755847  6667 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:47.756421  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.703523 11669 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.743402 11859 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.740078 11832 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:47.744324 11953 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:47.677350 12007 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.685485 11474 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.712625 11940 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:47.740247 11575 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:47.744936 11951 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.757472 11903 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.744560 11945 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:47.762575  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.712178 11999 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.767477 12003 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.792181  6671 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:47.705287 11852 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.750495 11660 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:47.759654 11828 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:47.754369 11854 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:47.784113  6667 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:47.786711  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:47.790015  6520 raft_consensus.cc:2410] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:47.744716 11821 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:47.803320  6597 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:47.802711  6591 raft_consensus.cc:606] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.783457 11940 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:47.769495  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 121763us
I20260709 13:18:47.795433  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:47.798806  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 613917us
I20260709 13:18:47.782249 11979 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.783083 11832 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:47.790701  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 134623us
I20260709 13:18:47.804018  6591 raft_consensus.cc:614] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:47.841602  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.843907  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:47.845053  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 166921us
I20260709 13:18:47.847321  6667 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:18:47.848500  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 53185.6us
I20260709 13:18:47.849678  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:47.852243  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:47.803655 11660 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:47.833281 11854 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:47.834059 12003 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.867221  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.833140 11821 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:47.870774  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:47.869959 11869 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:47.887221  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.877089 11889 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:47.877859  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.880168 12017 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:47.881089  6562 leader_election.cc:400] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11.
I20260709 13:18:47.881697  6667 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:47.890304  6562 leader_election.cc:403] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:18:47.890417  6671 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:47.845127 11859 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:47.891079 11741 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.897495  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:47.900944  6667 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:47.895818  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 119772us
I20260709 13:18:47.866592 11812 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:47.905686  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.918424  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.902869 11741 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:47.871461 11870 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:47.911123 11867 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.891193 11277 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:18:47.912575 11952 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:47.899047 11979 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:18:47.882207 11947 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:47.920842  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:47.925138  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:47.906451 11669 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.891750 11533 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.950657  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1373) took 2600 ms. Trace:
I20260709 13:18:47.906417 12019 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:18:47.954221  7231 rpcz_store.cc:276] 0709 13:18:45.350500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.507386 (+156886us) service_pool.cc:224] Handling call
0709 13:18:45.524760 (+ 17374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.667333 (+142573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.667364 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.667405 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:45.667435 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.950312 (+2282877us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400684cd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:47.950385 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.950562 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327427840}
I20260709 13:18:47.938942 11947 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.955606 11947 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:47.953645 11889 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:47.955680  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:47.917524 11812 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.957607  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.959873  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.961148  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.961709  7237 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:47.946257  6667 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:47.962587  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:47.962827  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:47.923074 12029 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:47.888906 12003 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.936692 11952 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:47.989239  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 119169us
I20260709 13:18:47.991748  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:47.960357 11277 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:18:47.993086  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72312.5us
I20260709 13:18:47.993436 12035 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:47.993997 12035 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:47.994481 12035 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:47.993777  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:47.996284  6562 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:47.997043  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:47.999641  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.938501 11909 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:47.922858 11854 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.003005 12027 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.964078 11889 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11.
I20260709 13:18:48.000914  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:47.935593 11996 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:18:48.064658  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:47.932253 11741 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.083544 11741 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.084057 11741 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.960752 11984 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.084514 11741 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:48.087863 11741 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.005061 11821 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.090268 11821 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.020785  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.091873  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.092968  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.093266  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.093720  6667 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:47.963982 11812 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:48.065261  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 102496us
W20260709 13:18:48.095515  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 98525.7us
I20260709 13:18:48.069895 12007 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.074817 11828 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12.
I20260709 13:18:48.076494 12035 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:47.964208 11533 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 13:18:48.078986 12022 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:18:48.080024 11903 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:48.095054  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 153811us
I20260709 13:18:47.990186 11732 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.107477  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.023566 11909 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:48.110597  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:48.111629  6519 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:18:48.000329 11952 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.020376 11277 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:48.006935 12029 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:48.141105  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.113312 12027 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:48.141645 11821 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:48.141337 11575 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.145552  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.149739 11947 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.150403 11952 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.151190 11949 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:47.947467 11832 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:48.154217  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47002.2us
I20260709 13:18:48.154798  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.114792 11885 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:48.172812  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.152042 11909 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.153724 11859 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:48.179709 11812 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.152429 11277 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:48.166474 11828 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:48.185034  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 280926us
I20260709 13:18:48.159106 12027 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.169032 11826 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.203754  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1517) took 1148 ms. Trace:
I20260709 13:18:48.204161  7229 rpcz_store.cc:276] 0709 13:18:47.055128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.690209 (+635081us) service_pool.cc:224] Handling call
0709 13:18:47.690509 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.690867 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.690909 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.697825 (+  6916us) log.cc:844] Serialized 26 byte log entry
0709 13:18:47.802958 (+105133us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:47.803009 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.803120 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.203534 (+400414us) raft_consensus.cc:1685] finished
0709 13:18:48.203554 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.203695 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3207,"raft.queue_time_us":5223,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":32475,"thread_start_us":56288,"threads_started":1,"wal-append.queue_time_us":146672}
I20260709 13:18:48.205971  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.194322  7231 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:48.196501  6520 raft_consensus.cc:2368] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
I20260709 13:18:48.170497 11999 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.175457 11949 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:48.186128 11909 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.215888 11812 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.188413 11885 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.199357 11277 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:48.210501  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.203187 12027 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:48.213132 11403 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.001s
I20260709 13:18:48.246210  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.244947 11533 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.249692 11533 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.250052 11838 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:48.250610 11838 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:48.251618 11838 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.258244  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.259903  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:48.261917 11828 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:48.262220  6513 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:48.264789  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2461) took 1153 ms. Trace:
I20260709 13:18:48.230161 11885 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:48.267930 11575 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.225962 11660 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.270954  6664 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:48.270326 11660 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.274466 12030 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:48.275633 12030 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:48.276538  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:48.279220  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:48.279601 12032 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:18:48.248833  7229 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:48.265223  6513 rpcz_store.cc:276] 0709 13:18:47.111681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.220165 (+108484us) service_pool.cc:224] Handling call
0709 13:18:47.220436 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.262779 (+1042343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.264482 (+  1703us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540007d5d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:48.264703 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395283712}
I20260709 13:18:48.261219  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.299554 11821 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:48.306972 11859 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:48.307267 11821 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.307621 11859 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.309487 11821 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.307552 11575 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.310300 11575 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.307113 11669 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:48.306200  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1376) took 1166 ms. Trace:
I20260709 13:18:48.245612 12037 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.318819  7228 rpcz_store.cc:276] 0709 13:18:47.139544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.305126 (+1165582us) service_pool.cc:224] Handling call
0709 13:18:48.305557 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.305790 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.305814 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.305861 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:48.305911 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.305985 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.306128 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:48.211696 11826 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.333272  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.333572  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.335641  7228 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:48.333101  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:48.300388 11999 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:48.340492  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.344489  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.315356 11859 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.248634 11403 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:48.360636  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.306357 11940 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.365442 12033 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:48.366029 12041 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:48.415391 12041 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.421913 12041 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:48.423027  6406 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 64b95375974b48d18c596a23c9cac75c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:48.366124  7228 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:48.428285 11869 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:48.428848 11869 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:48.429656 11869 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.367000 11889 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:48.226234 12029 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.436546  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.437188  6591 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
W20260709 13:18:48.372596  6696 heartbeater.cc:646] Failed to heartbeat to 127.5.72.190:46511 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.72.190:46511 timed out after 15.000s (SENT)
I20260709 13:18:48.374183  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:48.374373 11575 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.334426 12037 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.348809 12028 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:48.377250 11403 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.377983 12003 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.316004 11949 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.349588 11999 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.391183  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1520) took 1318 ms. Trace:
I20260709 13:18:48.391855 11826 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.392961 11885 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.393376 11947 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.393849 12035 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.337787 11854 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.436975 12014 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.405155 11909 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.406445 11990 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:18:48.408812 11953 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.000s
I20260709 13:18:48.408980 11277 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.410722 11940 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.436182 11587 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:18:48.411840 11868 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:48.395406 11867 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:18:48.413762 11952 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.430768  6589 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:48.465807  6588 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:48.432009  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.457254 11949 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:48.438349  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64240.4us
I20260709 13:18:48.444911  7235 rpcz_store.cc:276] 0709 13:18:47.072709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.696962 (+624253us) service_pool.cc:224] Handling call
0709 13:18:47.736976 (+ 40014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.883312 (+146336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.883346 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.885478 (+  2132us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.181140 (+295662us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:48.181177 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.181238 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.390903 (+209665us) raft_consensus.cc:1685] finished
0709 13:18:48.390933 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.391105 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3115,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4908,"thread_start_us":393720,"threads_started":2,"wal-append.queue_time_us":399860}
I20260709 13:18:48.324440 11669 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.441629 11859 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:48.474973 11867 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:48.459380  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.475461 11867 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:48.476500  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.476418 11867 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:48.477159  6591 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:48.477800 11867 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.478003  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.447778 11869 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.479722 11868 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:48.480127 11868 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:48.480880 11868 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:48.411576 12040 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:18:48.482169 11868 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.466423  7228 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:48.485746 11856 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.440481 12039 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:18:48.471031  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.472705 11277 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.474812 11669 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.487447  6591 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:48.507993  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.460628 12014 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.509027  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.509750  6591 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:48.509915 12014 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.498914 11859 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:48.510736 11859 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.512868 11859 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.513510 11940 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:48.514097 11940 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.516273 11940 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.516701  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.517861  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.488155  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.464432 11953 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.490355 12037 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.493563 11999 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:48.432591  6594 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.023s	sys 0.024s
W20260709 13:18:48.521284  6594 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.023s	sys 0.024s
I20260709 13:18:48.490322  7228 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:48.522428  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.471271 11979 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:48.523265  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.523538 11979 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:48.524369 11979 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.524976  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.525544 11979 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:48.526633  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.505316  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.526979 11953 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.527359  6589 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.527766  6667 raft_consensus.cc:2368] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:48.527817  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.528483  7235 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.488045 11909 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:48.529196  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:48.520573  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.509021 12030 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.531157 11953 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:48.510485  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.532018  6591 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.532900  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.533449  7237 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:48.536340 11719 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:48.537503  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2731) took 2490 ms. Trace:
I20260709 13:18:48.537889  6596 rpcz_store.cc:276] 0709 13:18:46.047220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.513715 (+466495us) service_pool.cc:224] Handling call
0709 13:18:46.514024 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.037658 (+523634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.537235 (+1499577us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b5400384ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:48.537443 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202324736}
I20260709 13:18:48.538723  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.539103  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.539268  6668 raft_consensus.cc:2376] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12. Re-sending same reply.
I20260709 13:18:48.540169  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.540230  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.541209  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.541342  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.530478  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.530339  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:48.542572  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1507) took 2111 ms. Trace:
I20260709 13:18:48.542980  7236 rpcz_store.cc:276] 0709 13:18:46.430792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.432150 (+  1358us) service_pool.cc:224] Handling call
0709 13:18:47.103888 (+671738us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:18:47.104076 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.615857 (+511781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.542357 (+926500us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54007a3d54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:48.542515 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316145280}
I20260709 13:18:48.544087  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.544577  7236 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:48.544809  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.506585  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:48.552942  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.552963 11836 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:48.554003 11999 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:48.555696 11859 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:48.558487  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:48.563369  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 60621us
I20260709 13:18:48.562232  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.566495  6671 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:48.551087  6589 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:48.549013  6562 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:48.550485  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:48.575021  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.550094  6591 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:48.564463  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:48.568463  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.553418 11719 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:48.576807  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:48.573596  7236 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:48.581187 12039 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:48.582012 12039 consensus_queue.cc:1243] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:18:48.582492 12039 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:48.578868  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.585031  7237 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:48.579581  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:48.585417  6522 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:48.587186  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.586406  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:48.567517 11836 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:48.590029  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:48.590716  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:48.590844 11856 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:48.579202 11859 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:48.591472 11856 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:48.590899  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.591889 11859 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:48.592338  6671 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:48.592442 11856 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.592839 11859 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.593247  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.588029  7237 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.589308 11669 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:48.588358 11999 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:48.595362 11999 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:48.595809 11999 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.589509 11836 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:48.596724 11836 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:48.591758  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.589677 11894 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:48.598784  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.598958 11894 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:48.599298  7236 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.600100 12041 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:48.600574 12041 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:48.599936 11894 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.600934  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.601462 12041 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.603834  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:48.604773  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.605008  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.608224  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.608958  6522 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:48.609791  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.579732 11909 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:18:48.610553 11909 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:48.610896 11909 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:48.611161  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.611677 11909 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.612838 11909 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:48.609877  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.614880  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.615316 12014 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:48.616546  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.615351  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1046) took 1024 ms. Trace:
I20260709 13:18:48.620360  6665 rpcz_store.cc:276] 0709 13:18:47.591215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.649700 (+ 58485us) service_pool.cc:224] Handling call
0709 13:18:48.615186 (+965486us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b54000e56d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:48.615296 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7672064}
I20260709 13:18:48.621445  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.615360  7233 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:48.617367  6515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 1029) took 2910 ms (client timeout 1948 ms). Trace:
I20260709 13:18:48.622433  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:48.626423  6515 rpcz_store.cc:269] 0709 13:18:45.707023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.707186 (+   163us) service_pool.cc:224] Handling call
0709 13:18:48.617137 (+2909951us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540014dcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:48.617266 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3150518528}
I20260709 13:18:48.627509  6665 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:18:48.627838  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:48.628459  6515 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:48.628868 11474 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.618039  6516 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:48.630330  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:48.630895  6521 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.632231  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.628594  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.633262  6665 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.633952  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2303) took 1039 ms. Trace:
I20260709 13:18:48.619309  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.634323  6664 rpcz_store.cc:276] 0709 13:18:47.594407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.705746 (+111339us) service_pool.cc:224] Handling call
0709 13:18:47.706100 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.271659 (+565559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.633744 (+362085us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54000cce54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:48.633896 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298342144}
I20260709 13:18:48.635111  7232 raft_consensus.cc:2410] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:48.635663  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.636404  6664 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:18:48.639247  7233 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:48.578181 11719 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:48.609018  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.641180  6667 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:48.643457  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.644810  7230 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:48.645933  7198 leader_election.cc:400] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:48.646251  7198 leader_election.cc:403] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:18:48.648918  6665 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:48.649132  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:48.650955  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.652496  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.656140  6516 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:48.569695  7235 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:48.654289  7237 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:48.658454  7237 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:48.659858  7236 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:48.660892  7236 raft_consensus.cc:1217] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Deduplicated request from leader. Original: 12.9->[14.10-14.10]   Dedup: 14.10->[]
I20260709 13:18:48.661862  7236 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:18:48.661451  7235 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:48.663095  7230 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:18:48.663717  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 73766.6us
I20260709 13:18:48.664880  6515 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:48.641116  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:48.666002  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:48.666577  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 272401us
I20260709 13:18:48.666388  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.667340  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:48.667446  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.672803  7198 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.681449  7230 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 13:18:48.689632  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:48.698511  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.704914  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.712910  6596 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:48.713039  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:48.717813  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.708793  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.707129  6521 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:18:48.719236  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.716993  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:48.668493 12014 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:48.686223 11832 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:48.686077 11956 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:48.720940  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:48.664494 11940 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:48.664364 11669 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:48.651422 11719 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.726042  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.781272  7236 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.715133 11974 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:48.710201 12037 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:48.783756 11660 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:48.733969 11970 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:48.786893 11970 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:48.789119 11970 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.722437  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:48.790169  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 67802.5us
I20260709 13:18:48.708088 11575 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:48.696990 12047 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:48.696398 11826 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:48.792698  7236 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
W20260709 13:18:48.792946  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 72095.7us
I20260709 13:18:48.741135 12019 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:48.716521 11277 raft_consensus.cc:2764] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:18:48.721823 12038 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:48.776180  6667 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:48.799744  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.741446 11885 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:48.797449  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:48.723516 11889 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:18:48.741355 12036 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:48.800148  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:48.808458  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:48.741513 11903 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:18:48.696977 12027 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.797849 12047 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.801348 12014 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:48.819379  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:48.823001  7236 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:48.814553  6521 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:48.830631  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.790707 11828 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:18:48.806106 11956 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:48.782552 11880 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:18:48.796631 11854 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:18:48.845188  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.845898 11947 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:48.846397 11947 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:48.815286 12019 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:48.846498  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.857046  6590 raft_consensus.cc:2368] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
W20260709 13:18:48.847308  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.820393 11885 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:48.858053  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 149315us
I20260709 13:18:48.861804  6521 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:48.868602  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:48.872233 11587 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:48.878247  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.879029  6667 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:48.839702 11910 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:48.879874  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.839115 12036 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:48.853469 11854 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:48.838982 12047 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:48.885481  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.863989 11956 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:48.892170 11947 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:48.893278 11956 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.877949 11679 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:18:48.898841  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.901367  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.856783 11836 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.903441  6667 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:18:48.888157 11910 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:48.908614  7228 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:48.913185  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.915019  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:48.919180 11910 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:48.923553 12045 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.416s	user 0.004s	sys 0.000s
I20260709 13:18:48.925506  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2713) took 1170 ms. Trace:
I20260709 13:18:48.925885  6519 rpcz_store.cc:276] 0709 13:18:47.755585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.197215 (+441630us) service_pool.cc:224] Handling call
0709 13:18:48.197503 (+   288us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:48.197840 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:48.209719 (+ 11879us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.433853 (+224134us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:48.458182 (+ 24329us) log.cc:844] Serialized 52 byte log entry
0709 13:18:48.458513 (+   331us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:48.458551 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.458644 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.881996 (+423352us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:48.925196 (+ 43200us) raft_consensus.cc:1685] finished
0709 13:18:48.925298 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.925445 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5860,"prepare.queue_time_us":13691,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":3110,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":8553,"spinlock_wait_cycles":23569792,"thread_start_us":160844,"threads_started":1,"wal-append.queue_time_us":280164}
I20260709 13:18:48.926935 12036 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:48.928113  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.928843  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.931430  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:48.937069 12046 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.426s	user 0.001s	sys 0.000s
I20260709 13:18:48.938232  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2464) took 1779 ms. Trace:
I20260709 13:18:48.900588 11679 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:48.938584  6517 rpcz_store.cc:276] 0709 13:18:47.158443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.319146 (+160703us) service_pool.cc:224] Handling call
0709 13:18:47.339473 (+ 20327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.360364 (+ 20891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.360414 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.266836 (+906422us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b2c000014c8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:18:48.267401 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.439550 (+172149us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:48.439591 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.440037 (+   446us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.936511 (+496474us) spinlock_profiling.cc:243] Waited 7.48 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:48.937971 (+  1460us) raft_consensus.cc:1685] finished
0709 13:18:48.937997 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.938170 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2018,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":19727,"spinlock_wait_cycles":251664512,"thread_start_us":171126,"threads_started":1,"wal-append.queue_time_us":197107}
I20260709 13:18:48.951658  6668 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:48.930454  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.959789  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.961501  6591 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:48.964490  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:48.966408  6517 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:48.968829  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.972086  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.978466  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:48.980005  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.980659  6517 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:48.981597  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.984025  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:48.984627  6667 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.984686  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:48.985395  6591 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 9.
I20260709 13:18:48.986251  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:48.986732  6591 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:48.990468  6589 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:48.971105 11947 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.027704  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.968906 11533 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:49.009320  6665 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:49.057490  6665 pending_rounds.cc:85] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 8
I20260709 13:18:49.057744  6665 pending_rounds.cc:107] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:49.057974  6665 raft_consensus.cc:2889] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:48.911954 11719 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:49.052644  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:48.915118 12019 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.069413  6521 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 10: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:48.999660  6591 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:49.039484  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2744) took 1994 ms. Trace:
W20260709 13:18:48.995733 12048 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 13:18:49.023118  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2292) took 2690 ms. Trace:
I20260709 13:18:49.003337  7229 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:49.063519  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.075788  6594 rpcz_store.cc:276] 0709 13:18:47.045149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.671193 (+626044us) service_pool.cc:224] Handling call
0709 13:18:47.671470 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.682954 (+ 11484us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:47.697733 (+ 14779us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.521621 (+823888us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.521673 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:48.521715 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.039185 (+517470us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b540019dcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.039250 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.039413 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28570,"prepare.queue_time_us":20618,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":66356,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6265,"spinlock_wait_cycles":25900416,"thread_start_us":796471,"threads_started":2,"wal-append.queue_time_us":176671,"wal-append.run_cpu_time_us":926,"wal-append.run_wall_time_us":1000998}
I20260709 13:18:49.079927  6667 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:49.032138 11951 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:49.005600 11695 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:49.082669  6666 rpcz_store.cc:276] 0709 13:18:46.332709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.374178 (+ 41469us) service_pool.cc:224] Handling call
0709 13:18:46.393263 (+ 19085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.661959 (+1268696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.022814 (+1360855us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540011e9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.023024 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2725995776}
I20260709 13:18:49.092151  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1382) took 1932 ms. Trace:
I20260709 13:18:49.016856 12039 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:49.100276  7229 rpcz_store.cc:276] 0709 13:18:47.160601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.002681 (+1842080us) service_pool.cc:224] Handling call
0709 13:18:49.003111 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.091866 (+ 88755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.092077 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:48.971576 11885 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.107108  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.120931  6594 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:49.057227 11947 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:49.072665 11996 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:49.132280  6665 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:49.120483  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.137029  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.137773  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:49.132723 12054 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 13:18:49.105594 12039 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:49.124382 12027 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:49.141148  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1044) took 1305 ms. Trace:
I20260709 13:18:49.150349  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.137828  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.157037  6513 rpcz_store.cc:276] 0709 13:18:47.835802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.306767 (+470965us) service_pool.cc:224] Handling call
0709 13:18:48.311437 (+  4670us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:48.311788 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.311817 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:48.318337 (+  6520us) log.cc:844] Serialized 52 byte log entry
0709 13:18:48.456664 (+138327us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:48.467913 (+ 11249us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.504565 (+ 36652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.504641 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.933104 (+428463us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b18003d6780. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:49.139161 (+206057us) raft_consensus.cc:1685] finished
0709 13:18:49.140945 (+  1784us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.141089 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36340,"prepare.queue_time_us":11039,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":40396,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":9992,"spinlock_wait_cycles":16662784,"thread_start_us":95668,"threads_started":1,"wal-append.queue_time_us":172230}
W20260709 13:18:49.148605 11868 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.005s
I20260709 13:18:49.170392  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.107671 11679 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.188848  6595 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:49.186559  6522 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:49.190186  6595 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:49.188901  6591 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:49.192061  6591 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:18:49.193500  6591 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:18:49.171306 12029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.012s	sys 0.009s
I20260709 13:18:49.195919  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56ba1578b97b4faa966168899769e067"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:49.198307  6594 raft_consensus.cc:606] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.198717  6594 raft_consensus.cc:614] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:49.199419 12061 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:18:49.199658 12062 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:18:49.199913  6594 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:49.188189 12057 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.191s	user 0.004s	sys 0.001s
I20260709 13:18:49.211658  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
W20260709 13:18:49.215852  7231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1389) took 1447 ms (client timeout 1876 ms). Trace:
I20260709 13:18:49.198326  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:49.141593 11996 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.207050  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2473) took 1005 ms. Trace:
I20260709 13:18:49.229278  6518 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: [9.8-12.9]
W20260709 13:18:49.204098  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.219009  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2747) took 2173 ms. Trace:
I20260709 13:18:49.256239  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:49.257925  6597 rpcz_store.cc:276] 0709 13:18:47.046037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.687782 (+641745us) service_pool.cc:224] Handling call
0709 13:18:47.688135 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.803725 (+115590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.218767 (+1415042us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002fbcd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.218951 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264441088}
W20260709 13:18:49.256997  7231 rpcz_store.cc:269] 0709 13:18:47.768947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.955405 (+186458us) service_pool.cc:224] Handling call
0709 13:18:49.215668 (+1260263us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540051d1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.215788 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417482496}
I20260709 13:18:49.256242  6515 rpcz_store.cc:276] 0709 13:18:48.201273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.665658 (+464385us) service_pool.cc:224] Handling call
0709 13:18:48.666075 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.666393 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:48.722084 (+ 55691us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.854959 (+132875us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.855516 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.855666 (+   150us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:48.855696 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.855779 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.206856 (+351077us) raft_consensus.cc:1685] finished
0709 13:18:49.206876 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.206998 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19892,"prepare.queue_time_us":78906,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3129,"raft.queue_time_us":2192,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":43693,"thread_start_us":56277,"threads_started":1,"wal-append.queue_time_us":192040}
I20260709 13:18:49.261001  7231 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:49.259126  7228 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:49.267923  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2707) took 2216 ms. Trace:
W20260709 13:18:49.197371 12029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.013s	sys 0.009s
I20260709 13:18:49.273936 12014 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:49.275003 12014 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:49.203794 12071 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:49.196133 11836 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:49.217675 11852 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:49.214232 11947 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:49.281409  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 426648us
I20260709 13:18:49.204545 11679 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.218080 11907 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:49.302814 11907 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:49.303702 11907 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.304868 11907 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:49.305478  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 1040) took 1545 ms. Trace:
I20260709 13:18:49.305859  6520 rpcz_store.cc:276] 0709 13:18:47.759691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.924909 (+165218us) service_pool.cc:224] Handling call
0709 13:18:49.305242 (+1380333us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54003f7fd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.305394 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573001472}
I20260709 13:18:49.207223 11441 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.211256 11719 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:49.259477  6594 raft_consensus.cc:606] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.220670 11832 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:49.285104 12054 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:18:49.299335 12073 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:18:49.268709  6518 rpcz_store.cc:276] 0709 13:18:47.051782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.179060 (+127278us) service_pool.cc:224] Handling call
0709 13:18:49.219774 (+2040714us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b5800010598. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ceee 0000000000375939 00007f04bdfd01e0 00007f04be0990c4 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:18:49.219920 (+   146us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:49.267610 (+ 47690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.267864 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30715904}
I20260709 13:18:49.310962  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.314090  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.317637  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2455) took 2279 ms. Trace:
I20260709 13:18:49.318133  6514 rpcz_store.cc:276] 0709 13:18:47.034308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.062363 (+ 28055us) service_pool.cc:224] Handling call
0709 13:18:47.062688 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.081359 (+ 18671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.309062 (+2227703us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b54003bbd54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.313144 (+  4082us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41249536}
I20260709 13:18:49.323611  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.331033  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.329092  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:49.338145  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:49.270648 12039 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.317394 11595 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:49.339838 11573 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:18:49.345239 11836 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:49.348879 11996 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:49.349030 11836 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:49.350041 11836 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:49.354187 11999 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.001s
I20260709 13:18:49.330927 11952 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:49.340693 12084 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.334779 11947 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:49.370862  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.393182  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.400456  7236 pending_rounds.cc:85] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 7
I20260709 13:18:49.400825  7236 pending_rounds.cc:107] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:18:49.401064  7236 raft_consensus.cc:2889] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:49.401427 12047 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 13:18:49.401938 12047 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:49.403469  6596 raft_consensus.cc:2368] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15.
I20260709 13:18:49.407058 11852 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:49.407625 11852 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.409750 11852 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:49.410575  7228 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:49.376122 11947 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:49.411830  7228 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:49.412439 11947 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.414600  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1057) took 1516 ms. Trace:
I20260709 13:18:49.414981  6664 rpcz_store.cc:276] 0709 13:18:47.898128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.657109 (+758981us) service_pool.cc:224] Handling call
0709 13:18:48.657404 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.657634 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:48.869644 (+212010us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.072041 (+202397us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.072358 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.414100 (+341742us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b18003cdd20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:49.414248 (+   148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:49.414292 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:49.414331 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.414389 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.414547 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":437,"prepare.queue_time_us":570448,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":1424,"raft.queue_time_us":200,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":18286,"spinlock_wait_cycles":69494528,"thread_start_us":413586,"threads_started":2,"wal-append.queue_time_us":266989}
W20260709 13:18:49.422281  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.423039  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 270656us
I20260709 13:18:49.423511  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1393) took 1522 ms. Trace:
I20260709 13:18:49.423791  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2696) took 1518 ms. Trace:
W20260709 13:18:49.423941  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.423892  7234 rpcz_store.cc:276] 0709 13:18:47.900876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.991505 (+ 90629us) service_pool.cc:224] Handling call
0709 13:18:49.423316 (+1431811us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400692dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.423457 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2995505408}
W20260709 13:18:49.424535  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.424185  6671 rpcz_store.cc:276] 0709 13:18:47.914517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.006466 (+1091949us) service_pool.cc:224] Handling call
0709 13:18:49.006873 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.007141 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:49.151229 (+144088us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.423315 (+272086us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.423481 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:49.423524 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:49.423567 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.423633 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.423743 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195311,"prepare.queue_time_us":84630,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":12732,"raft.queue_time_us":196939,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":219871,"threads_started":2,"wal-append.queue_time_us":144184}
I20260709 13:18:49.432945  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.437063  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.438750  6488 leader_election.cc:400] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15.
I20260709 13:18:49.439114  6488 leader_election.cc:403] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:18:49.445930  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:49.355631 11999 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.462443  6514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "3ceee54e49d041be9f7f48a2052709c5"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:18:49.463171  6514 raft_consensus.cc:606] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Received request to transfer leadership to TS 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:49.463632  6514 raft_consensus.cc:614] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:49.444459 11954 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:49.468315 11586 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:49.469130 11586 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:49.469475 11586 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:49.406525 12047 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:49.371124 12084 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.474639  7228 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:49.447309 11573 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.462473 11956 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.467350 11868 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:49.464995 11714 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.484782  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:49.487524  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:49.492815  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:49.471909 11954 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:49.502106  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1161) took 1147 ms. Trace:
I20260709 13:18:49.502729  6590 rpcz_store.cc:276] 0709 13:18:48.336345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.344205 (+  7860us) service_pool.cc:224] Handling call
0709 13:18:49.483628 (+1139423us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54002f71d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.483811 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176428288}
I20260709 13:18:49.494105  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:49.505087  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:49.467125 11885 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:18:49.506012 11695 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:49.471374 11952 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:18:49.470291 11586 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.510391  7200 leader_election.cc:400] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 9 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10.
I20260709 13:18:49.480158 11579 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.527666  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:49.527716  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:49.486028 11868 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:49.502377 11996 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.500033 11940 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.507944 11836 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:49.540750  6514 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:49.541565  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:49.508767 12029 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:49.537482 11868 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:49.506481 11956 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:49.562054  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2743) took 2513 ms. Trace:
I20260709 13:18:49.546789 12029 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.562569  6592 rpcz_store.cc:276] 0709 13:18:47.044800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.046849 (+  2049us) service_pool.cc:224] Handling call
0709 13:18:49.558001 (+2511152us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54002fd5d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.558151 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292090624}
I20260709 13:18:49.550263 11586 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:49.537534 11579 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:49.581110  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 102754us
I20260709 13:18:49.582774  6517 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:49.604749  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:49.567533 12029 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:49.612622  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:49.614578  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:49.617173  6593 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:49.620867  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:49.569455 11996 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:18:49.605618 12084 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:49.560844 11954 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.631165  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1066) took 1054 ms. Trace:
I20260709 13:18:49.631696  6664 rpcz_store.cc:276] 0709 13:18:48.576656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.416643 (+839987us) service_pool.cc:224] Handling call
0709 13:18:49.416919 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:49.417239 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:49.417531 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.519161 (+101630us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:49.531605 (+ 12444us) log.cc:844] Serialized 26 byte log entry
0709 13:18:49.531796 (+   191us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:49.531844 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.531934 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.630883 (+ 98949us) raft_consensus.cc:1685] finished
0709 13:18:49.630910 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.631083 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":170,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":13908,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":8610,"thread_start_us":92378,"threads_started":1,"wal-append.queue_time_us":156916}
I20260709 13:18:49.634389  6664 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:49.636957  6521 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:49.638264  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 93929.2us
I20260709 13:18:49.640719  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2681) took 2048 ms. Trace:
I20260709 13:18:49.641081  6663 rpcz_store.cc:276] 0709 13:18:47.591979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.677617 (+ 85638us) service_pool.cc:224] Handling call
0709 13:18:47.677930 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.754764 (+ 76834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.754792 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.754828 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:47.754859 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.640524 (+1885665us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b54000e5bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.640566 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.640670 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176231168}
I20260709 13:18:49.645717  6667 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:49.651649 12041 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:49.652060 12041 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:49.652884 12041 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.653894 12041 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.660645  7229 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:18:49.664846 12074 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.459s	user 0.000s	sys 0.001s
W20260709 13:18:49.668107 12079 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.456s	user 0.001s	sys 0.000s
W20260709 13:18:49.670519 12076 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.468s	user 0.000s	sys 0.002s
I20260709 13:18:49.675824  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2722) took 1076 ms. Trace:
I20260709 13:18:49.676175  6522 rpcz_store.cc:276] 0709 13:18:48.598961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.631968 (+ 33007us) service_pool.cc:224] Handling call
0709 13:18:49.675649 (+1043681us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54003ccb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.675749 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273034496}
I20260709 13:18:49.678292 11966 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.688565 11966 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:49.690949 12069 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.495s	user 0.000s	sys 0.002s
I20260709 13:18:49.691023  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2287) took 3564 ms. Trace:
I20260709 13:18:49.691480  6669 rpcz_store.cc:276] 0709 13:18:46.127126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.127263 (+   137us) service_pool.cc:224] Handling call
0709 13:18:46.132661 (+  5398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.598441 (+1465780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.690790 (+2092349us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54001cb1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.690964 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2867980672}
W20260709 13:18:49.695741  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.697124  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.682646 11990 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.698652  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2296) took 2105 ms. Trace:
I20260709 13:18:49.698984 11990 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:49.701206 11990 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:49.699159  6662 rpcz_store.cc:276] 0709 13:18:47.593299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.680131 (+ 86832us) service_pool.cc:224] Handling call
0709 13:18:48.615456 (+935325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.615847 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.615873 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.616327 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.825326 (+208999us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:48.825378 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.825464 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.894692 (+ 69228us) raft_consensus.cc:1685] finished
0709 13:18:48.894712 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.698350 (+803638us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54000e56d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.698572 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4014,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":2150166784,"thread_start_us":208612,"threads_started":1,"wal-append.queue_time_us":261463}
W20260709 13:18:49.702988 12072 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.499s	user 0.000s	sys 0.002s
I20260709 13:18:49.704449  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1542) took 1209 ms. Trace:
I20260709 13:18:49.705018  7237 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:18:49.706097 12080 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.500s	user 0.001s	sys 0.000s
I20260709 13:18:49.706542  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:49.708034 12077 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.504s	user 0.002s	sys 0.000s
I20260709 13:18:49.708135  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2474) took 1478 ms. Trace:
I20260709 13:18:49.709903  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2298) took 2116 ms. Trace:
W20260709 13:18:49.710418  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.710321  6670 rpcz_store.cc:276] 0709 13:18:47.593648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.681884 (+ 88236us) service_pool.cc:224] Handling call
0709 13:18:47.698293 (+ 16409us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:47.702066 (+  3773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.702111 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:47.712486 (+ 10375us) log.cc:844] Serialized 78 byte log entry
0709 13:18:49.033901 (+1321415us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b18003b47c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:49.171014 (+137113us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:49.171056 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.171223 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.705515 (+534292us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:49.709691 (+  4176us) raft_consensus.cc:1685] finished
0709 13:18:49.709712 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.709848 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3339,"raft.queue_time_us":46863,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":5834,"spinlock_wait_cycles":63175296,"thread_start_us":135698,"threads_started":1,"wal-append.queue_time_us":1462393}
I20260709 13:18:49.708662  6516 rpcz_store.cc:276] 0709 13:18:48.229780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.701401 (+471621us) service_pool.cc:224] Handling call
0709 13:18:48.701798 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.709893 (+  8095us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:48.831320 (+121427us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.686147 (+854827us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b18004dbe40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a
0709 13:18:49.686597 (+   450us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.686793 (+   196us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:49.687310 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:18:49.687463 (+   153us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:49.687512 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.687595 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.707886 (+ 20291us) raft_consensus.cc:1685] finished
0709 13:18:49.707911 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.708071 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20192,"prepare.queue_time_us":120769,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":43208,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":4668,"spinlock_wait_cycles":11371776,"thread_start_us":62492,"threads_started":2,"wal-append.queue_time_us":50176}
I20260709 13:18:49.727658  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2619) took 3638 ms. Trace:
I20260709 13:18:49.728081  6593 rpcz_store.cc:276] 0709 13:18:46.089483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.586606 (+497123us) service_pool.cc:224] Handling call
0709 13:18:46.635656 (+ 49050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.617631 (+2981975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.727400 (+109769us) spinlock_profiling.cc:243] Waited 2.98 s on lock 0x7b54003a2354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.727593 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6855778560}
I20260709 13:18:49.731115 11942 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:49.704808  7235 rpcz_store.cc:276] 0709 13:18:48.494955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.663137 (+168182us) service_pool.cc:224] Handling call
0709 13:18:48.707949 (+ 44812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.820334 (+112385us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:48.839297 (+ 18963us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.839511 (+   214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.839559 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:48.839602 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.018461 (+178859us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b18003c9640. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:49.704249 (+685788us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540051aed4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:49.704294 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.704397 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3303,"prepare.queue_time_us":22678,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":4567,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":2972,"spinlock_wait_cycles":266678400}
I20260709 13:18:49.737789 11838 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.755975  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:49.757565  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.760659 11403 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:49.761358 11403 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:49.762806 11994 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:49.763734 11994 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:49.764564  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2749) took 1399 ms. Trace:
I20260709 13:18:49.765054  6588 rpcz_store.cc:276] 0709 13:18:48.365481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.063272 (+697791us) service_pool.cc:224] Handling call
0709 13:18:49.764312 (+701040us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400080a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.764487 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1243876096}
I20260709 13:18:49.765786 11579 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:49.767916 11403 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:49.769573  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:49.771148  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:49.772053 11714 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:49.772549 11714 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:49.772987 11714 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:49.778842 11403 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:49.783035 11966 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:49.785296  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.790563  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:49.798499  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2704) took 1006 ms. Trace:
I20260709 13:18:49.798971  6664 rpcz_store.cc:276] 0709 13:18:48.792455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.633825 (+841370us) service_pool.cc:224] Handling call
0709 13:18:49.634174 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.798266 (+164092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.798441 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:49.800191  6668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1061) took 1513 ms (client timeout 1866 ms). Trace:
W20260709 13:18:49.800597  6668 rpcz_store.cc:269] 0709 13:18:48.287089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.542018 (+254929us) service_pool.cc:224] Handling call
0709 13:18:49.800038 (+1258020us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54003f5f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:49.800138 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771800960}
I20260709 13:18:49.802445  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:49.620954 11999 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:49.812078  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:49.815835  6668 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:49.816809  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:49.812470  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:49.818784  6664 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:49.821003  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:49.821662  6588 raft_consensus.cc:606] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:49.826710  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:49.802307 11579 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.822603 11277 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:49.840205  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:18:49.826923 12078 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.623s	user 0.004s	sys 0.001s
I20260709 13:18:49.808854 11942 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:18:49.880241  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 274259us
I20260709 13:18:49.844415 11277 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:49.833055 11966 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:18:49.841115 11579 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:49.929842 12046 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:18:49.934283 11277 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:49.950186  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 208417us
I20260709 13:18:49.952761 11573 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15.
W20260709 13:18:49.956837 12087 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
I20260709 13:18:49.962975  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2631) took 1250 ms. Trace:
I20260709 13:18:49.971072  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:49.971642  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:49.963454  6591 rpcz_store.cc:276] 0709 13:18:48.712583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.194226 (+481643us) service_pool.cc:224] Handling call
0709 13:18:49.194492 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.194721 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:49.195084 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.673830 (+478746us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b180044a0a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:49.962567 (+288737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:49.962649 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:49.962703 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.962772 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.962893 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":118,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":18612,"spinlock_wait_cycles":4324096,"thread_start_us":288425,"threads_started":1}
I20260709 13:18:49.977098  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1530) took 1557 ms. Trace:
I20260709 13:18:49.985423  6517 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:49.987221  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:49.978376  7233 rpcz_store.cc:276] 0709 13:18:48.419962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.640876 (+220914us) service_pool.cc:224] Handling call
0709 13:18:48.653117 (+ 12241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.653426 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:48.708276 (+ 54850us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.866371 (+158095us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.870205 (+  3834us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.396147 (+525942us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b1800394ea0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:49.732773 (+336626us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b1800394ea0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:49.739257 (+  6484us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b1800394ea0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:49.976798 (+237541us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:49.976845 (+    47us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:49.976886 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.976946 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.977045 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19381,"prepare.queue_time_us":662995,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":1870,"raft.queue_time_us":12697,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":8536,"spinlock_wait_cycles":31590272,"thread_start_us":385405,"threads_started":3,"wal-append.queue_time_us":203824}
W20260709 13:18:49.973887 12089 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 13:18:50.002400  7235 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:49.996187  7237 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:49.998862  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:49.999344  7233 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:49.995148 11956 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:50.013754  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 183618us
I20260709 13:18:50.014989  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:50.016369  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:50.028440  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 144788us
I20260709 13:18:50.038378  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:49.998202 11838 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.976567 12019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.011s	sys 0.007s
I20260709 13:18:49.988804 11940 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:49.976933 12030 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:18:50.006824 11832 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:49.980659 11951 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:18:49.990147 11702 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
W20260709 13:18:50.081634 12019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.012s	sys 0.007s
W20260709 13:18:50.131400  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 1.56295e+06us
W20260709 13:18:50.132439  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.133839  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:50.119220 11966 consensus_queue.cc:1243] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:18:50.137295  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 57606.2us
W20260709 13:18:50.179991  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46225.2us
I20260709 13:18:50.180727  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:50.153925  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:50.093914 12089 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:50.089197 12095 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:50.082448 11867 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:18:50.118150 11502 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:50.137439 11586 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.003s
I20260709 13:18:50.216881  6515 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:50.227272  6518 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:50.124945 12098 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:50.083742 11838 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:50.187083 12071 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.003s
I20260709 13:18:50.221347  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:50.229911  6488 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.021s	sys 0.041s
W20260709 13:18:50.233810  6488 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.021s	sys 0.041s
I20260709 13:18:50.231401  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:50.137089 12102 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:50.212913 11885 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:50.215870 11999 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.229364 11669 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.251751  7237 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:50.220875 11832 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.214488 11940 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.230291 12098 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.248174 12102 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:50.253990 12101 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:18:50.220006 12103 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:18:50.249424 12083 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.399133 12099 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:18:50.480573 11714 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.707s	user 0.007s	sys 0.003s
W20260709 13:18:50.479004 12104 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
W20260709 13:18:50.485260 12029 consensus_meta.cc:287] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.864s	user 0.002s	sys 0.006s
I20260709 13:18:50.322916 11885 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:50.230129 11502 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:50.475091 12115 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:18:50.326930 11579 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:50.400195  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 219508us
I20260709 13:18:50.655829  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:50.327334 12107 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 13:18:50.513715 11695 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.981s	user 0.010s	sys 0.001s
I20260709 13:18:50.464416  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2320) took 1935 ms. Trace:
W20260709 13:18:50.474295 11868 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.907s	user 0.005s	sys 0.003s
I20260709 13:18:50.335083 11940 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.322405 12110 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:18:50.222465 12095 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.337774 12102 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:50.461318 12108 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:18:50.322039 11669 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.479740  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2707) took 1642 ms. Trace:
I20260709 13:18:50.319131 11999 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:50.517217 12041 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.863s	user 0.009s	sys 0.005s
I20260709 13:18:50.436012  6520 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:50.326342 11832 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.328336 12098 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:50.523252 11838 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:50.653275  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 413119us
I20260709 13:18:50.705919 11996 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.548759 12064 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.522594 11277 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.569s	user 0.003s	sys 0.003s
I20260709 13:18:50.548861 12030 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.657613  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:50.548693 11979 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:50.720777  6663 rpcz_store.cc:276] 0709 13:18:48.837167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.643014 (+805847us) service_pool.cc:224] Handling call
0709 13:18:49.643272 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:49.643586 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:49.643610 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:49.644015 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 13:18:50.094128 (+450113us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:50.094169 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.094251 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:50.479541 (+385290us) raft_consensus.cc:1685] finished
0709 13:18:50.479559 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.479687 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":449754,"threads_started":1,"wal-append.queue_time_us":755200}
I20260709 13:18:50.653100 12019 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.674340 11579 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:18:50.717310 11660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.013s	sys 0.059s
W20260709 13:18:50.711669 12099 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:18:50.698510 11695 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:50.684823  6665 rpcz_store.cc:276] 0709 13:18:48.528659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.398358 (+869699us) service_pool.cc:224] Handling call
0709 13:18:49.398710 (+   352us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:49.463318 (+ 64608us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:49.480337 (+ 17019us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.683677 (+203340us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b18001cc600. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:49.993367 (+309690us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:50.004043 (+ 10676us) log.cc:844] Serialized 78 byte log entry
0709 13:18:50.004191 (+   148us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:50.004518 (+   327us) log.cc:844] Serialized 14 byte log entry
0709 13:18:50.016423 (+ 11905us) log.cc:844] Serialized 14 byte log entry
0709 13:18:50.016866 (+   443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.016993 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:50.464151 (+447158us) raft_consensus.cc:1685] finished
0709 13:18:50.464176 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.464341 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64187,"prepare.queue_time_us":19402,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":116,"raft.queue_time_us":68602,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":60507,"spinlock_wait_cycles":26558464,"thread_start_us":309029,"threads_started":1,"wal-append.queue_time_us":727025}
I20260709 13:18:50.564998 11714 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:50.562323 12083 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:50.738255  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 80711.8us
I20260709 13:18:50.740707 11885 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:50.781111 11885 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:50.651918 12029 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.705190 11868 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:50.766086  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.767031 12129 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.768714  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 1061) took 1242 ms. Trace:
W20260709 13:18:50.763957 11660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.013s	sys 0.060s
I20260709 13:18:50.780377  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:50.784153  6520 rpcz_store.cc:276] 0709 13:18:49.526257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.526403 (+   146us) service_pool.cc:224] Handling call
0709 13:18:50.768517 (+1242114us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400044054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:50.768649 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1575913856}
I20260709 13:18:50.736194 11575 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.739023 11838 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.790148 12122 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:50.790439 11869 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:50.790803 12041 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:50.790957 12122 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:50.742035 11277 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.781631 11885 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:50.798717 12121 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:50.808547 11502 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:50.801345 12092 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.174s	user 0.000s	sys 0.006s
I20260709 13:18:50.773661 12019 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:50.811290 11832 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:50.812511 12019 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:50.812788 11999 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:50.813740 12093 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.185s	user 0.009s	sys 0.009s
I20260709 13:18:50.814990 11573 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.815467 11573 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:50.817389 11573 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:50.817898 11573 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.741914 12064 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:50.820071  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2485) took 1234 ms. Trace:
I20260709 13:18:50.820518 12064 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:50.821126  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1549) took 1718 ms. Trace:
I20260709 13:18:50.788411 12129 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.823194 12129 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:50.824546 12094 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.197s	user 0.016s	sys 0.005s
I20260709 13:18:50.825160 12041 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:50.827600  6590 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:50.828202  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:50.828821  6590 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:50.828972  6589 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:50.829710  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:50.830417  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:50.830812  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:50.830976  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:50.831590  6592 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:50.831774  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:50.832357  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.832428  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:50.832685  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:50.833313  6593 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:50.833811 11990 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.835182  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:50.835950 12103 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.836376 11852 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:50.836473 12103 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:50.836958 11852 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:50.837571  6593 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:50.837787 11852 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.792532 12127 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:50.803267 12096 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.168s	user 0.012s	sys 0.006s
I20260709 13:18:50.805439 12095 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:50.794816 11869 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:50.831084  6591 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:50.834312 11403 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:50.806914 11669 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:50.760726 12030 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.801869 11277 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:50.841567  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:50.852152  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:50.884409  6561 connection.cc:256] RPC call timeout handler was delayed by -1.7301088300000629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:18:50.884722  6521 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:50.821499  7236 rpcz_store.cc:276] 0709 13:18:49.102557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.399938 (+297381us) service_pool.cc:224] Handling call
0709 13:18:49.400270 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:49.401752 (+  1482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:49.401783 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:49.402303 (+   520us) log.cc:844] Serialized 52 byte log entry
0709 13:18:49.575275 (+172972us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:49.575325 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.575390 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:50.813140 (+1237750us) spinlock_profiling.cc:243] Waited 7.58 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:50.820806 (+  7666us) raft_consensus.cc:1685] finished
0709 13:18:50.820829 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.821069 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":17440000,"thread_start_us":172629,"threads_started":1,"wal-append.queue_time_us":225814}
I20260709 13:18:50.838891  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:50.792083 12122 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:50.834492  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:50.842005  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:50.916457  6519 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:50.855741  6518 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:18:50.916890  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75387.3us
I20260709 13:18:50.820434  6517 rpcz_store.cc:276] 0709 13:18:49.585221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.620586 (+ 35365us) service_pool.cc:224] Handling call
0709 13:18:50.819916 (+1199330us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54002161d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:50.820012 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767268096}
I20260709 13:18:50.917649  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:50.751551 11996 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.765133 11979 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.875332  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2702) took 2083 ms. Trace:
I20260709 13:18:50.923768  6596 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:50.818332 11573 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.843961 12019 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.915448 12122 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:50.867851 11502 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.845320 12126 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.838586 12103 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:50.926344  6671 rpcz_store.cc:276] 0709 13:18:48.792018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.428962 (+636944us) service_pool.cc:224] Handling call
0709 13:18:49.429255 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:49.429609 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:49.429638 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:49.430596 (+   958us) log.cc:844] Serialized 26 byte log entry
0709 13:18:49.559275 (+128679us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:49.559312 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.559375 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:50.187571 (+628196us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:50.824246 (+636675us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b240021b5e8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5e1a9c4 00007f04b5e18acb 00007f04b5fd3dc9 00007f04b5fd3a55 00007f04b5fd309d 00007f04b8c0510e 00007f04b8c1c88c 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221
0709 13:18:50.875073 (+ 50827us) raft_consensus.cc:1685] finished
0709 13:18:50.875101 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.875245 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":600,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":431620608,"thread_start_us":128369,"threads_started":1,"wal-append.queue_time_us":195444}
I20260709 13:18:50.867592 11940 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:50.839392 11867 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:50.843082 11910 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.936496  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:50.937052  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:50.839783 12123 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:50.835217 11990 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.924799 11669 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:50.922864 11942 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:50.939603 12130 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:50.953997 11942 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.954869 12041 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.955369 12041 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.956346 11942 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:50.798007 11575 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.956746 12083 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:50.957490 12041 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:50.797868 11838 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.960352 11838 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:50.957439 12083 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:50.961087  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1062) took 1434 ms. Trace:
I20260709 13:18:50.961443 12083 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.961597  6515 rpcz_store.cc:276] 0709 13:18:49.526727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.526868 (+   141us) service_pool.cc:224] Handling call
0709 13:18:49.527232 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.227121 (+699889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.960801 (+733680us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54003f7fd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:50.961017 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1585580800}
I20260709 13:18:50.951526  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:50.963151  7236 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:50.963896 12029 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:50.954242 12114 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:18:50.932796 11856 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:50.915249 11403 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.968093  7236 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:50.915098 12127 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.947938 11867 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:50.952890 11910 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.938417 11869 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.977326  6516 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:18:50.972249 11954 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.342s	user 0.014s	sys 0.019s
W20260709 13:18:50.988472 11954 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.359s	user 0.014s	sys 0.019s
I20260709 13:18:50.964799 11940 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:50.949731 12129 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:18:50.980253  7236 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:50.986009 11869 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:50.942499 12019 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.969710 11942 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:50.955931 12030 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:50.945693 11502 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.952656 12123 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:50.948645 12126 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.006729  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:50.997243  7236 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:51.009497  7235 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.016048  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:50.995436 11940 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.011759  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:51.025796  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:50.975363 11856 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.029187  6518 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:51.000814 11277 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.001682 11575 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.048442 11868 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.045492  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:51.053748  6518 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.047757  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:51.042857  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.023257 11669 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.022888 11940 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.002192 12103 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:18:51.034291 12030 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.041587 11714 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:50.996693 12102 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.006316 11942 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:50.977250 12122 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.025976 12029 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:18:51.068977 11979 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:51.075116 12133 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:51.064558 12083 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.075336 12098 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.078430 11996 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.079718 11885 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.017539 11867 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.087160  6514 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: [14.10-16.11]
I20260709 13:18:51.051383 12116 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:18:51.048569 11277 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.090066  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.099200  6596 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:51.100566  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.091490  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.101291 11502 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.101662  6597 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:51.102890  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.103574 11942 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:51.103806  6597 raft_consensus.cc:2368] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:51.104890  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.105567 11990 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.105688  6597 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:51.092700  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.106971  6592 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:51.109364 12113 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.884s	user 0.003s	sys 0.000s
I20260709 13:18:51.094177  6591 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:51.101300  6596 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
W20260709 13:18:51.111950 12088 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.481s	user 0.003s	sys 0.002s
I20260709 13:18:51.112016 11885 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.094761  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.113674  6588 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:51.113775 12098 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:51.116460 12090 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.490s	user 0.005s	sys 0.000s
W20260709 13:18:51.118302 12117 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.680s	user 0.000s	sys 0.001s
I20260709 13:18:51.097931  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.119801  6592 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:51.124997  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.127272 12019 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:51.128188 12131 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:18:51.085600 12029 consensus_queue.cc:1243] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:51.128103 11856 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:51.119840  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2648) took 1760 ms. Trace:
I20260709 13:18:51.145298  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.132469  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.145884  6519 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:51.096539  6590 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:51.146703  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.147665  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.124882 11651 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:51.141876  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.142688  6518 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:51.143361  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.152292  6596 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:51.128080  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.151175 11651 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:51.153740  6597 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:51.145711  6594 rpcz_store.cc:276] 0709 13:18:49.359244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.359367 (+   123us) service_pool.cc:224] Handling call
0709 13:18:49.359675 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:49.360036 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:49.360065 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:49.360602 (+   537us) log.cc:844] Serialized 52 byte log entry
0709 13:18:49.508776 (+148174us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:49.508823 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.508891 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.115860 (+1606969us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:51.119515 (+  3655us) raft_consensus.cc:1685] finished
0709 13:18:51.119539 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.119777 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6402,"spinlock_wait_cycles":396100352,"thread_start_us":126964,"threads_started":1,"wal-append.queue_time_us":195558}
I20260709 13:18:51.154099 11651 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.154579  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.155073  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2634) took 2440 ms. Trace:
I20260709 13:18:51.068472 12030 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.144829  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.155947  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.148814  6590 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:51.066195 11669 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.146749  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.157191  6591 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.089792 12122 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.149133  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.160089 11669 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.101361 11910 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.155421  6595 rpcz_store.cc:276] 0709 13:18:48.714167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.196650 (+482483us) service_pool.cc:224] Handling call
0709 13:18:49.558344 (+361694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:49.558788 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:49.558828 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:49.559298 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:18:49.824260 (+264962us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:49.824295 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.824370 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:50.741826 (+917456us) raft_consensus.cc:1685] finished
0709 13:18:50.741858 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.154835 (+412977us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54002fd5d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:51.155018 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6976,"spinlock_wait_cycles":831075072,"thread_start_us":207022,"threads_started":1,"wal-append.queue_time_us":399012}
I20260709 13:18:51.130610 12102 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.104931 12123 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.167133 12119 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:51.167431  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.168355 11277 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.168854 11940 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:51.171114  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.172997  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:51.182672  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.182751  6590 raft_consensus.cc:2376] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12. Re-sending same reply.
W20260709 13:18:51.131796  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64603.3us
I20260709 13:18:51.174360  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.174996  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.183753  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.178829  6588 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:51.184204  6597 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:51.179733  6589 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.181674  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.184837  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.185382  6593 raft_consensus.cc:2376] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12. Re-sending same reply.
I20260709 13:18:51.155476 11651 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:51.171996 11966 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.192670 11966 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.202387  6591 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:51.203841  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:51.204449  6670 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.204751  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.205317  6662 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:51.210901 11869 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:51.211503 11869 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.212320 11940 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:51.212898 12102 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.213405 12102 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.213891 11869 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.215006  6668 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:51.215199  6667 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:18:51.215538 12102 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.201409 11277 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.215564  6667 pending_rounds.cc:85] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 10
I20260709 13:18:51.217620  6667 pending_rounds.cc:107] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:18:51.217852  6667 raft_consensus.cc:2889] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:51.218142  6667 pending_rounds.cc:107] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:18:51.218343  6667 raft_consensus.cc:2889] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:51.222651  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.223381  6668 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:51.223961  6670 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:51.226503  6589 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:51.227030  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.229296  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:51.229298  6670 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:51.231719 11966 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.232164  6665 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:51.232704  6665 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:51.235970  6663 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:51.236519  6663 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:51.235170  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.240708  6670 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:18:51.217370 12127 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.216585 11277 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.237097 12039 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.251032  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:51.248088  6663 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.246954  6665 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.253857  6663 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.253873  6665 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:51.256429  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:51.272264  6519 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:51.272539  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:51.280952 11586 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:51.281468 11586 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:51.282347 11586 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.283490 11586 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.272339 11856 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:51.291546  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:51.292410  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.293684  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:51.299301  6663 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:51.303071 12084 raft_consensus.cc:993] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:18:51.303655 12084 raft_consensus.cc:1081] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:18:51.287639 12038 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:18:51.313009  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:51.295286 11942 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:51.313658  6665 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:51.312361 11502 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:51.344086 11953 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.344900 11953 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.351924  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2488) took 1765 ms. Trace:
I20260709 13:18:51.352512  6521 rpcz_store.cc:276] 0709 13:18:49.586058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.637861 (+ 51803us) service_pool.cc:224] Handling call
0709 13:18:49.811015 (+173154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.922148 (+1111133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.351588 (+429440us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003a75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:51.351839 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468847488}
I20260709 13:18:51.328617 12135 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.332890 11869 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:51.359920 11869 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.360342 11869 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.361585 12095 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:51.320664 11979 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:18:51.336338 11502 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.373099  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.374246  6513 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:51.374935  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.378854  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.382081  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.383154  6591 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:51.384580  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.386412  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.389282  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:51.379637  6597 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:51.391983  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.392706  6521 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:51.404241  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.405000  6595 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:51.363270 11277 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:51.388501 12156 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:51.408658  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.415371  6520 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:51.362183 12095 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.412297  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.388785 11502 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.389489 12037 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.389236 12083 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.434089  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:51.403623 12137 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.443252  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:51.421547 12030 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.365795 12135 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.448053  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:51.445102 12134 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:51.457023  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.457767  6670 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:51.455277  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 142338us
I20260709 13:18:51.440959 12037 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.457252 12167 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.460286  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.461319  6663 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:51.461494  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.462787  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:51.461788  6662 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:51.440531 12127 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.465123  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:51.467829  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:51.477028  6665 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:51.452910 12064 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:51.479780  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:51.441992 12083 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:51.507830  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.512491 12135 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.473781 12134 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:51.473820 12160 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.455443 12122 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:51.481153 12084 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:51.481830 11852 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.446595 12137 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.529263  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.481957 11953 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:51.533758 11940 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:51.537590 12037 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:51.542835 12100 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.912s	user 0.001s	sys 0.006s
I20260709 13:18:51.517937 12095 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.480661 12030 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:18:51.480719 12127 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.475625 12167 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.548115 11852 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.550850 11940 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.553925 12084 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:51.544138 12160 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.566946 12167 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.573123 12084 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.575486 12137 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.576901 11979 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:51.577435 12127 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.579159 12083 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.579578 12159 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:51.583055 12159 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:51.583297 12121 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:51.583855 12121 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:51.584242 12159 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.584863 12121 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.582149 12119 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:51.577834 12135 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:51.591076 12121 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:51.587244 12119 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:51.588426 12135 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:51.595239 12135 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.600080 12126 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.597397 12160 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:51.610846 12119 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.612548 12135 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.613543 12119 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.653178  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:18:51.658870  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.670943  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1066) took 1445 ms. Trace:
I20260709 13:18:51.671514  6514 rpcz_store.cc:276] 0709 13:18:50.225824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.225991 (+   167us) service_pool.cc:224] Handling call
0709 13:18:50.229832 (+  3841us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:51.088445 (+858613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.670112 (+581667us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b54003f75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:51.670752 (+   640us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962277120}
I20260709 13:18:51.682197  6596 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:51.666419  6417 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 8 to 11, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 11 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:51.686697  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:51.687723  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 199244us
W20260709 13:18:51.690249  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.692736  6665 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:51.696990 11573 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.703003 11669 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.715554  6417 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 5 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 8 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.729668 11669 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.735337 12158 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:51.754439  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 286578us
W20260709 13:18:51.756235  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 64200.9us
W20260709 13:18:51.766853  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.766876 11942 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:51.768170 11403 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.768211 11942 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:51.770345 11942 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.772120 11942 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.772330 11702 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:51.770061  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:51.763204  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 275833us
I20260709 13:18:51.779450 11953 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:51.780464  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:51.781985 12083 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:51.782423 12083 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.782873 12083 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.787205 12137 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:51.787863 12137 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.788335 12137 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.782883 11852 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.794983 12019 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:51.798978 11940 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:51.800642  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:51.802044 11277 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:51.803629 12160 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:51.803501  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:51.804481 12160 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:51.805224  7230 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:51.805613  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:51.806964  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.806011 12160 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.807098 11953 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:51.820354  7236 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.822263  7236 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:51.825977 12172 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:51.827816  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:51.837657 11979 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:51.839751 12083 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.846921 12083 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.838459 11867 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:51.848172 12039 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.013s
I20260709 13:18:51.859967 11669 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.868837  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:51.876186  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.865604 11979 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:51.878566  6516 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:51.880264  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.881178  6520 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:51.881858  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.882431  7230 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:51.881244  6514 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:51.887290  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:51.888134  7235 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:51.890892 12017 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:18:51.895376  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 67763.2us
I20260709 13:18:51.895457  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:51.887650  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:51.899712  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 94356.3us
I20260709 13:18:51.900408 11838 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:51.911120 11838 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.914093 11838 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.906942 11826 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.914716 11669 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.915828 11826 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.916145 11669 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.918090 11826 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.925128  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:51.926879 11910 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:51.927433 11910 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.927834 11910 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:51.928910  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:51.937148  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.938169  6664 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:51.938709  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.939730  6670 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:51.939845  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.941227  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.941604  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.942003  6670 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:51.943534  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.943743  6662 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:51.903290 12017 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:51.945247  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.945691  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.946025  6662 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:51.938299 11910 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:51.946321 12017 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.947252  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.862041 12158 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.948745  6669 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:51.949322  6670 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:18:51.949980 12158 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:51.951406  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.952557  6670 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:18:51.953527  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:51.956103  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:51.957094 11856 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.015s
I20260709 13:18:51.957515  6417 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:51.958753  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:51.959084  6417 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.961217  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:51.962527  6662 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:51.964066 12017 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:51.973666  6662 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:18:51.966665 11579 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:51.919147 11669 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:52.010895 11579 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.014195 12137 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.971611  6417 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 12 to 13, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 13 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:52.015829  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 87105.7us
I20260709 13:18:51.966009 12084 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.018944 11573 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:52.019732 11573 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:52.019323 12084 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.020339 11573 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.975484  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.021626 11579 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.023958  6669 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:51.980686  7230 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:51.981767  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:51.982864  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:52.026099  7232 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.026279  7237 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:51.976112 11910 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:51.986706  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:51.987311 11838 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.029137 11838 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.987515  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.031042  7229 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:51.988161  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.032255  6668 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:51.988534  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:51.988484  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:52.033366  7233 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
W20260709 13:18:52.033864  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45630.7us
I20260709 13:18:51.989199 11940 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.990201  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:52.001767  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.035740 11838 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:51.970163  6416 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 330) took 35137 ms (client timeout 15000 ms). Trace:
I20260709 13:18:52.036785 11838 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:51.957684 11856 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:52.043610  6665 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:18:52.036823  6416 rpcz_store.cc:269] 0709 13:18:16.832224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.834745 (+  2521us) service_pool.cc:224] Handling call
0709 13:18:51.969887 (+35135142us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:43.056084 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:43.060351 (+  4267us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:43.060408 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:18:43.060430 (+    22us) tablet.cc:662] Decoding operations
0709 13:18:43.316452 (+256022us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:43.344548 (+ 28096us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:43.347355 (+  2807us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:18:43.424209 (+ 76854us) tablet.cc:701] Row locks acquired
0709 13:18:43.424260 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:18:43.424447 (+   187us) write_op.cc:270] Start()
0709 13:18:43.424598 (+   151us) write_op.cc:276] Timestamp: P: 1783603123424381 usec, L: 0
0709 13:18:43.424635 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:18:43.425377 (+   742us) log.cc:844] Serialized 6312 byte log entry
0709 13:18:44.233786 (+808409us) op_driver.cc:464] REPLICATION: finished
0709 13:18:44.481704 (+247918us) write_op.cc:301] APPLY: starting
0709 13:18:44.481806 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:44.492965 (+ 11159us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:44.493081 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:44.600121 (+107040us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b907e32a 00007f04b9003f14 00007f04b9003ec3 00007f04b9003e38 00007f04b9003df3 00007f04b90030dc 00007f04b902ce16 00007f04b9014cbb 00007f04b8fff39d 00007f04b9002d75
0709 13:18:45.137647 (+537526us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b180045e440. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:18:45.436204 (+298557us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b180045e440. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8dc00ec 00007f04b8dc2af7 00007f04b8dc3fb0 00007f04bcad81bb 00007f04bcb1f5d4 00007f04bc9cb8da 00007f04bc9ce3cb 00007f04bc9cdfd3 00007f04bcacce34
0709 13:18:45.540556 (+104352us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:45.540974 (+   418us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:18:45.541043 (+    69us) write_op.cc:312] APPLY: finished
0709 13:18:45.541521 (+   478us) log.cc:844] Serialized 148 byte log entry
0709 13:18:45.694531 (+153010us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:45.761229 (+ 66698us) write_op.cc:454] Released schema lock
0709 13:18:45.769669 (+  8440us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:45.832619 (+ 62950us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:45.835703 (+  3084us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1021702,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":236236,"apply.run_cpu_time_us":131377,"apply.run_wall_time_us":1341441,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70975,"num_ops":16,"prepare.queue_time_us":238437,"prepare.run_cpu_time_us":26677,"prepare.run_wall_time_us":539508,"raft.queue_time_us":240158,"raft.run_cpu_time_us":19398,"raft.run_wall_time_us":395354,"replication_time_us":792300,"spinlock_wait_cycles":68813184,"thread_start_us":638432,"threads_started":5,"wal-append.queue_time_us":382743,"wal-append.run_cpu_time_us":30411,"wal-append.run_wall_time_us":2418559}]]}
I20260709 13:18:52.023777  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:52.034986 11940 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:51.986224  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.050491  7235 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:52.039014 11910 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:52.050684 11940 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:52.051651 11910 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.054060 11856 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:52.017755  6417 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 9 to 11, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 11 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:52.038792 12084 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:52.040035  6488 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.526s (SENT) [suppressed 15 similar messages]
W20260709 13:18:52.010107 11869 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.649s	user 0.016s	sys 0.001s
I20260709 13:18:52.064505  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.065317  6596 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:52.062568  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:52.070183  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:52.062237 12137 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.082531  7232 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:52.089131  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.074906 11910 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.094521  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:52.096086  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.099295  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 59428.8us
I20260709 13:18:52.099474  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:52.100225  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 82036.9us
I20260709 13:18:52.101359  7236 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:18:52.102329  7236 pending_rounds.cc:85] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 8
I20260709 13:18:52.104568  7233 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:52.067662 11869 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.103597  7236 pending_rounds.cc:107] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:18:52.112293  7236 raft_consensus.cc:2889] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:52.103127 11502 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.009s
I20260709 13:18:52.123179  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:52.124100  6520 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:52.124992  6594 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 10.
I20260709 13:18:52.126049  6595 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:52.126566 11856 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:52.128649  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 74866.6us
I20260709 13:18:52.129853  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.135449  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:52.121835  7235 raft_consensus.cc:2376] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12. Re-sending same reply.
I20260709 13:18:52.136197  6522 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.140604  6516 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:18:52.143745  6519 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:18:52.147921 11869 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:52.149262  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.150215  6518 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:52.151415  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:52.151809  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.152879  6517 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:52.126717 11502 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.127079 11856 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:52.164978  6513 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:52.155890  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:52.166731  6595 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:52.167060 11856 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:52.167613  6667 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:52.167714  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.172086  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.174711  6588 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:52.176168  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.175588  6417 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 7 to 9, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 9 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:52.166923 12158 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:52.177047  6590 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:52.164474  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.178938  6593 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:52.176544  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.149304  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.184067  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.177929  6417 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:52.188958  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 51122us
I20260709 13:18:52.193158  7229 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 13:18:52.194531  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1403) took 1209 ms. Trace:
I20260709 13:18:52.195174  7236 rpcz_store.cc:276] 0709 13:18:50.979759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.979893 (+   134us) service_pool.cc:224] Handling call
0709 13:18:50.980155 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.808260 (+828105us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540049d520. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fff06f 00007f04b9077f60 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62
0709 13:18:52.161601 (+353341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.188723 (+ 27122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290095488}
I20260709 13:18:52.138674 11910 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:18:52.205271  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56215us
I20260709 13:18:52.209406  6520 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:52.210521  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.211349  6593 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:52.218662  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.228024  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.229235  6593 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:52.230585  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.231846  7229 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
W20260709 13:18:52.228034  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.235857  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.226562  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:52.212060 11502 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:52.245034  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.250151  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.243891  6486 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.255685  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:52.255833  6417 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 7 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 8 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:52.260937  6519 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:52.263170  6593 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:18:52.263986  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.263530  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.266075  6594 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:52.223484 12084 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:18:52.274264  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.275985  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.277220  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:52.307456  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.313557  6520 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:52.326321  6522 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:52.327860  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.330004  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:52.339478  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:52.340740 11573 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.341463 12083 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:52.341938 12083 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.342284 12083 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:52.293578 12017 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 13:18:52.342758  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 130557us
I20260709 13:18:52.344110  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.344323  6417 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 6 to 10, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 10 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:52.345288 11573 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:52.346122 11669 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:52.346707 11669 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:52.347925  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 113176us
I20260709 13:18:52.347649 11669 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.348744  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.350411 11994 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:52.350867 11994 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:52.351286 11994 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.352994 12039 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:52.353467 12039 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:52.354511 12039 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.355906 12039 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.356954  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.357677  7236 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:52.358949  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.359614  7237 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:52.360107 11826 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:52.360551  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.361112 11940 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:52.361359  7230 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:52.361589 11940 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:52.362607  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.362710 11940 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.365629 12084 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:52.367413 11940 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:52.367231 12084 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:52.367890 11595 raft_consensus.cc:2764] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:18:52.312237 12137 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:52.369266 12137 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:52.369727 12137 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:52.380041  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 192226us
W20260709 13:18:52.380717  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.387905 11867 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:18:52.388952  6515 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:52.379614 11826 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:18:52.391866  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2496) took 1019 ms. Trace:
I20260709 13:18:52.392272  6515 rpcz_store.cc:276] 0709 13:18:51.372471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.374586 (+  2115us) service_pool.cc:224] Handling call
0709 13:18:52.391601 (+1017015us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400044f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:52.391773 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2329819392}
I20260709 13:18:52.397756  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.398468  7236 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:52.401996  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.410876  7228 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:52.413158  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.422377 11826 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:52.422880 11826 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:52.423419 11826 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:52.426541  7231 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:52.427282  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.428757  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.429642  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.430683 12039 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:52.431064 12039 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.431398 12039 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.432037 12137 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.436937  7235 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:52.435719  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:52.475935  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 72868.2us
I20260709 13:18:52.477237  7231 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:52.479434  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.480036  7233 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:52.462795 11403 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:52.478904  6417 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 3 to 7, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 7 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:52.463374 11595 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:18:52.484987  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 144007us
I20260709 13:18:52.489499 12137 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:52.490274 12137 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:52.500625 12137 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:52.490397 12179 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.099s	user 0.001s	sys 0.002s
W20260709 13:18:52.494594 12180 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 13:18:52.494494  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:52.495731  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:52.506603 11910 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:52.495632 11826 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.482789 11856 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.502060 12137 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.510998 11856 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:52.517679  6519 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:52.495741  6417 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 7 to 11, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 11 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:52.498201  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:52.520923 12178 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.104s	user 0.000s	sys 0.002s
I20260709 13:18:52.522832 12135 raft_consensus.cc:2764] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:18:52.526219 12135 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.526813  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:52.528402  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.530809 11595 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.532441 12135 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:52.535526 12135 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:52.539670  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.541587  6515 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:52.533692 11595 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.547384  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.547936  7229 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:52.550961 12084 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:52.551452 12084 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.551846 12084 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.552491 11826 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:52.557114  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:52.557632 12177 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:18:52.557967  6666 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:52.557847  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:52.559784  6663 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:52.565243 11573 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:52.565423  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.565959 11573 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.566989  6667 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:52.569890  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 351043us
I20260709 13:18:52.572563  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:52.585744  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.588793  7234 raft_consensus.cc:2410] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:52.570760 11826 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:52.589645 11826 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.589864  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.590840  7229 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:52.546450 12135 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.595323 11403 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:52.595831 11403 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:52.590039 11826 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.601500  6667 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:52.562229 11595 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:18:52.609912  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:52.604727 11940 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.645306  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.649305  7228 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:52.659412 12191 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:52.660053  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:52.655346 11940 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.662793  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:52.663678 11856 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:52.664932  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:52.665648 11856 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:52.666031 11856 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.666369 11856 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:52.669119 12192 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:18:52.682145  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.681918 11826 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.682327 11994 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.684366 11826 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:52.603541 12039 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:52.691629  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 276748us
W20260709 13:18:52.694644  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.691552  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.696508  7229 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:18:52.662573 11940 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:52.703416  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 93673.7us
I20260709 13:18:52.690140 11573 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.695309 11595 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:18:52.699121  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 112496us
I20260709 13:18:52.702080  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:52.699812  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 227285us
W20260709 13:18:52.714460  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.714943 11910 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:52.712738  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 206691us
I20260709 13:18:52.719278  7234 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:52.726958  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.739910 12083 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.745208 11579 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:18:52.745368  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44203.9us
I20260709 13:18:52.745792 11579 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:52.746577  7200 leader_election.cc:400] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14.
I20260709 13:18:52.748955  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:52.751142 12119 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:52.751629 12119 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:52.752580 12119 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.753106  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.754053  7233 raft_consensus.cc:2410] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:18:52.754213 12119 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:52.754889  7230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:18:52.756386  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:52.756469  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.757082  6515 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.757721  7237 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:52.758347 12135 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:52.758601  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:52.759364 12187 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:18:52.759071 12135 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.761261 12135 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:52.762338 12184 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:18:52.755652  7230 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:52.770474  7230 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:52.763541  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:52.751629  7228 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:18:52.771865  7237 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:52.769714  6515 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:52.773445  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:52.734115 11856 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.776511  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:52.784394  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:52.785481  6520 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:52.786327  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:52.777288  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.777573  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 239348us
I20260709 13:18:52.789337  6667 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:52.754925 12083 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.784428  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:52.795172  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.802488  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.803508  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:52.770326 11994 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.805151  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.810596  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:52.808719  6665 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:52.775928 11403 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.809062 11826 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:52.816028  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:52.816567 11826 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:52.817407 11826 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.823029  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:52.818101  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:52.789337 12196 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:52.833967  6670 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:52.835667  6669 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:52.841557  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:52.842387  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 116701us
I20260709 13:18:52.819721  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.845216 11403 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:52.845909  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 97059.9us
I20260709 13:18:52.837456  6519 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:52.848845  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:52.840777 12084 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.854344  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.820545 11940 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.716066 11910 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.825125 11573 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:52.873092  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:52.876076  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:52.855798 12196 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:52.874343 11940 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.886070  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:52.890754  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:52.891360 11940 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:52.891577  6671 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:52.875047 11502 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:18:52.893281 11502 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:52.893785 11502 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:52.893219 12196 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.891966 11573 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.844795 12083 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:18:52.894603 11502 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.909991  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:52.903085 12197 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:52.920331 11573 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.907444 11502 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.921945 11579 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:52.923336 11579 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:52.936789  7230 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:52.946830  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 52004.6us
I20260709 13:18:52.949530  7233 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:52.949602  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48650.5us
I20260709 13:18:52.952652 11403 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:52.903873 12201 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:52.959986  7230 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:52.963279 11826 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.950111 12019 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:52.966794 12019 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:52.967320 12019 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:52.967445 11403 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.968132 11403 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:52.970160 11403 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:52.972675 12039 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:52.977478  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 67587.8us
I20260709 13:18:52.978705  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:52.933153 11940 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:52.932842 12083 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.966207 11826 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:52.995064  6562 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:52.997414  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:52.999409 12206 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:53.000921 12206 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:53.002826  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:53.003770  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 121223us
I20260709 13:18:53.004592  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:52.993700 11826 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:18:53.006531 11826 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:53.008270  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.009027 12206 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:53.010035 12206 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:52.951086 12208 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:53.011236 12208 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.011648 12208 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.012187 12206 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.012760 12206 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.015522 12039 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:53.015888  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.017630  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.017931 12039 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:53.018292 12017 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:53.015064 12206 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:53.018851 12083 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:53.021409 12083 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:53.010025  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:53.031265  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 80884.3us
I20260709 13:18:53.032066  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:52.981889 12201 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:53.053150  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.056579  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.057749  6592 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:53.065531  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:53.066560  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.066880 12137 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:53.067335  6522 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:53.067492 12137 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:53.067832  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:53.068336  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.068823 11867 raft_consensus.cc:2764] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:18:53.069196  6519 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:18:53.069062  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:53.068010 12137 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:53.069986 11573 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:53.071796  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:53.072732 11573 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:53.073107 11573 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.073444 11573 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:53.079015  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:53.079921 11403 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:53.080274 11403 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:53.080685 11403 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:53.080240  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:53.089198  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.099059  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.090152 12083 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:18:53.107633 11940 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:53.108486  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.108901 12220 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:53.109352 12220 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.109596  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.110235  6590 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:53.110291  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.099685 12195 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:18:53.111037  6592 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:53.111470 12195 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:53.102784 12192 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:53.111904 12195 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:53.112145 12192 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.112594 12192 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:53.112824 12195 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:53.114346 12195 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:53.114799  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 82797.9us
I20260709 13:18:53.121398  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:53.122833  7198 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.123121 12205 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:53.123522 12205 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:53.123760 12218 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:53.123920 12205 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:53.124576 12218 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:53.109702 12220 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:53.127813  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.116394 12084 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:53.117372 11856 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:53.118027 11838 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:53.115536  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 105601us
I20260709 13:18:53.115918 12194 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:53.129918  6589 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:18:53.132298  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 153752us
I20260709 13:18:53.082820 11669 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:53.083067 12201 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.147194  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.149165  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.151402  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.154824  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.155782  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.159415  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.161731  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:53.162660  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.163122  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:53.163671  7234 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.159758 11403 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:53.165366  6592 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.122248 12223 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:53.166149  6593 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:53.131785 11838 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:53.175163  6590 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:53.170279 12194 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.176931  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.177639  6597 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:53.165025 12196 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:18:53.179701  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.180344  6595 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:53.180382  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.180555 11910 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.181824  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:53.182387  7232 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.185854 11669 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:53.186336 11669 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:53.180401 11994 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:53.190707 11994 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.150396 12218 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:53.132453 12197 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:53.160305  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.172679  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:53.135448 11826 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:18:53.216718 11403 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:53.182029 11910 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.256665  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:53.257452  6671 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:53.267729  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:53.271229  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.272183  6519 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:53.274199  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:53.282415  6670 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:53.283058  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:53.287132  7232 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
W20260709 13:18:53.288026  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 152009us
I20260709 13:18:53.296057  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.263885 12196 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:53.298781  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.302376  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:53.303212  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.303501  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:53.304677  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.305249  6597 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:53.304878 11573 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.306819 11856 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:53.307024  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:53.307348 11856 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.307806 11856 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:53.308533  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.308739  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:53.307835  6591 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:53.311496  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 136045us
I20260709 13:18:53.309787  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:53.313409  6664 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:18:53.325966  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:53.326251  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 153634us
I20260709 13:18:53.326982  6519 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:53.293215 12223 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:53.342188  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:53.315268 11502 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:53.349859 12237 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:53.350248  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:53.350932 12219 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:18:53.350426 12237 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:53.354224  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:53.357820 12237 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:53.312772 12137 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:53.334167 12223 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.364989  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.365837  6519 raft_consensus.cc:2410] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 13:18:53.368896  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.372402  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.348635 11502 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.377089  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.378655  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.359534 11826 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:53.380401  6589 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:53.359376 12222 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:53.381296  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72631.9us
I20260709 13:18:53.311054 11838 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.371963 11940 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:53.388623  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.368417 12223 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.395637 12194 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:53.354493 12197 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:53.397666 12232 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 13:18:53.399425 12221 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:18:53.400599 12235 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:53.401124  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.376497 11994 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:53.456616  6592 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:53.378566 12084 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:53.457491  6591 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:53.457495  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 83106.2us
I20260709 13:18:53.458451  6597 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:53.378538 11502 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:53.479157  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:53.494935  6520 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:53.497448  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:53.489068  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 122520us
I20260709 13:18:53.500101  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:53.487454 12019 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.520s	user 0.016s	sys 0.007s
I20260709 13:18:53.520756  7234 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:53.387966 12222 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.495637 12194 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.531504  6515 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:53.532513  6595 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:53.480563 12208 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.547235  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:53.557056  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:53.559226  6515 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:53.518266 11940 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:53.495500 11669 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.576726  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:53.542016 12194 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.588665  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:53.624382  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47737.6us
W20260709 13:18:53.590322 12225 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.275s	user 0.005s	sys 0.000s
I20260709 13:18:53.591432  6514 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:53.592231  6520 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:53.581837 12192 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:53.599532 12231 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:18:53.504905 12235 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:18:53.608115  6515 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:53.575618 12230 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:18:53.581954 11856 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:53.615221 12245 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:53.616348 12239 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.617969 12238 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:53.617635 12223 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:53.619553 12242 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:53.620164 12220 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.615216  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:53.528285 12019 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.624784  6522 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:53.594373 12137 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:53.631244  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.680658  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.536188 12222 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.608362 12208 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:53.645861 12224 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:53.647053 12017 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.570803 11910 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:53.611776 12237 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.661798  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.692184  6520 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:53.667088  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.663807  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.622839 11940 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.672600  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:53.624210 11826 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:53.679333  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:53.693393  6589 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:18:53.679848  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.680009  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:53.697201  6517 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.698124  6670 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:53.652408  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.698885  6595 raft_consensus.cc:2376] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12. Re-sending same reply.
I20260709 13:18:53.685667  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:53.687171 11403 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:53.687928  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:53.689452  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:53.582365 12205 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.711409  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.687714 12224 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.714790  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:53.715559  6513 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:53.711130  6594 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:53.692770 12238 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.698308 11940 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:53.694411 11826 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:53.702019 12245 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:53.755831  6513 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:53.718566 11994 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.718590  6594 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:53.730027 12240 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.736861  6518 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:53.695029 12197 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:53.703615 12208 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.755048  6671 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:53.676371 12239 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:53.729122  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47941.1us
I20260709 13:18:53.717653 12224 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.767339  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.771001  7232 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:53.775297  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 278813us
W20260709 13:18:53.744355 12195 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.626s	user 0.014s	sys 0.007s
W20260709 13:18:53.756067 12244 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:18:53.698557 12017 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.782541  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:53.786298  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.759894 11826 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.794036  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:53.794168  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 178999us
W20260709 13:18:53.797472  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 318451us
I20260709 13:18:53.756402 12245 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:53.799973  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.772027 12247 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.800107  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.797291  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:53.808449  6596 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.811081  6517 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:53.799909  6594 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.812573  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.815562 12259 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:53.816370 12259 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.850286  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.811672  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:53.853470  7234 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
W20260709 13:18:53.853884  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 56682.5us
I20260709 13:18:53.855078  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:53.800563 11910 raft_consensus.cc:2764] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:18:53.744097 12238 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:53.865428  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:53.779253 12208 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.869292  7229 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:53.867341  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 60591.7us
I20260709 13:18:53.779922 12197 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:53.871651  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 89520us
I20260709 13:18:53.872362  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:53.873488  6594 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:53.873423  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:53.818955 11867 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:53.855963 12242 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:53.885957  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.886799  7233 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:53.906242  7234 raft_consensus.cc:2376] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 13. Re-sending same reply.
W20260709 13:18:53.907367  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.863263 12194 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:53.874763 11669 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:53.951119 11856 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:53.955684 12266 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:53.957379 12249 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:18:53.958143 12243 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:18:53.959000 12203 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:53.959594 12238 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:53.959497  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 54598.9us
I20260709 13:18:53.959807 11502 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:53.961352 12261 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:53.875862 12192 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:53.962280 12197 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:53.886271 12256 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.778090 12240 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.967279  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:53.968758 11573 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:53.974993 11994 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:53.979235  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 106952us
W20260709 13:18:53.954921  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 63668us
W20260709 13:18:53.986774  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 66515.7us
I20260709 13:18:54.082095  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:53.989931  6596 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:18:53.992120  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 118473us
I20260709 13:18:54.083276  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:54.084661  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:54.085673 12265 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:18:54.071847 12238 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:18:54.087618  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:53.821590 12247 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.060104 12203 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.987622 12266 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:54.097216  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:54.003772 12223 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.089057 11403 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.108260 11403 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.004477 12220 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.112794  6664 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:54.069303 11856 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:54.106210  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.072220 12237 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:54.110584  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 255617us
I20260709 13:18:54.121851  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:54.122107  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:54.093317  7229 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:54.114886  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.123708  6592 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:54.115511  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.124066  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:54.116504  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.118995  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:54.099426 12203 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.126904  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.132226  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.132853  6595 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:54.110224 12205 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.106233 12266 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:54.146885 12017 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:54.132081 12255 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:18:54.179528 11403 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:54.179237  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 94639.9us
I20260709 13:18:54.091754 12256 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:53.919680 12245 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.096230 12206 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:18:54.169677  6666 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:18:54.170717  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48699.9us
I20260709 13:18:54.162683 12253 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:54.175568 12223 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:54.174165  6588 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:54.173817 12269 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:54.182224  6594 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:54.132383 12191 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:18:54.184715  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 123335us
W20260709 13:18:54.189652  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 62834.1us
I20260709 13:18:54.173890 11573 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:18:54.205575  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.161290 11838 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.230501  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.230953  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.231356  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:54.231982  6516 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:54.174533 12237 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:18:54.232210 12256 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.233389  6522 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:54.234676 12273 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.234783  6517 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:54.235216 12273 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.237398 12273 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.237916 12273 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:54.239146 11994 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:54.239603 11994 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:54.240983 12241 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.642s	user 0.004s	sys 0.000s
I20260709 13:18:54.240669 11994 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.217269 12269 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:54.242794 12269 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:54.244498 12247 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:54.254443 12257 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.255174 12257 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.257848 12257 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.183126 12239 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:54.258550 12272 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:18:54.259512 11940 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.230695 12264 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:54.262401  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.231479  6513 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.215220  6592 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:18:54.242372 11502 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:54.178675 12222 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.287145 12222 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:54.226200 12208 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.221319 12253 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:54.289258 12253 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.282718 12264 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:54.291672 12264 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.283408 12277 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:54.269264 12191 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:54.271718 11573 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:54.276137 12257 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.309453  6513 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:54.314524  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2357) took 1008 ms. Trace:
W20260709 13:18:54.314877  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 52553us
I20260709 13:18:54.277884 11856 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:54.277720 11940 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.316244  6670 rpcz_store.cc:276] 0709 13:18:53.305984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.323890 (+ 17906us) service_pool.cc:224] Handling call
0709 13:18:53.336703 (+ 12813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.705702 (+368999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.314262 (+608560us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54000e56d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:54.314463 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830715648}
I20260709 13:18:54.320120 11573 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.300087 11910 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.326843  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:18:54.324746 12201 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.159s	user 0.009s	sys 0.004s
I20260709 13:18:54.338776 12265 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:54.340049  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.341643 12265 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:54.344059 12191 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.345224 12019 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.345978  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.348035 12259 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.348443 12019 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:54.364596  7237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1582) took 1610 ms (client timeout 1795 ms). Trace:
W20260709 13:18:54.365374  7237 rpcz_store.cc:269] 0709 13:18:52.765004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.795281 (+ 30277us) service_pool.cc:224] Handling call
0709 13:18:54.363861 (+1568580us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54006a3bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:54.364316 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934372096}
I20260709 13:18:54.365173  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.367679 12240 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.368445 12240 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:54.322151 12257 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:54.375974 12246 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.792s	user 0.009s	sys 0.002s
I20260709 13:18:54.376864 12257 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.330605 11910 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.379621 11910 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.380748 12203 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.383035  6596 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:54.411473 12202 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.411985 12202 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.413448  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.414115 12202 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.414888  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.336706 12201 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.377564  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2347) took 1128 ms. Trace:
W20260709 13:18:54.418309  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 188431us
I20260709 13:18:54.386116  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.386803  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.421388  6592 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.388036  6589 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:54.388221 12268 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:54.392596  6670 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:54.394515  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:54.425248  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.395025 12257 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:54.396490 12206 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.401504 12019 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:54.405571 12269 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:54.408543  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 185644us
I20260709 13:18:54.449777  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:54.409114  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 200236us
I20260709 13:18:54.453310  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.409242  6590 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:18:54.417502  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 335470us
I20260709 13:18:54.385941 12245 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.419279 12194 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:54.415500  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:54.408128 11826 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.001s
I20260709 13:18:54.424868  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.418907  6668 rpcz_store.cc:276] 0709 13:18:53.249280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.249444 (+   164us) service_pool.cc:224] Handling call
0709 13:18:53.269524 (+ 20080us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.294835 (+ 25311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.294888 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.322124 (+ 27236us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.496237 (+174113us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:53.496304 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.496394 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:54.375294 (+878900us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:54.377217 (+  1923us) raft_consensus.cc:1685] finished
0709 13:18:54.377241 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.377477 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4793,"raft.queue_time_us":17279,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":89604992,"thread_start_us":134627,"threads_started":1,"wal-append.queue_time_us":238938}
I20260709 13:18:54.429675  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.468465  6595 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:54.432392  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.469461  6596 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:18:54.407836 12224 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.617s	user 0.005s	sys 0.006s
I20260709 13:18:54.475457  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:54.475816 12224 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.476089  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:54.476159  6668 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:18:54.477224 12208 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:18:54.477843 12224 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.477977 12223 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:54.479382  6664 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:54.479921  6513 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:54.430822 12220 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:54.430538  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.470467 11826 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.471611  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.471920  6665 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:18:54.489039  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73652.5us
I20260709 13:18:54.472286 12269 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.472698  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.473115  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.473694  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:54.473167 12239 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:54.474304  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:54.491822  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.473114 11502 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:54.491842  6670 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:54.478490 12224 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:54.482066  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.482066  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:54.482133  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.486352  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.433483 11910 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.487691  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.466559  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.495450  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.523547  6663 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:54.517521  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.502666  6666 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:18:54.499955  6592 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:54.515833  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:54.525558  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 52936.8us
I20260709 13:18:54.516314  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:54.521199  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:54.517436  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.502563  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.519116  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:54.520448  6517 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:54.516733  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.527208  6596 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:54.497107  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:54.528367  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:54.528873  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:54.529845  6592 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:54.529991  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:54.530992 11867 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:54.531153 12192 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:54.531651 11867 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:54.532162  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:54.532950 12234 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.958s	user 0.000s	sys 0.003s
I20260709 13:18:54.533074 12222 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:54.533762 12222 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:54.467351 11856 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.534672 12222 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.537323  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1586) took 1391 ms. Trace:
I20260709 13:18:54.537698  7236 rpcz_store.cc:276] 0709 13:18:53.162022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.170081 (+  8059us) service_pool.cc:224] Handling call
0709 13:18:53.175142 (+  5061us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:53.213482 (+ 38340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.213515 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:53.229960 (+ 16445us) log.cc:844] Serialized 52 byte log entry
0709 13:18:53.332420 (+102460us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:53.332478 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.332568 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:54.532117 (+1199549us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:54.537053 (+  4936us) raft_consensus.cc:1685] finished
0709 13:18:54.537073 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.537269 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":516,"raft.queue_time_us":13125,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":39317,"spinlock_wait_cycles":16035200,"thread_start_us":41671,"threads_started":1,"wal-append.queue_time_us":218595}
I20260709 13:18:54.525735  6521 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:54.540782  6514 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:54.554317  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:54.556198  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:54.556949 11838 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:54.557636 11838 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:54.558082 11838 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:54.559031 11838 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.527905 11910 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.561939 11910 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:54.491822 12084 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:54.518579 12199 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.564648  6517 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:54.554574  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2679) took 1418 ms. Trace:
I20260709 13:18:54.544220  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:54.566412  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 211496us
I20260709 13:18:54.569343  7230 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:18:54.578289  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:54.433866 12257 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.448140 12206 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.563019  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.447780 12266 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.584790  6588 rpcz_store.cc:276] 0709 13:18:53.135905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.136047 (+   142us) service_pool.cc:224] Handling call
0709 13:18:54.554339 (+1418292us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b540019d2d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:54.554482 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310118784}
I20260709 13:18:54.592296  6518 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:18:54.592767  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63654.6us
I20260709 13:18:54.593591  7230 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:18:54.592136  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:54.616513  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:54.614246  7234 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:54.619531  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:54.620277  7229 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:54.628222  7229 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:54.629403  6667 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:54.527770 12205 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:54.616962  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:54.528432 12277 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:54.483707 12194 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.640352  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.540729 12261 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:54.430328 12268 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.527732 12220 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:54.591459 12224 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:54.642023  7229 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:54.582778 11910 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:54.762269  6518 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:54.632248  7230 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:18:54.763005  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 265974us
I20260709 13:18:54.580464 11573 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:54.753523  7236 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
W20260709 13:18:54.759418  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 138177us
I20260709 13:18:54.769549  7236 pending_rounds.cc:85] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 9
I20260709 13:18:54.769876  7236 pending_rounds.cc:107] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:18:54.770112  7236 raft_consensus.cc:2889] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:54.770393  7236 pending_rounds.cc:107] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:18:54.770607  7236 raft_consensus.cc:2889] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:54.760391  6595 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:54.775691  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.776763  6592 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:54.765453  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:54.785795  7231 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:54.787439  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.788892  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.641345 11940 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:54.760468  6593 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:54.763942  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:54.822602  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1424) took 1059 ms. Trace:
W20260709 13:18:54.824357  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.827637  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.835160  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 65261.1us
I20260709 13:18:54.836795 12084 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:54.581056 12017 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.184s
I20260709 13:18:54.755399 12261 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:54.851192  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:54.590611 12196 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.199s
I20260709 13:18:54.877099  6670 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:54.848137  7235 rpcz_store.cc:276] 0709 13:18:53.763706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.767122 (+  3416us) service_pool.cc:224] Handling call
0709 13:18:53.767562 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.767848 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:53.855109 (+ 87261us) log.cc:844] Serialized 14 byte log entry
0709 13:18:54.406243 (+551134us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b180044cd40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:18:54.822189 (+415946us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:54.822269 (+    80us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:54.822334 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.822430 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.822545 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58564,"prepare.queue_time_us":219402,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":6219,"raft.queue_time_us":419395,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5882,"spinlock_wait_cycles":3556736,"thread_start_us":614235,"threads_started":3,"wal-append.queue_time_us":269079}
I20260709 13:18:54.862224  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.886798  6597 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.847687 12205 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:54.533020 11867 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:54.864673  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:54.841276  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:54.899838  7234 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:54.900017  6597 raft_consensus.cc:2410] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:18:54.873768  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 256843us
I20260709 13:18:54.875095  6588 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:54.630748 12260 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.213s
I20260709 13:18:54.767668 11910 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:54.612371 11669 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.177s
I20260709 13:18:54.894565  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:54.618221 12199 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.901006  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:54.916968 11573 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:54.917618 12205 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.920789 12305 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.034513 12305 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.036749 12284 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:55.037400 12247 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:55.037891 12247 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:18:55.039208 12287 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.394s	user 0.000s	sys 0.000s
I20260709 13:18:55.039376 12309 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:55.039637 12206 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:55.039978 12309 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:55.041152 12281 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.459s	user 0.001s	sys 0.000s
W20260709 13:18:55.042536 12288 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
W20260709 13:18:55.048254  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 137112us
W20260709 13:18:55.051684  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 296260us
I20260709 13:18:54.881456 12261 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.037411 12284 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.117995  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1086) took 1468 ms. Trace:
I20260709 13:18:54.895179 12203 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:54.895783 12222 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.042433 12309 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:55.125811  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76607us
I20260709 13:18:55.126040  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:55.126706  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:54.800518 12202 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 13:18:55.122318 11403 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:54.890802 12272 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:55.126721  6515 rpcz_store.cc:276] 0709 13:18:53.649764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.649916 (+   152us) service_pool.cc:224] Handling call
0709 13:18:53.673894 (+ 23978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.679323 (+  5429us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:53.691717 (+ 12394us) log.cc:844] Serialized 14 byte log entry
0709 13:18:54.261902 (+570185us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b18001fd740. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:18:54.789837 (+527935us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.052626 (+262789us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.116431 (+ 63805us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:55.116752 (+   321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.117476 (+   724us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.117925 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20499,"prepare.queue_time_us":624377,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":335778,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":16047,"spinlock_wait_cycles":6752128,"thread_start_us":618983,"threads_started":3,"wal-append.queue_time_us":836835}
W20260709 13:18:55.125206 11838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.005s	sys 0.008s
I20260709 13:18:55.125120 12222 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:54.921896 11910 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.052012 12264 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:54.931056 12199 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:55.198987 12290 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:18:55.198712 12312 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:18:55.173556 11838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.011s	sys 0.014s
I20260709 13:18:55.120092 12201 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.128657 12197 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:55.167716 12272 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.127669 12284 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:55.198865 12282 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.191500 12269 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:54.932293 11573 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:55.293648  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 167013us
I20260709 13:18:55.215466 11669 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:55.215562 11826 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:55.216307 12223 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:55.216641 12264 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.223439  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:55.292265  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:55.292567  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:55.119916 12220 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.307736  6668 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:55.304697  7230 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:55.308900  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:55.227257  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 348634us
I20260709 13:18:55.226012  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:55.227383  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 463503us
I20260709 13:18:55.311668  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:55.272193  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.279597  6663 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:55.273497  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 372533us
I20260709 13:18:55.309547  7230 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:18:55.285499  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.315546  7234 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:55.233749  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.272449  7229 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:55.272720  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.273072  7231 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:18:55.285877  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:55.283824  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:55.320169  6664 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:55.269230  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.321779  7235 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:55.322130 11867 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:55.215047 12238 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:55.323037  6671 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:55.142104 12224 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.184430 12084 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.269552  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.283921  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:55.326341  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:55.328023  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:55.334522  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:55.336836 11403 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:55.300235 11669 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:55.337379 11403 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:18:55.337745  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 111806us
I20260709 13:18:55.226223 12192 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:55.336648  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:55.275962 12272 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.281996 12202 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:55.333169 12238 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.327412 12268 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:55.354537  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:55.354333  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:55.355772  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44189.2us
I20260709 13:18:55.351759 11403 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.220218 12205 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.348883  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:55.360592  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:55.349475  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:55.379771  6671 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
W20260709 13:18:55.340140  6486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.002s	sys 0.019s
W20260709 13:18:55.380605  6486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.003s	sys 0.019s
I20260709 13:18:55.359588  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:55.352523  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:55.330283 11940 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:55.314479 12223 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.334041 12266 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:55.206713 12253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.014s	sys 0.009s
I20260709 13:18:55.374583  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:18:55.372591  6668 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:55.364380 12283 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:55.323184 12137 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:18:55.285153 12269 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.404213  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:55.399631 12203 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.359560 12194 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.348466 12218 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:55.364691 12280 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:55.275907 12197 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.355096 12257 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.322676 11867 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:55.313750 12282 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:55.367399 12202 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:55.377849 12238 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:55.427675  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:55.401582 12253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.015s	sys 0.010s
W20260709 13:18:55.443604  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 69075.5us
I20260709 13:18:55.354048 11826 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:18:55.371317 12314 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:55.354645 12302 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:18:55.400985 11856 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:55.412276 12208 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:55.412590 12195 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:18:55.418352 12294 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:55.393213 12313 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:55.418572 12205 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:18:55.462415  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 149357us
I20260709 13:18:55.498057  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:55.438016  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52500.6us
I20260709 13:18:55.469676  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:55.430523 12266 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:55.434677 12192 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:55.445626 12019 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:55.445351 12280 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:55.454629 12310 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:18:55.479218 12277 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:55.506546  6518 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:55.506819  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 121181us
I20260709 13:18:55.500641  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:55.516741 11403 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.443823 12203 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:55.487408 12208 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.452665 12194 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:18:55.526219  6520 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:18:55.526162  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:55.504683  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:55.498102 12313 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.418725 12283 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:55.500891 12302 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:55.497821 12294 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:55.536950  6517 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:55.452196 12218 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.523319 11856 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:55.540835 11838 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:55.541111 12313 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.522195 12266 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.525681 12222 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.552640 12192 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.553614 12283 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:55.511332 12201 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:55.542217 11403 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:55.556069  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:55.566833 12294 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:55.609490  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53497.8us
I20260709 13:18:55.610600  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:55.540758 12331 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 13:18:55.565811  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 85916us
I20260709 13:18:55.552459 12137 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:55.551399 12293 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 13:18:55.512290 11940 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.631810  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:55.599696 12205 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:55.597913  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:55.542408 12208 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.559279  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:55.570919 11856 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.613531 11403 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.619300  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:55.653746 11403 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:55.621155  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:55.454604 11669 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.628932 12218 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:55.631008  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.590238  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2691) took 1116 ms. Trace:
I20260709 13:18:55.635759  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:55.639760  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:55.612398 11838 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:55.643540  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:55.643705  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:55.643934  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46107.1us
I20260709 13:18:55.644315  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.671866  7234 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:55.672118  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:55.645190  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:55.646066  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:55.672719  6521 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:55.606896 12302 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:55.675436  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:55.622772 12137 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:55.682437 12317 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:55.682948 12317 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:55.682652 12137 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.683430 12317 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.683948 12137 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:55.685808 11994 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:55.675261  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:55.686446  6514 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:18:55.686748 12330 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:55.687237 12330 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:55.570580 12318 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:18:55.669642  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:55.669873  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.664996  6590 rpcz_store.cc:276] 0709 13:18:54.455889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.528798 (+ 72909us) service_pool.cc:224] Handling call
0709 13:18:55.119985 (+591187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.120373 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.120432 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.120512 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:55.120573 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.120691 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.572336 (+451645us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54002fe4d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:55.572505 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076915584}
I20260709 13:18:55.661603 11856 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:55.607205 12264 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:55.675962  6597 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:55.676031  6589 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:55.677094 11403 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.681294 12325 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:55.688819 12218 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:55.667212  6513 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:55.612130 12222 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:18:55.606356 12259 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:18:55.712599  6595 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:18:55.642341  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:55.772042  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 129810us
W20260709 13:18:55.758353  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 139141us
I20260709 13:18:55.637008 12205 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:55.627208 12223 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:55.597177 12202 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.752427 12314 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:55.753783 12269 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:55.791721  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:55.798863  6595 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:55.689119 12336 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:55.768929 12325 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:55.817889  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:55.827656  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 378189us
I20260709 13:18:55.831555  6590 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:55.796180 12300 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.261s	user 0.001s	sys 0.001s
I20260709 13:18:55.826334  6589 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:55.787725 12331 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:55.788622 11867 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.848111  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:55.852102  6597 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:55.865832  6590 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:18:55.825356 12336 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.830951 12325 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:55.847942 12332 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:18:55.869148  6513 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:55.869210  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:55.873236  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:55.876528  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:55.877390 12223 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:55.877806 12223 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:55.878209 12223 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.877987 12283 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:55.878509 12264 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:55.881816 12334 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.882381 12334 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.884817 12334 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:55.885427 12334 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:55.885820 12334 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.886204 12334 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.867951 11867 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:55.865125 12203 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.893915  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:55.895275  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:55.871088 12336 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:55.886807 12195 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:55.906966 12345 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.907537 12345 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:55.909895 12345 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.911710  6595 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:55.910751 12195 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.920924 12344 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:55.921566 12344 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:55.923455 12343 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:18:55.889207 12269 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 13:18:55.905355 12323 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.341s	user 0.000s	sys 0.002s
I20260709 13:18:55.901367 12266 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:55.891600 12315 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.263s
I20260709 13:18:55.955871  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1096) took 1505 ms. Trace:
I20260709 13:18:55.870852 12205 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.887477 12328 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.320s
W20260709 13:18:55.957357 12272 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
I20260709 13:18:55.852165 12302 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:55.976504  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 79266.6us
I20260709 13:18:55.911306 11867 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:55.982695  6486 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.536s (SENT)
I20260709 13:18:55.966522 12330 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:55.886816 12314 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:55.759029 12260 consensus_queue.cc:1175] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "53fee9894731461a821a131be24dbbca" responder_term: 12 status { last_received { term: 9 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.9-11.9]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:55.960726  6669 rpcz_store.cc:276] 0709 13:18:54.450304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.450446 (+   142us) service_pool.cc:224] Handling call
0709 13:18:54.487631 (+ 37185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:54.488052 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.488101 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:54.502160 (+ 14059us) log.cc:844] Serialized 26 byte log entry
0709 13:18:54.823133 (+320973us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:54.873579 (+ 50446us) log.cc:844] Serialized 14 byte log entry
0709 13:18:54.873836 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.873922 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.955630 (+1081708us) raft_consensus.cc:1685] finished
0709 13:18:55.955651 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.955808 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":221483,"prepare.queue_time_us":50900,"raft.queue_time_us":704,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":32481,"spinlock_wait_cycles":21913856,"thread_start_us":307239,"threads_started":1,"wal-append.queue_time_us":619822}
I20260709 13:18:55.977285 12269 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:56.002381  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:56.021694 12245 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.554s	user 0.008s	sys 0.004s
I20260709 13:18:56.024762 12322 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.025321 12322 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.026450 12223 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.008841 12328 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:56.028549 12223 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:56.013842 12269 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:56.014436  6486 leader_election.cc:336] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.589s (SENT)
I20260709 13:18:56.031205  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:56.035140 12269 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:56.035544 12261 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.859s	user 0.011s	sys 0.000s
I20260709 13:18:56.036082 12328 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:18:56.038132  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.002501 12205 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.040566 12324 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.163s
I20260709 13:18:56.052995 12324 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:56.053345 12324 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:55.977721 12272 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.013543 12330 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:56.051636 12356 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:18:56.073472 12361 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:56.074295 11910 consensus_meta.cc:287] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.872s	user 0.011s	sys 0.000s
I20260709 13:18:56.074079 12361 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.074779 11910 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.076423 12361 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.050422 12329 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.078138  6519 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
W20260709 13:18:56.088673 12347 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:18:56.089089  6521 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:56.082887 12329 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.091825 12195 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.036089 12261 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:56.150507 12284 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.000s
I20260709 13:18:56.107348  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:56.111832 12357 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:18:56.114188 12297 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:18:56.126240 12017 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:56.127667 11994 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.128456 12280 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.130972 12305 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:56.141148 12335 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.600s	user 0.005s	sys 0.000s
W20260709 13:18:56.142827 12346 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.381s	user 0.000s	sys 0.004s
I20260709 13:18:56.144173 12341 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:56.145208 12315 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:56.145802  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:56.146721 12266 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:56.150331  6589 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:56.151084 12247 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.152244 12199 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:56.077071 11910 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.103713  6663 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:56.163014  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.105527  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.054158 12324 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.085865 12350 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:56.170647  6670 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 10: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:56.216403  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.216912  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.217646  6670 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:56.217788  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.218380  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.171204  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.058686 12208 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.175882  6521 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:56.078166 11856 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.224105  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:56.175128 12238 consensus_meta.cc:287] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.684s	user 0.003s	sys 0.008s
I20260709 13:18:56.174275  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:56.078970 12330 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.172495 12269 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.174263 12017 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:56.191965  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:56.206957  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 61230.2us
I20260709 13:18:56.232074  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:56.208829  6522 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:56.177623 11994 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.214885 12328 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:56.235054  6667 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:18:56.235167  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.235764  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:18:56.215267 12366 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:18:56.215793 12324 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:56.150581 12336 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.222289  6514 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:56.221819  6670 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:56.216364 12350 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.196875 12266 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:56.175248 12284 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.228899  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:56.228986  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 54807.4us
I20260709 13:18:56.253562 12350 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.254962  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1439) took 1032 ms. Trace:
I20260709 13:18:56.254807 12245 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:56.256031  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:56.256290 12245 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:56.257158  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:56.257318 12327 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:56.257696 12327 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:56.258504 12327 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.259572  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.259734 12327 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:56.261072 12202 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:56.261588  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:56.263996 12359 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:18:56.264895  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:56.204316 12341 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:18:56.224043 12310 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.000s	sys 0.010s
I20260709 13:18:56.228154 12361 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.231556 12265 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:56.258913  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:56.231348 12305 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:56.191578 12268 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:56.234269 12247 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:56.239046 12317 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.004s
I20260709 13:18:56.258279  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:56.334648  6593 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:18:56.272213  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:56.284317  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:56.285856  6591 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:56.235122 12328 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.293306  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:56.294090  6592 raft_consensus.cc:2368] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17.
I20260709 13:18:56.227993 12238 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.296293 12223 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:56.255364  7228 rpcz_store.cc:276] 0709 13:18:55.222089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.222217 (+   128us) service_pool.cc:224] Handling call
0709 13:18:55.226730 (+  4513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:55.269074 (+ 42344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.269101 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:55.278400 (+  9299us) log.cc:844] Serialized 52 byte log entry
0709 13:18:55.357669 (+ 79269us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:55.498727 (+141058us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.501191 (+  2464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.501641 (+   450us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.140288 (+638647us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:56.248545 (+108257us) raft_consensus.cc:1685] finished
0709 13:18:56.254731 (+  6186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.254890 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30984,"prepare.queue_time_us":254520,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":71046,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":31989,"spinlock_wait_cycles":24176000,"thread_start_us":165128,"threads_started":2,"wal-append.queue_time_us":157633}
W20260709 13:18:56.299553  6666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2367) took 1894 ms (client timeout 2041 ms). Trace:
I20260709 13:18:56.279888 12272 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:56.319802 12313 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.713s	user 0.006s	sys 0.005s
I20260709 13:18:56.320569 12330 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:56.321101 12362 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
I20260709 13:18:56.254488 11573 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:56.324157 12325 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.326574 12195 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.327198 12341 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:56.328979 12363 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:18:56.329582 12261 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.248240 12197 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:56.392529  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 600858us
I20260709 13:18:56.393067  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2370) took 1053 ms. Trace:
I20260709 13:18:56.393280  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:56.336768 12305 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.395278 11826 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:18:56.312827 12294 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.004s
I20260709 13:18:56.395851  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:18:56.284049 12310 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.000s	sys 0.010s
I20260709 13:18:56.396159 12294 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.397938 12316 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:56.398326 12294 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.296464 12361 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:56.401544  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.402383 12361 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.403131  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:56.403791 12294 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:56.404310 12294 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.406123 12201 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:18:56.406555  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.407243 11838 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:56.407681 11838 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:56.407807 12201 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:56.393538  6664 rpcz_store.cc:276] 0709 13:18:55.339190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.339381 (+   191us) service_pool.cc:224] Handling call
0709 13:18:55.339657 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.423928 (+ 84271us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:55.790528 (+366600us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.991469 (+200941us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.991526 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:55.991570 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.392814 (+401244us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54000b3954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:56.392868 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.393006 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21266,"prepare.queue_time_us":202476,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":8158,"raft.queue_time_us":6469,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2524,"spinlock_wait_cycles":193248128,"thread_start_us":534657,"threads_started":2,"wal-append.queue_time_us":372011}
I20260709 13:18:56.412412  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.413024  6664 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:18:56.413833  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.412438 12294 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.340948  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.328979 11994 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.414865  6664 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:56.416798  6664 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:56.395869 12305 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.418367  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.419559  6662 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:18:56.419875  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.421136  6662 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:56.422281 12344 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.360944  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.361641  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:56.341014  6666 rpcz_store.cc:269] 0709 13:18:54.393431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.394263 (+   832us) service_pool.cc:224] Handling call
0709 13:18:56.287262 (+1892999us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b54003fc854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:56.287468 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176272128}
I20260709 13:18:56.360967 12313 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.365149  6516 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:18:56.366791 12309 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:56.457954  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.334199 12268 consensus_queue.cc:1243] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:56.345409 12223 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.378504 12325 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:56.341781 12350 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:56.341171 11910 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.338846 12364 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:56.337872 12328 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.379068 12195 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.381546  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.382866 12208 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.435627 12084 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:56.499652 12084 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:56.383356 12336 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:56.384948 12320 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.832s	user 0.004s	sys 0.000s
I20260709 13:18:56.508481 12208 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:56.490262 12283 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:18:56.445261 12257 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.509194  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:56.387046 12365 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:18:56.498025 11669 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:56.484589 12247 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.387167 11856 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:56.388233 12341 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:56.510679 12247 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.512974 12247 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.297061 12266 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.514322 12206 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:56.514693 12313 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.398494 12316 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.515472 12206 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:56.516269  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.398864  6663 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:56.517053 12316 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.400252 12019 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.399467 12353 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:56.517686 12019 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.518079  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:56.518733  6596 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:56.520030 12019 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.408098 11838 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:56.520855 12223 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.521886 12208 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:56.522325 12208 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:56.342886  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.415390  6522 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:56.297034 12271 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:56.345640 12302 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:56.523188 12208 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:55.124063  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:56.523823 12271 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:56.523931 12302 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:56.524413 12208 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.524518 12307 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:56.524915 12271 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.525154 12307 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.526041 12271 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:56.347815 12238 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:56.423426  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:56.445379 12329 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:56.335942  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63976.3us
I20260709 13:18:56.480880 12309 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:56.480758 12324 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.395908 11826 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:56.495009  6521 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:56.494824  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:56.338083 12317 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.499500  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.554454  6596 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:56.555437  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:56.555858  6520 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:56.500794  6590 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:18:56.383118 12304 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.077s	user 0.002s	sys 0.000s
I20260709 13:18:56.449309 12282 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.501482  6671 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:56.501754  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:56.505532  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.484067 12305 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:56.509752 12257 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.296064 12017 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.511003 11940 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:56.389999 12322 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.233644 12269 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.516346 12206 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:56.517765 12353 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.464241  6518 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.348536  6515 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:56.527714  6591 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:18:56.426605  6665 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:56.461393  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:56.538358  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.502405 11910 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.542425  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:56.543047 12195 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.542972 12340 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:56.543998  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.544256  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.466457 12327 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:18:56.545271 12330 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:18:56.545826  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 122500us
I20260709 13:18:56.630576  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.630702  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:56.632026  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:56.632094  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.545948 12329 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:56.551388  7233 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:56.557242  6588 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:56.557150  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:56.557976  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:56.551437 12334 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.658s	user 0.005s	sys 0.005s
W20260709 13:18:56.563823  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 69067us
I20260709 13:18:56.524116 12284 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.568203  7200 leader_election.cc:400] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17.
I20260709 13:18:56.573685  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.674870  7200 leader_election.cc:403] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:18:56.555116 12344 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:56.541123  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:56.611682  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 69314.6us
I20260709 13:18:56.675137  7230 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:56.680361  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.609372 12206 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:56.552580 12324 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:56.616278  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 154729us
I20260709 13:18:56.616590  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:56.609525  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.609133 11940 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.610733  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.625663  6671 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:56.627179  6516 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:56.627748  7236 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:56.571370 12326 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:56.629576  6670 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:56.641916  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.632869  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:56.703491  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:56.703907  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 71102.1us
I20260709 13:18:56.704403  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.665989  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:56.509060 12084 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:56.675392  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 107331us
I20260709 13:18:56.708257  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.681136  7234 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:56.633069 12322 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:56.682132  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:56.580127 12361 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:18:56.555240 12315 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.716858  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.717070  6665 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:56.664863 12314 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:56.635532  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:56.719038  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 83577.1us
I20260709 13:18:56.686091  6590 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.634140 12282 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:56.626964 12309 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.704174 12307 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.706808  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.722424  6516 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:56.618561 12191 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:56.727833 12191 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.712612 12322 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:56.728318 12191 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:56.728583 12322 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:56.730129 12280 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:56.709795  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.730675  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:56.731081  6514 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:56.731631 12269 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:56.731920  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.732587  7233 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:56.733434  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.734596 12353 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:56.736819  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.736734  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:56.737118  6515 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:56.737540  6588 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:56.704566  6486 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:56.725464 12314 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.741544 12314 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:56.743311  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:56.683794 12344 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:56.748039 12344 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:56.747920 12328 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.618664 12305 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.752161 12305 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.715102  6666 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:56.753017  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.753247  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:56.753791  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.754184  6517 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:56.683082 11669 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.754508 12316 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:56.755414 12316 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:56.756819  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:56.757449  7235 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:56.722491 12309 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:56.707957  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:56.723639  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:56.724014  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.727210  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.700318  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:56.758253  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:56.705233  7230 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:56.711980 12199 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:56.773576  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65707.5us
I20260709 13:18:56.734459  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:56.727620  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:56.758548  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54060us
I20260709 13:18:56.774454  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:56.678539 12334 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.769153  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:56.713907 12237 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:56.756477 12198 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:56.778977  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 150148us
I20260709 13:18:56.750474 12328 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.767407 11826 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:56.812191  6588 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:56.813691  6590 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:18:56.786805  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:56.790858  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:56.804394  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.714043 12341 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:18:56.816900  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:56.818524  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.824680  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.825497  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.805358  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:56.773110 12319 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:56.829209  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.773209 12199 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:56.860035  6519 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:56.782562 12245 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:56.780761 12198 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.815799 12238 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:56.832147  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:56.832876  6593 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:56.806783 11826 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:56.775483 12206 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.835645  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.835747 12310 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:56.816556 12361 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:56.889252 12361 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:56.889704 12361 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:56.892649 12316 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:56.781929 12282 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:18:56.864147 12341 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:56.843001 11838 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.894140 12341 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.895253  7228 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:56.864449 12319 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:56.896122 11838 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:56.844831 12317 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.848220 12256 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:56.816004 12223 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:18:56.896886 12319 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.867086 12198 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.900214 12324 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:56.900835 12257 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.901799 11910 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:56.903066 12206 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:56.903806  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.904475  7236 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:18:56.905326  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.905905 11826 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.906672  6662 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:56.907287 11826 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:56.905911  7236 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:56.908222 12344 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.909210  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.888041 12238 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:56.887593 12310 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:56.910326  7236 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:56.911140 12310 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:56.912243  7236 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:56.912797  7236 pending_rounds.cc:85] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 8
I20260709 13:18:56.913139  7236 pending_rounds.cc:107] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:56.913478  7236 raft_consensus.cc:2889] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:56.849082 12272 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:56.826207 12325 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:56.869503  7235 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:56.756935 12315 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:56.870433  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 66042.8us
I20260709 13:18:56.921619  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:56.773269 12197 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:18:56.826354 12307 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:56.885903  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.942314  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.947407  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:56.864162 12199 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:56.898352 11867 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.930718  6518 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:18:56.910185 11940 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.884791 12328 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:56.944429  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:56.953737  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:56.962396  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:56.831259  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44552us
I20260709 13:18:56.912308 12310 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:56.896137 12341 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:56.971140  7233 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:56.968284  7230 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:56.979692  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:56.980930  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:56.981421  6513 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:18:56.982332  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 233307us
I20260709 13:18:56.984637  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:56.984252  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:56.939970 12272 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:56.943104 12203 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:56.992091  7236 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:56.949771 12238 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.065624  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.064914 12341 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.071336  7233 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:56.971585 12256 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:57.079142  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.079142  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:56.977869 12324 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:57.082682  7235 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:57.087864  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.093914  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:56.942781 12307 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:57.107349  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:57.107481  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.139618  7236 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:18:57.108791  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 111910us
I20260709 13:18:57.110743  6588 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:18:57.070518 12223 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:57.114843  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.148186  6670 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 12: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:57.148104  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:57.117131  6663 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:18:57.149734  6562 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.637s (SENT) [suppressed 20 similar messages]
I20260709 13:18:57.150025  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.130004  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:56.914744 12314 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.106276 12273 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:57.142416  7233 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:57.116253 12256 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:57.122831 12084 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:57.133772 11940 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.117630  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.107681 12305 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:18:57.155762  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.131178 12307 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.167325  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:57.169283  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:57.173522  6670 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:57.172905  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.139374 12344 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:57.176235  7231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.000s	sys 0.033s
W20260709 13:18:57.184926  7231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.000s	sys 0.033s
I20260709 13:18:57.137938 12345 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:57.141767 11669 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17.
W20260709 13:18:57.178524  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.186343  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.188467  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:57.189100  6665 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:57.147758 12223 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:57.114912 12341 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.159523 12273 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:57.161286 12364 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:57.177217 12349 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.303s	user 0.008s	sys 0.007s
W20260709 13:18:57.172773 12271 consensus_meta.cc:287] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.646s	user 0.006s	sys 0.005s
I20260709 13:18:57.198283  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.172164 12269 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:57.183470 12385 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:18:57.135761 12324 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.176133 12327 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:57.187574 12345 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.200630  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.187228 12326 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:57.292295  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.304139  6663 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:18:57.200940 12223 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:57.189978 12328 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:18:57.320109 12223 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:57.312661  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.313017 12345 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:57.313844  6662 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:57.324630 12314 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.222591 12364 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.321164 12269 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:57.190095 12203 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:18:57.315270 12327 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:57.174952 11940 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.333364  7234 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:57.334070  6662 raft_consensus.cc:2410] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:18:57.334831 12364 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:57.337431 12384 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:18:57.338017 12394 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:18:57.340373 12269 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:57.342572 12309 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.001s
W20260709 13:18:57.347651 12208 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.823s	user 0.012s	sys 0.000s
I20260709 13:18:57.361730  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:57.349211 12322 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.620s	user 0.006s	sys 0.006s
I20260709 13:18:57.374984  7232 raft_consensus.cc:2368] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17.
I20260709 13:18:57.375115  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:57.377941  6521 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
W20260709 13:18:57.368420 12191 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 13:18:57.392247  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1444) took 1772 ms. Trace:
I20260709 13:18:57.335748 12245 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:18:57.392885  7229 rpcz_store.cc:276] 0709 13:18:55.620165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.620294 (+   129us) service_pool.cc:224] Handling call
0709 13:18:55.625982 (+  5688us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.628182 (+  2200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.628222 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.628680 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.698259 (+ 69579us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:55.698292 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.698365 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.391821 (+693456us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b18004593a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04bcf5558f 00007f04b5f94571 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd
0709 13:18:56.944036 (+552215us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:57.391886 (+447850us) raft_consensus.cc:1685] finished
0709 13:18:57.391922 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.392122 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4738,"raft.queue_time_us":139878,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12521,"spinlock_wait_cycles":15377536,"thread_start_us":69222,"threads_started":1,"wal-append.queue_time_us":158692}
I20260709 13:18:57.397719  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:57.232153 12256 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:57.403164 12361 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:18:57.395949 12315 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:57.414604 12237 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:18:57.447844 12391 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:57.447873 12237 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:57.448768 12237 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:57.453974 12361 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.457190 12361 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:57.459410 11826 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.003s
I20260709 13:18:57.459846  6589 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:57.465603 12237 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.479391  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:57.480187  6522 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:57.468457 12240 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:57.379205 12322 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.492226  6522 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:57.491600 12240 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:57.398489 12191 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:57.513420 12198 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.000s
I20260709 13:18:57.505272 12237 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:57.446985 12327 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.521333 12341 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:57.552760 12199 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.005s
W20260709 13:18:57.550261 12206 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.647s	user 0.004s	sys 0.004s
W20260709 13:18:57.546579 12280 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.816s	user 0.009s	sys 0.001s
I20260709 13:18:57.558210 11856 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:57.553591 12266 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:57.575645  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2786) took 1170 ms. Trace:
I20260709 13:18:57.581765  6591 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:18:57.581666  6665 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:57.586802  6664 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:18:57.583580  6517 rpcz_store.cc:276] 0709 13:18:56.400036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.419634 (+ 19598us) service_pool.cc:224] Handling call
0709 13:18:57.569753 (+1150119us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540007dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:57.569892 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767018368}
I20260709 13:18:57.587252  6664 pending_rounds.cc:85] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 5
I20260709 13:18:57.587601  6664 pending_rounds.cc:107] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:18:57.587872  6664 raft_consensus.cc:2889] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:57.582659 12237 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:57.582692 12277 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.528088 12198 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.580718 11856 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:57.586408 12341 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.550526 12240 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.602077  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:57.576301 12199 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.617615  6665 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:57.604712 12237 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.629470  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1115) took 1196 ms. Trace:
I20260709 13:18:57.609122 12277 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.629488 12191 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.629980  6513 rpcz_store.cc:276] 0709 13:18:56.433175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.727473 (+294298us) service_pool.cc:224] Handling call
0709 13:18:56.727741 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.981840 (+254099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.629209 (+647369us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540031fbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:57.629400 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582573312}
I20260709 13:18:57.631934 12199 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.639034  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:57.642761 11856 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.644146 11856 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:57.635550  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.648562  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:57.619349 12341 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.621582 12307 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.658562  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:57.701987  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:57.656008  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.676751  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2778) took 1200 ms. Trace:
I20260709 13:18:57.674172  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2823) took 1161 ms. Trace:
I20260709 13:18:57.642168 12345 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.670324  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:57.704470  6667 rpcz_store.cc:276] 0709 13:18:56.476421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.492659 (+ 16238us) service_pool.cc:224] Handling call
0709 13:18:56.492937 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.202100 (+709163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.202151 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.202209 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:57.202254 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.676525 (+474271us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54000e65d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:57.676570 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.676684 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630298368}
I20260709 13:18:57.705451  6593 rpcz_store.cc:276] 0709 13:18:56.513217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.536291 (+ 23074us) service_pool.cc:224] Handling call
0709 13:18:57.673914 (+1137623us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540019e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:57.674096 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406304640}
I20260709 13:18:57.711589  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:57.634275 12198 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:57.716650  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:57.661103 12280 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:57.668089 12277 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:57.717792  6665 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:18:57.701296 12353 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.659060 12208 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:57.705379 12238 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:57.707311 12327 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:57.720273  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:57.713326 11826 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.724648 12269 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:18:57.727748 12302 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.728366 12245 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:57.722445 11669 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:57.754098  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.725481 12327 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:57.739988  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:57.757599  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.755472  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:57.732674 12208 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:57.763301  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 57355.7us
I20260709 13:18:57.719604 12353 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:57.765035  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 94789us
I20260709 13:18:57.754248 12245 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:57.767129  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.767743  6665 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
W20260709 13:18:57.767906  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.769963  6667 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:57.759363 12327 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:57.767867  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:57.733598 11826 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.766386 11669 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.773370 12245 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.773908 12345 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:57.775166 11669 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.775686 11669 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:57.776415  6667 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:57.777541  6667 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:57.777694 12364 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:57.783957  6592 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:57.785348  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.783965 12390 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.703s	user 0.004s	sys 0.000s
I20260709 13:18:57.787011 12353 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.787344 12409 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.787846 12409 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:57.789686  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.790102  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.790166 12409 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:57.791793  6665 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:57.785279 12208 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.792845 12341 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.798025 12307 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:57.798117 12273 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:57.805775  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.806258  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.806747 12364 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:57.810892 12405 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.111s	user 0.002s	sys 0.000s
I20260709 13:18:57.809237 12327 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.815183 11826 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:57.815333 12272 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:57.815714 12257 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.815867  6670 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:57.816687  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.817299  6593 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:18:57.817749  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:57.818120  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.818909  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.819381  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.818842  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.821341  6590 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 9.
I20260709 13:18:57.822410  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.819577  6592 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:18:57.851166  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:57.882980  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:57.821907  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:57.917138  6595 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:57.917642 11826 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:57.917924 12017 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.917201  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:18:57.919222  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 65816.8us
I20260709 13:18:57.932600  6637 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:57.945148  6637 leader_election.cc:403] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:18:57.938887  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 1.21833e+06us
I20260709 13:18:57.921429  7200 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:57.948127  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:57.949644  6665 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:57.950354  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:57.954941  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 181991us
I20260709 13:18:57.955969  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:57.957232  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 135388us
I20260709 13:18:57.958847  6588 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:57.958470  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:57.960285  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:57.964299 11867 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:57.965258 11856 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:57.966838 12269 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:57.971064  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.971642  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.972190  6562 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.795s (SENT)
I20260709 13:18:57.972537  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:57.973464 12353 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:57.974108  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:57.974749 12084 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.974818 12341 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.975313 12084 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.975664 12341 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:57.977516 12084 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:57.978046 12084 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:57.970928 12361 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:57.979180 12361 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.979568 12361 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:57.981262 12307 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:57.981665 12307 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.982012 12307 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:57.887286 12327 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:57.984880 11826 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:57.985460 11826 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.985922 11826 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:57.989271  6670 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:57.987095  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:57.994112  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44598.7us
I20260709 13:18:57.995187  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:57.999670  6486 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.114s (SENT)
W20260709 13:18:58.003268  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.003906  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.009660  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:57.816206 12257 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.014817  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:58.017750  6665 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:57.978482 12341 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:57.992271  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:58.025704  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:58.026747  6670 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:57.963171 12238 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:58.027746 12266 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:18:58.016183 12084 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:18:58.028108 12202 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:18:58.041103 12307 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.043316 12307 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.011878 12314 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.012110 12364 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:58.046247  6589 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
W20260709 13:18:58.048403  7234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1615) took 1468 ms (client timeout 1922 ms). Trace:
W20260709 13:18:58.052706  7234 rpcz_store.cc:269] 0709 13:18:56.580459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.714431 (+133972us) service_pool.cc:224] Handling call
0709 13:18:58.048206 (+1333775us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54005195d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.048335 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038707584}
I20260709 13:18:58.028708 12409 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:58.072170  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 116299us
I20260709 13:18:58.011474 11867 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:18:58.076499  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:58.079345  6665 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:18:58.080330  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:58.033820 12341 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:58.097005  6668 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:58.052429 12364 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:58.097801 12406 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:58.094251  6671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2764) took 1917 ms (client timeout 2114 ms). Trace:
I20260709 13:18:58.102327 12364 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.099449 12277 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:58.099634 12313 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:58.112879 12326 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.116737 12283 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.115653  6671 rpcz_store.cc:269] 0709 13:18:56.176683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.283649 (+106966us) service_pool.cc:224] Handling call
0709 13:18:58.094013 (+1810364us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000cba54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.094163 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381216512}
I20260709 13:18:58.128320  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:58.126163 12313 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:58.130136 12313 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.130813 12326 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.131851 12313 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.133049 12326 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.125675 12410 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:58.115367 12238 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.082649 12409 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.125598 12084 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:58.134545 12238 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.134897 12084 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.136049 12409 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.136675 12238 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.120496  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1114) took 1021 ms. Trace:
I20260709 13:18:58.137398  6663 rpcz_store.cc:276] 0709 13:18:57.099882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.195309 (+ 95427us) service_pool.cc:224] Handling call
0709 13:18:58.120325 (+925016us) spinlock_profiling.cc:243] Waited 72 ms on lock 0x7b54000e6ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.120436 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165657472}
I20260709 13:18:58.138507  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:58.139701  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.140414  6663 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:58.141084  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:58.141680 12084 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.134137 12283 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.144070 12223 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.144469 11940 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.144582 12283 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.126300 12277 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:58.124686  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48262.9us
I20260709 13:18:58.145481 12277 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:58.146795  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:58.148425 11940 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:58.150490  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:58.125757  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.112602 12341 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:18:58.088415  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 96274.7us
I20260709 13:18:58.141706  6663 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:58.180157  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 258772us
I20260709 13:18:58.180997  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.181576  6592 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.179706  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:58.187397  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 137121us
I20260709 13:18:58.150368 12314 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:58.188930  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.190099  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:58.197459  7232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1603) took 1691 ms (client timeout 1795 ms). Trace:
W20260709 13:18:58.197894  7232 rpcz_store.cc:269] 0709 13:18:56.506484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.506594 (+   110us) service_pool.cc:224] Handling call
0709 13:18:58.197264 (+1690670us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54006a3bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.197395 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603174656}
W20260709 13:18:58.200479  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50056.3us
I20260709 13:18:58.203255  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:58.204828 12197 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:58.205513 12197 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:58.207178  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.208246  7234 raft_consensus.cc:2410] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:58.209553  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.210404  7230 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:58.212405 12410 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:58.212785 12273 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:58.212983 12410 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.213549 12410 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.207321 12238 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:58.216359  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2393) took 1482 ms. Trace:
I20260709 13:18:58.216835  6668 rpcz_store.cc:276] 0709 13:18:56.733863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.773859 (+ 39996us) service_pool.cc:224] Handling call
0709 13:18:58.094157 (+1320298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.102599 (+  8442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.216170 (+113571us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000cba50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.216304 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2973487360}
I20260709 13:18:58.216907 12238 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.218181  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.219931  7230 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:58.222134  7234 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:58.224759  6419 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:58.226569  7231 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:58.227475 11910 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.228020 11910 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.228045  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1108) took 1247 ms. Trace:
I20260709 13:18:58.230600 11910 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.230883 12238 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.230636  6662 rpcz_store.cc:276] 0709 13:18:56.980480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.980642 (+   162us) service_pool.cc:224] Handling call
0709 13:18:58.227860 (+1247218us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54001e0354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.227990 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764097280}
I20260709 13:18:58.232973  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.232687 12197 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.233546  6662 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:18:58.234556 12324 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.234851  6662 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:18:58.236835  6662 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:58.237080  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 2828) took 1690 ms. Trace:
I20260709 13:18:58.238502  6591 rpcz_store.cc:276] 0709 13:18:56.546812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.805791 (+258979us) service_pool.cc:224] Handling call
0709 13:18:58.236840 (+1431049us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54002fdfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.237006 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612305664}
I20260709 13:18:58.239187  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2761) took 2063 ms. Trace:
I20260709 13:18:58.239566  6664 rpcz_store.cc:276] 0709 13:18:56.175849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.417521 (+241672us) service_pool.cc:224] Handling call
0709 13:18:56.417774 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.588387 (+1170613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.238959 (+650572us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540030ff54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:58.239134 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2679908608}
I20260709 13:18:58.242177  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.245587 12302 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:58.246605  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.247597  7229 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:58.250125  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.250733  7237 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:58.250679  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:58.245965  7230 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:58.255052  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:58.259195 12322 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.261248  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.261940  6594 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:58.267746  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:58.269615 12348 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:58.270046 12348 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.270413 12348 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.274804 12198 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:58.266496  6592 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:18:58.275725 12198 consensus_queue.cc:1243] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:18:58.276412 12198 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:58.276647  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:58.276906 12198 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:58.277401  7231 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.278066 12198 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.284965 12198 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:58.286911  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.286805 12256 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:58.173609 11940 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.185456 12206 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.125976 12406 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.198015 12408 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:58.333094  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:58.309103  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.334034  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.295426  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.303742  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.335032  6590 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:58.315855  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.335914  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.304565  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:18:58.308287  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.319849  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.320982  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.317251  7228 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
W20260709 13:18:58.338152  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.339792  6592 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:18:58.340972  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.341691  6592 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:18:58.328073  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.328905  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.297026  6665 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:18:58.343576  6520 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:58.302502  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:58.346992  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44583.7us
I20260709 13:18:58.346796  6419 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:58.350660  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:58.347808  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:58.351915  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.352593  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.353225  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:58.294359  6417 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 251) took 34879 ms (client timeout 15000 ms). Trace:
I20260709 13:18:58.353636  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.367287  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.367884  6595 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:58.368009  6593 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:58.354592  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.354842  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.370960  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:58.303175 12340 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.373328  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:58.373504 12340 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.374431  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.375605  7232 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.376037 12406 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.376479 12277 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.365473 12408 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:18:58.378079 12408 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.378878  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.379225  6594 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:58.379635  6588 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:58.358497 12314 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:18:58.187422 12283 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.149s
I20260709 13:18:58.178515 12341 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.380726  6594 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:58.381124  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:58.381733  7237 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:58.361763  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.381363  6517 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:58.362742  6515 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:58.382843  6592 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:18:58.383443  6593 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:18:58.385411  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.386395  6590 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:58.390846  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:58.398255  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.401503  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.401571  6517 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:58.406402  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.407814  7236 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:18:58.366870  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:18:58.411463 12416 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:58.412235 12416 consensus_queue.cc:1243] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:18:58.414108 12404 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.641s	user 0.003s	sys 0.002s
W20260709 13:18:58.414690 12420 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:18:58.416476 12403 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.649s	user 0.001s	sys 0.002s
I20260709 13:18:58.322914 12266 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:18:58.351966 12238 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:58.398357 12277 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.407832  6419 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 4 to 11, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 11 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:58.351145 12273 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.431346  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.411386  6519 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:18:58.411406  6521 raft_consensus.cc:606] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:58.429463  7232 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:58.435325  6515 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:58.441452  6595 raft_consensus.cc:2410] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:18:58.443079  6670 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:58.451300  6594 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:58.455276  7237 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:58.453284  6419 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 8, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:58.466372  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.488401  6590 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:58.488643  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:58.472456  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:58.490422  6592 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:18:58.495003  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 124128us
I20260709 13:18:58.495970  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:58.299294 11910 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
W20260709 13:18:58.436282 12407 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.644s	user 0.000s	sys 0.003s
I20260709 13:18:58.473580  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:18:58.445886 12017 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.002s
I20260709 13:18:58.511840  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.347008 12206 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.507537  6519 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:18:58.324668 11940 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.430485 12266 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:58.481294  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 130755us
I20260709 13:18:58.535104 12266 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.535830 12266 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:58.503314 12202 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:18:58.504724  6419 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 11, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 11 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:58.539671 12202 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:18:58.540377 12202 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:58.540828 12202 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:58.540903  6419 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 14, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 14 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:58.544974 12413 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:58.545425 12413 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.545553  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2776) took 2069 ms. Trace:
I20260709 13:18:58.545760 12413 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:58.546139  6669 rpcz_store.cc:276] 0709 13:18:56.475785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.475966 (+   181us) service_pool.cc:224] Handling call
0709 13:18:56.476296 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.570955 (+ 94659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.571004 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.580594 (+  9590us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.668840 (+1088246us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b180030be00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:18:57.769157 (+100317us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:57.769184 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.283464 (+514280us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b54001ed7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:58.283519 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.435676 (+152157us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:18:58.545265 (+109589us) raft_consensus.cc:1685] finished
0709 13:18:58.545296 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.545481 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35366,"raft.queue_time_us":35588,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":43265,"spinlock_wait_cycles":875180928,"thread_start_us":100172,"threads_started":1,"wal-append.queue_time_us":1159157}
I20260709 13:18:58.498409 12273 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.552305 12273 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.476688 12199 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:58.482167 12191 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.553365 12199 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.553752 12191 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.555644 12199 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.556422 12191 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:58.556739  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:58.557727 12324 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:58.484753 12416 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.558966  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:58.455494 12237 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.518913  6596 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:18:58.446444 12257 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.580844 12400 raft_consensus.cc:2764] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:18:58.501631 12269 consensus_meta.cc:287] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 13:18:58.581857 12269 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:58.461918 11669 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:18:58.440526 12334 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:58.582265 11940 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:58.582587 11669 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:58.583055 11669 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:58.442020 11826 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.583822 11669 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.584263 12364 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.584976 11669 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.585821 11826 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:58.435031 12238 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.586783 12206 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.587317 12364 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.588383 12238 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.527165 12326 raft_consensus.cc:2764] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:18:58.589469  6671 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
W20260709 13:18:58.591084 12353 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.617s	user 0.005s	sys 0.005s
I20260709 13:18:58.591606 12353 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:58.577841 12361 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.598s	user 0.006s	sys 0.004s
I20260709 13:18:58.592722 12361 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.597412 12326 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.598055 12326 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.604053 12341 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.542042 12202 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.605784 12202 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.619503 12340 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.641197  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:58.533802  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:58.524690  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:58.642552  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:58.531486 12406 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:58.487579  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.560698  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:58.515390  7228 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:18:58.513456  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.559543  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.442745 12240 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:58.370265  6417 rpcz_store.cc:269] 0709 13:18:23.415091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.415355 (+   264us) service_pool.cc:224] Handling call
0709 13:18:58.294235 (+34878880us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:53.113099 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:53.113820 (+   721us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:53.113871 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:18:53.113894 (+    23us) tablet.cc:662] Decoding operations
0709 13:18:53.131104 (+ 17210us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:53.131204 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:53.131236 (+    32us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:18:53.183962 (+ 52726us) tablet.cc:701] Row locks acquired
0709 13:18:53.184020 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:18:53.184197 (+   177us) write_op.cc:270] Start()
0709 13:18:53.184341 (+   144us) write_op.cc:276] Timestamp: P: 1783603133184133 usec, L: 0
0709 13:18:53.184379 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:18:53.185052 (+   673us) log.cc:844] Serialized 4710 byte log entry
0709 13:18:53.742542 (+557490us) op_driver.cc:464] REPLICATION: finished
0709 13:18:53.881455 (+138913us) write_op.cc:301] APPLY: starting
0709 13:18:53.881578 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:53.970939 (+ 89361us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:53.971048 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:54.086351 (+115303us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:54.178785 (+ 92434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 13:18:54.195393 (+ 16608us) write_op.cc:312] APPLY: finished
0709 13:18:54.229323 (+ 33930us) log.cc:844] Serialized 113 byte log entry
0709 13:18:54.229515 (+   192us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:54.229656 (+   141us) write_op.cc:454] Released schema lock
0709 13:18:54.229723 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:54.230146 (+   423us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:54.230173 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2214510,"reported_tablets":44,"spinlock_wait_cycles":1792,"child_traces":[["op",{"apply.queue_time_us":138505,"apply.run_cpu_time_us":21014,"apply.run_wall_time_us":348699,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54263,"num_ops":12,"prepare.queue_time_us":355435,"prepare.run_cpu_time_us":26622,"prepare.run_wall_time_us":215672,"raft.queue_time_us":211965,"raft.run_cpu_time_us":21696,"raft.run_wall_time_us":152857,"replication_time_us":532765,"thread_start_us":419544,"threads_started":4,"wal-append.queue_time_us":146209,"wal-append.run_cpu_time_us":9831,"wal-append.run_wall_time_us":1905358}]]}
I20260709 13:18:58.578459  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:58.527542 12322 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:58.453711 11856 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:18:58.535194 11910 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:58.644116  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:58.649029  6521 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:58.652961  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 128371us
I20260709 13:18:58.724049  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:58.653522  6593 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:58.655788  6513 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
W20260709 13:18:58.668928  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 108329us
I20260709 13:18:58.649730  6514 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:18:58.650859  6663 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:58.654596  6419 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 10 to 13, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 13 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:58.730211  7231 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:18:58.718567  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1233) took 2123 ms (client timeout 1558 ms). Trace:
W20260709 13:18:58.731979  6596 rpcz_store.cc:269] 0709 13:18:56.548574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.556811 (+  8237us) service_pool.cc:224] Handling call
0709 13:18:58.671539 (+2114728us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b540039f8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:58.671637 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4321921408}
W20260709 13:18:58.678198  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.735010  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.736405  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.735203  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.739939  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.744361  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:58.745165  6662 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:18:58.746713  6670 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:58.748277  6419 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 12, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 12 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:58.750849  6419 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 10, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 10 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:58.756040  6419 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:58.758419 12353 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.763365  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:58.661562 11867 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.042s	sys 0.054s
I20260709 13:18:58.763648  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:58.770704  7233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.034s	sys 0.020s
I20260709 13:18:58.641708 12345 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.777630  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:58.791031  7233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.034s	sys 0.020s
W20260709 13:18:58.775302 12197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.016s	sys 0.036s
I20260709 13:18:58.668290 12406 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:58.788733  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.795542  6521 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:18:58.638247 12410 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.707923 12341 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:58.717445 12400 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:18:58.721972 11910 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:58.766320 11826 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:58.766413 12324 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:58.793599  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:18:58.821956  6667 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.011s	sys 0.019s
W20260709 13:18:58.822355  6667 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.019s
I20260709 13:18:58.794188 12237 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.800354  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:58.792997 12197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.016s	sys 0.036s
I20260709 13:18:58.815877  6419 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:58.703751 12416 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:58.824232  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.793195 11867 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.042s	sys 0.054s
W20260709 13:18:58.824658  6597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.009s	sys 0.005s
W20260709 13:18:58.836731  6597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.009s	sys 0.005s
I20260709 13:18:58.837989  7235 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
W20260709 13:18:58.827886  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 480109us
I20260709 13:18:58.829363  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:58.826006 12208 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.011s	user 0.007s	sys 0.006s
I20260709 13:18:58.839267  6520 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:18:58.839563 12208 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.007s	sys 0.006s
I20260709 13:18:58.827940 11940 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:58.797324 12406 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:58.850872  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 345542us
I20260709 13:18:58.830518  7234 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
I20260709 13:18:58.797804 12345 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.814582 12314 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:58.833339  6419 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:18:58.827436  6562 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.010s	sys 0.022s
I20260709 13:18:58.823774 12361 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:58.795976 12412 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:58.795567 12353 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.853845  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:58.830991 12341 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:18:58.846993 12348 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.576s	user 0.007s	sys 0.003s
W20260709 13:18:58.858645  6562 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.010s	sys 0.022s
I20260709 13:18:58.836807 11826 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:18:58.863308 12313 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.005s
I20260709 13:18:58.821787 12400 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:58.837217 12256 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.003s
I20260709 13:18:58.796888 12271 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.893525  6419 catalog_manager.cc:5697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:18:58.831962  6521 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:58.864797  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:58.867146  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:58.869657  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 562604us
W20260709 13:18:58.899887  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.879402  7229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.015s	sys 0.017s
I20260709 13:18:58.879616  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.901168  6514 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:18:58.902159  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.881683  6515 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:58.903581  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.904280  6515 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:18:58.897927  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.906126  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:58.908392  6514 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:18:58.861562  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:18:58.911036  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49557.4us
I20260709 13:18:58.834431 12237 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.912045  6563 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:58.870524 12273 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.862520 12084 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:18:58.855882  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:18:58.895622 12423 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:18:58.924373  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68550.8us
W20260709 13:18:58.913197  7229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.015s	sys 0.017s
I20260709 13:18:58.915716  6514 pending_rounds.cc:85] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 9
I20260709 13:18:58.896432  6663 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:18:58.926734  6663 pending_rounds.cc:85] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 12
I20260709 13:18:58.927006  6663 pending_rounds.cc:107] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:18:58.927248  6663 raft_consensus.cc:2889] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:58.853070 11910 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.920163  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:58.929540  6515 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:58.912678  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45631.1us
I20260709 13:18:58.921492  6521 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:18:58.870563 12199 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:18:58.865971 12334 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:18:58.928043  6514 pending_rounds.cc:107] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:18:58.865581 12418 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:18:58.925824 12273 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.869107 12302 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:18:58.902376 12271 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.930876  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:58.880080 12348 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.934412 12433 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:58.939806 12433 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:58.940240 12348 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:58.919166 12361 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:58.940810 12361 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.941226 12361 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.942906  6517 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:58.943689 12198 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.001s
I20260709 13:18:58.944839  6522 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:58.946107 12314 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.947690 12400 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.920600 12340 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:58.948613 12340 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.948985 12340 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:58.950946  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.951481 12364 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:58.951984 12364 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:58.952546 12416 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:58.952740 12364 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:58.930497 12206 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:58.954741 12206 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:58.955586 12206 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.923404 11573 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:58.937354 12302 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:58.960196 12302 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:58.960506  6419 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 8, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 8 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:18:58.961369  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2399) took 1005 ms. Trace:
I20260709 13:18:58.937570 12273 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.878813 12417 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:18:58.937891 12334 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:58.963467 12334 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:58.911777 12245 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.136s	user 0.013s	sys 0.031s
I20260709 13:18:58.964262 12334 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:18:58.964936 12245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.189s	user 0.013s	sys 0.031s
I20260709 13:18:58.965461 12334 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:58.960940 12302 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.966024  6666 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:58.934882  6514 raft_consensus.cc:2889] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:58.940409 12271 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:18:58.987776  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 165054us
W20260709 13:18:58.896746  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 431280us
I20260709 13:18:58.928049 12326 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:58.961786  6668 rpcz_store.cc:276] 0709 13:18:57.956266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.219214 (+262948us) service_pool.cc:224] Handling call
0709 13:18:58.219551 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.222373 (+  2822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.222401 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.222903 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.670033 (+447130us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:58.670075 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.670143 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.961178 (+291035us) raft_consensus.cc:1685] finished
0709 13:18:58.961197 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.961318 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":81,"thread_start_us":446640,"threads_started":1,"wal-append.queue_time_us":571628}
I20260709 13:18:58.968022  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:58.921466  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:18:58.994562  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 73515.6us
I20260709 13:18:58.962523  6419 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:58.990861  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 60450.8us
I20260709 13:18:58.997319  6419 catalog_manager.cc:5697] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 9, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 9 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:58.999668  6593 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:59.000619  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.001262  6596 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:59.003186  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.005313  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.995771  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.008713  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:58.962857 12019 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:58.938584 12237 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.005921  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:58.950132 12361 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.012154  6488 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.019110  6668 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:18:59.019744  6596 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:18:59.020385  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.995282  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:59.025264  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:18:59.023627  7200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.004s	sys 0.006s
W20260709 13:18:59.029289  7200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.004s	sys 0.006s
W20260709 13:18:59.029549  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 540974us
I20260709 13:18:59.022105  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:59.029682 12436 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:59.023294  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:59.030969  6670 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:59.038234  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:59.009933 12017 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:59.039047 12341 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.039628 12341 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.040030 12341 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.030368 12436 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:59.045696  6596 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:18:59.046075  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.047762  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.048051  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.056224  6521 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:59.049603  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54598.9us
I20260709 13:18:59.009629 12257 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:59.009815 12202 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:59.101438  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 45782.8us
I20260709 13:18:59.167336  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:59.113472  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.018682 12019 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:18:59.020704 12419 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
W20260709 13:18:59.101814  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59618us
W20260709 13:18:59.140024  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 127945us
I20260709 13:18:59.107213 12273 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.140622  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.107283  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:59.208423  6665 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:59.208729  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.209488  7231 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:18:59.209645 12424 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:18:59.133317  6513 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:59.141621  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.210856  6596 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:18:59.138934  6590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:59.211724  6590 raft_consensus.cc:606] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
W20260709 13:18:59.102310 12413 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 13:18:59.212877 12413 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:58.998458 12406 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.173081  6592 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
W20260709 13:18:59.183938  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.110512 12430 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:58.940704 12433 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.101627 11940 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:59.203431  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.035449 12302 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:59.144258 12313 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:59.166348 12257 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.174445  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2396) took 2007 ms. Trace:
I20260709 13:18:59.214075  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:18:59.221705  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 82261.9us
I20260709 13:18:59.140547 12410 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:59.223766  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.224550  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:59.225483  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.226655  6590 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:18:59.206247  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.230189  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.166239 12202 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.234043  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:18:59.252099  6595 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:59.210668 12198 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:18:59.101181 12412 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:18:59.244057  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.279053 12412 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:59.279872  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:59.282677  7228 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:18:59.241978  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.242178  6666 rpcz_store.cc:276] 0709 13:18:57.166761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.764046 (+597285us) service_pool.cc:224] Handling call
0709 13:18:58.220098 (+456052us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:18:58.220278 (+   180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.966452 (+746174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.174233 (+207781us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540011c6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:59.174389 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717687296}
I20260709 13:18:59.249859  7234 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:59.257009  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.284833  6513 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:59.287531  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:59.229235 11940 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:59.288851  7231 raft_consensus.cc:2368] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:59.251325 12257 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:59.255177 12202 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.234447 12451 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:59.257805  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:59.263581  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2788) took 1443 ms. Trace:
W20260709 13:18:59.260238 12434 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:18:59.264780 11573 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.228796 12273 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.265230 12436 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.222262 12444 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.291411 11940 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:59.308142  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.310454 12449 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:59.310871 12449 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:59.311569 12341 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.311715 12449 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.312995 12449 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:59.313651 12341 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:59.316313 12197 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:59.316763 12197 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:18:59.317590 12197 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.293525 12416 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:18:59.318805  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:59.319550  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1629) took 1067 ms. Trace:
I20260709 13:18:59.319926  7230 rpcz_store.cc:276] 0709 13:18:58.297217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.301486 (+  4269us) service_pool.cc:224] Handling call
0709 13:18:58.301909 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.497694 (+195785us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:58.517557 (+ 19863us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.864893 (+347336us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.864961 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:58.865019 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.319316 (+454297us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54006cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:18:59.319374 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.319493 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21659,"prepare.queue_time_us":28306,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":23269,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2125,"spinlock_wait_cycles":500366464,"thread_start_us":323442,"threads_started":1,"wal-append.queue_time_us":415913}
I20260709 13:18:59.322060  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.322695  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:59.323503  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.324116  7235 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:18:59.324954  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.320977 12257 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.325083  6521 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:59.325639  7235 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 10: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:59.328989  7230 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:59.273501 12353 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:18:59.278079 12240 consensus_meta.cc:287] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.000s
I20260709 13:18:59.256687 12199 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:18:59.291446  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.264150 12430 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:59.304097  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:59.289077 12410 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:59.335371  7228 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:59.326349  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.335892  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.336710  7235 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:59.296670  6667 rpcz_store.cc:276] 0709 13:18:57.819979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.820086 (+   107us) service_pool.cc:224] Handling call
0709 13:18:57.961399 (+141313us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:57.961920 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.961963 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:58.013783 (+ 51820us) log.cc:844] Serialized 52 byte log entry
0709 13:18:58.174262 (+160479us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:58.822789 (+648527us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.826110 (+  3321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.826190 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.263359 (+437169us) raft_consensus.cc:1685] finished
0709 13:18:59.263381 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.263525 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":749794,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":33,"raft.queue_time_us":10842,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":3706,"thread_start_us":597937,"threads_started":2,"wal-append.queue_time_us":702583}
I20260709 13:18:59.250638 12340 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.324788 11940 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:59.273128 12266 consensus_meta.cc:287] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.737s	user 0.005s	sys 0.003s
I20260709 13:18:59.343760  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.344630  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.357247  7232 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:59.358284  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.359035  7232 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:18:59.345273  6671 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:18:59.357111  7230 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:18:59.327061 12444 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:59.363590  7231 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:59.369122  7237 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
W20260709 13:18:59.356365 12438 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.183s	user 0.003s	sys 0.000s
I20260709 13:18:59.334911 12364 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:59.356730 12425 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.423s	user 0.001s	sys 0.000s
I20260709 13:18:59.360273 12238 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.385442  7228 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:18:59.403064  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.403867  7232 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:18:59.405128  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:59.403818  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.406600  7228 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:18:59.416560  6669 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:59.370220 12444 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.301226 12019 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.362792 12266 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.429092  7232 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:59.387099 12456 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.431766  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:59.435055  7232 raft_consensus.cc:2376] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17. Re-sending same reply.
I20260709 13:18:59.435736  6596 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:59.442658  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:18:59.448752  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 126160us
I20260709 13:18:59.448956  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:18:59.449275  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1628) took 1199 ms. Trace:
I20260709 13:18:59.449790  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:59.449865  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.450882  7228 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:18:59.400462 12238 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:59.453363  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 250011us
I20260709 13:18:59.438731 12353 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.449784  7233 rpcz_store.cc:276] 0709 13:18:58.250054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.250237 (+   183us) service_pool.cc:224] Handling call
0709 13:18:58.250580 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.250911 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.250938 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.251341 (+   403us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.791586 (+540245us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:58.791636 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.791739 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.449014 (+657275us) raft_consensus.cc:1685] finished
0709 13:18:59.449042 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.449205 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":413,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":539832,"threads_started":1,"wal-append.queue_time_us":629475}
I20260709 13:18:59.457084 12326 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:59.457247  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.478402 11910 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.486362 12314 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:59.486888 12314 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:59.487041 12418 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.487473  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.487538 12418 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.487779 12314 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.488724  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.489784  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.490587 12198 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.463387 12271 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:59.466513  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.493968  6636 leader_election.cc:400] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12.
I20260709 13:18:59.495178  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:18:59.495282  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 267593us
I20260709 13:18:59.496970  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:59.471472 12328 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:59.473780 11856 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:59.475494 12437 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.530s	user 0.002s	sys 0.000s
W20260709 13:18:59.450846 12454 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:18:59.440568 12019 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:59.501344  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.490939 12429 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:59.490788 12199 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:59.457763 12237 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:59.443737 12324 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.492503  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1245) took 1308 ms. Trace:
I20260709 13:18:59.453163 12322 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:59.503770  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 278637us
I20260709 13:18:59.489841 12418 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.480145 12017 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:59.446861 12240 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:18:59.521309  6588 rpcz_store.cc:276] 0709 13:18:58.183678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.380263 (+196585us) service_pool.cc:224] Handling call
0709 13:18:58.380546 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.380756 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:18:58.834449 (+453693us) log.cc:844] Serialized 14 byte log entry
0709 13:18:59.295067 (+460618us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b18003c9100. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:18:59.295319 (+   252us) log.cc:844] Serialized 14 byte log entry
0709 13:18:59.328935 (+ 33616us) log.cc:844] Serialized 14 byte log entry
0709 13:18:59.492109 (+163174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:59.492161 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:59.492202 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.492279 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.492448 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118059,"prepare.queue_time_us":343785,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":3892,"raft.queue_time_us":53749,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":60138,"spinlock_wait_cycles":27371264,"thread_start_us":800650,"threads_started":3,"wal-append.queue_time_us":479812}
I20260709 13:18:59.484859 12413 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:59.480782 12406 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:59.480119 12361 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:59.500504 12328 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:59.443981 12456 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.566896 12461 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.528769 12322 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:18:59.575049  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.509416 12326 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:59.573312 12257 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.491645 12353 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.579164 12361 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.578603 12340 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.579293 11940 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:59.568686 11573 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:18:59.569666 12455 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:18:59.587693  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.565452 12017 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:18:59.610371  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46016.6us
W20260709 13:18:59.593778  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 137090us
I20260709 13:18:59.527052 12198 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:18:59.507246 11856 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:18:59.568228 12442 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:18:59.595402 12406 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.597563 12322 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:59.596762 12413 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.603503 12273 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:59.620682  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 164993us
I20260709 13:18:59.625752  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:18:59.627668  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:59.566653 12429 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:18:59.607810 11573 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:59.604332 12266 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.564189 12324 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.610975 12340 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:18:59.614133 12456 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.645623  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.622606 12406 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.661731  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.647027  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.662534  6519 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:59.647845  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.656514  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.663153  6520 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 10.
I20260709 13:18:59.663578  6515 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:59.664304  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.646893  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.654901  6514 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.646147  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.609879 12361 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.657912  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:59.659032  6662 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:59.703836  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.646304 12461 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:59.665469  6517 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:59.614274 12202 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.704628  6521 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
I20260709 13:18:59.660542  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:59.703840  6671 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:59.721585  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.667526  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:18:59.706666  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79079.4us
I20260709 13:18:59.723580  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:18:59.622697 12457 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:59.668087  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.707978  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:59.704878  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:59.707978  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.710503  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.666720 12340 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.650780 12307 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.711519  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:59.712185 12324 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.713939  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:59.751675  6666 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:59.714551  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.714989 12237 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.666574 12406 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.666395 12470 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:59.661821 12471 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.721663  6663 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.718778  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:18:59.684072  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.727353  6514 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:18:59.729116  6521 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:18:59.736641  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 210121us
I20260709 13:18:59.738333  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.739132  6520 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:59.737260  6518 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:18:59.732744  6671 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:18:59.753280  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.624097 12328 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.755867  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.643471 12413 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.709625 12461 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:18:59.755069 12206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.014s	sys 0.010s
W20260709 13:18:59.754984  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 257992us
I20260709 13:18:59.763299  6517 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:59.779521  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.728552 12469 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:59.787842  6520 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:59.784976  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:18:59.783747  6513 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:18:59.788877  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.789688  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:18:59.758807 12470 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.792804  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.789178  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:18:59.765599 12326 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.801550 12481 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:59.803977 12461 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.805321 12461 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:59.806875 12412 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:59.786906 12206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.015s	sys 0.010s
I20260709 13:18:59.759292 12334 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:18:59.780680 12464 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:59.815330  6517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:18:59.812721  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.825491  6517 raft_consensus.cc:606] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.825958  6517 raft_consensus.cc:614] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:59.826313  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.829205  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:18:59.758333 12307 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.826237  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:59.824682 12477 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:59.763628 12237 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.781239 12471 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.836347 12202 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.817961 12326 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.837025 12353 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:59.791461 12469 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.758543 12017 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.797967 12470 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:59.849558  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.855414 12469 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:59.873962  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2409) took 1318 ms. Trace:
I20260709 13:18:59.850594  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.851306  6520 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:18:59.839742 12199 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.853242  6521 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:59.853488  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.881834  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.859951 12238 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.862782 12345 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.860067  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:18:59.868208  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.840219 12464 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:18:59.845731 11940 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:59.852138  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.876399  6522 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:59.841156 12328 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.854282  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:18:59.860268 11867 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.883700  6520 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
W20260709 13:18:59.886912  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 60643us
I20260709 13:18:59.874975  6669 rpcz_store.cc:276] 0709 13:18:58.555767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.578193 (+ 22426us) service_pool.cc:224] Handling call
0709 13:18:59.873516 (+1295323us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540011dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:18:59.873813 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":723633536}
I20260709 13:18:59.837598 11826 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:18:59.895949  6515 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:18:59.841408 12429 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.898257  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:59.907189  6595 raft_consensus.cc:606] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.907559  6595 raft_consensus.cc:614] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:59.909124 12449 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:59.908991 12464 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.909857 12449 consensus_queue.cc:1243] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:18:59.847978 12348 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:18:59.914081 12474 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:18:59.915761 12237 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.916373 12237 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:59.918120 12307 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:18:59.921034 12475 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:18:59.896816 11940 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.899762  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:18:59.895759  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 172280us
I20260709 13:18:59.869155 12477 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:59.904804 11856 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.927045 12340 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.927558 12361 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:59.898216 12314 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:18:59.908022 11867 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.932547  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:18:59.934953  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 72585.1us
I20260709 13:18:59.940747 12485 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:59.937587 11856 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:59.943112 11856 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.943468 11856 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:59.946074 12471 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.935798 11910 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:59.947757 11910 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:18:59.948508 11910 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:18:59.949055  6663 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:18:59.939296 12314 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.949683 11910 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:59.950122  6513 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:18:59.950965 12314 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.951254 11867 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.953303 12413 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:18:59.953052 11856 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.955119 11856 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:18:59.958042 12412 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.930198 12353 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:59.959169  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:18:59.959290 12353 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.959852  7237 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:18:59.960736  6595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:18:59.961328  6595 raft_consensus.cc:606] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:18:59.961629 12353 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:18:59.931111 11940 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:59.962747  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:18:59.964486 12444 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.525s	user 0.006s	sys 0.003s
I20260709 13:18:59.964862  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:59.964911 12444 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:18:59.968411  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:59.969559  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:18:59.970870  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:59.904018  6522 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 11.
I20260709 13:18:59.950914  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:18:59.933779  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:18:59.972736  7235 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.971161  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:18:59.974253  6666 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:18:59.940459  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:59.976897  6666 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:59.974050  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:18:59.994971  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 205847us
I20260709 13:18:59.995129  6486 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:18:59.985730  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:00.000202  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:19:00.012741  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79035.7us
I20260709 13:19:00.019208  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:18:59.976656 12477 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:00.023041 12491 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:00.023659 12491 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:00.025985  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.026664  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:00.027558  6588 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:00.012005  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71603.8us
I20260709 13:19:00.031131  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:00.032332  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:00.033385  6594 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:19:00.024562 12491 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.040418  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.047693  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:00.048774  6590 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:00.065023  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:00.066107  6595 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:00.066140  7235 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:18:59.993898 11826 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:00.036407 12413 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:00.104758  6588 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:00.025393 12208 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:18:59.980242 12353 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 13:19:00.012035 12493 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:00.043648 11910 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:00.175906  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:00.176105  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 175999us
W20260709 13:19:00.075289 12488 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:19:00.182289  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:00.183724  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:00.060343 12477 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:00.187847  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:00.110219 12413 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.235708  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:00.237499  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 218392us
I20260709 13:19:00.239013  6666 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:00.239964  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.240950  7235 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
I20260709 13:19:00.242050  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.242889  7231 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:00.244288  7233 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:00.180676 11910 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.191198  7237 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:00.180871 12493 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.249624  7232 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:00.258183  7230 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:19:00.258347  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2556) took 1618 ms. Trace:
I20260709 13:19:00.263250  6516 rpcz_store.cc:276] 0709 13:18:58.640241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.644710 (+  4469us) service_pool.cc:224] Handling call
0709 13:19:00.258149 (+1613439us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b5400044554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:00.258291 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528056704}
I20260709 13:19:00.218214 12019 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.267580  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.268460  7231 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:00.269511  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:00.252737 12322 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.005s
I20260709 13:19:00.232885 12444 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:00.247340 12413 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:00.180565 12208 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.280903  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:00.254267 11910 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:00.282608  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.257076 12493 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:00.293661  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.278072 12490 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.309489  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.292152 12444 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:00.312952  7237 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:00.317408  7232 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:00.281513 12322 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:00.328922  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.300271 12353 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:00.278163 12019 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:00.331833  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.336577  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.319177 12444 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:00.338856  7232 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:00.349179 12019 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:00.357014  6666 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:00.313835 11940 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.362464  6668 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:19:00.362434  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 120987us
I20260709 13:19:00.337952 12500 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.350569 12444 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:00.314189 12490 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.351452 12353 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:00.372668  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:00.339699 12477 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.379539  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:00.380153  7233 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:00.484275 12208 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:00.504846 12461 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
I20260709 13:19:00.512944 12503 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:00.484616 12322 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:00.484984 12494 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:19:00.487931 11940 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:00.489801 12490 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.490679 12353 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:00.494071 12328 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.000s
I20260709 13:19:00.370343 12500 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.369863 12496 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:00.525296  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 496221us
I20260709 13:19:00.545783  7233 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:00.571991 12470 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.683s	user 0.007s	sys 0.004s
W20260709 13:19:00.573287 12406 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.807s	user 0.010s	sys 0.001s
I20260709 13:19:00.598009  6517 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:19:00.561470 12503 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.581310  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:00.572019 12504 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:19:00.605625  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:00.609309  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 396892us
W20260709 13:19:00.612890  6667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.001s
W20260709 13:19:00.614120  6667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.010s	sys 0.001s
W20260709 13:19:00.607044  6670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.006s	sys 0.003s
W20260709 13:19:00.618340  6670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.006s	sys 0.003s
I20260709 13:19:00.622452  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:00.600536 12245 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.630812  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:00.612764 11856 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:00.639178  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:00.642045  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:00.594472 12470 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.631714  6522 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:00.598400 12486 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:00.603210 12322 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:19:00.654822  6522 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:00.658464  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:00.660266  6516 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:00.628512 12238 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:00.602656 12406 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:00.664583 11669 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.013s	sys 0.018s
W20260709 13:19:00.668140 11669 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.014s	sys 0.018s
I20260709 13:19:00.668378  6665 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:00.602357 12496 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.632598 12485 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:00.633939 11856 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.672664 12485 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:00.672719 11856 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:00.674135 12485 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:00.611364 12503 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:00.671447 12469 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.799s	user 0.008s	sys 0.005s
W20260709 13:19:00.687295  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 314699us
I20260709 13:19:00.688391  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:00.669565 12238 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:00.635046 12245 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.691669 11669 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:00.693007  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:00.699455  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:00.695891 12238 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.702590 11669 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:00.701922 12412 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:00.712800 12197 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:00.719079 11669 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.719637 11669 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.721261 12513 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:00.722409 12500 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:00.724277  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.690701 12469 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.724820  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:00.725754  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.726135  7231 raft_consensus.cc:1217] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Deduplicated request from leader. Original: 10.10->[12.11-13.12]   Dedup: 12.11->[13.12-13.12]
I20260709 13:19:00.726358  7237 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:00.726668  7231 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [12.11-13.12]
I20260709 13:19:00.728417  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:00.728941 12340 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:00.729537 12340 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.731655 12340 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.721964 12513 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.734750  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:00.734886  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:00.735514  6515 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:00.735790  6522 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:19:00.732643 12501 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:19:00.752475  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.756060  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.758445 12406 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.751219 12413 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.761341 12413 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:00.762166  7229 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:00.763353  6667 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:19:00.762557 11910 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.795060 11910 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.771615  6520 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:19:00.784394  6670 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:00.796214  6520 pending_rounds.cc:85] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 9
I20260709 13:19:00.796538  6520 pending_rounds.cc:107] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:19:00.796801  6520 raft_consensus.cc:2889] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:00.775421 11573 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.767724  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:00.798735 12408 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:00.799332 11573 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:00.769093  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:00.800783  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:00.801448  6515 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:00.801546 12408 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:00.801911 12408 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.802275 12408 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:00.807336  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:00.807971  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:00.808197 12324 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:00.808733 12324 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:00.765548 12513 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:00.779943  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:00.768862 12509 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:00.810593 12509 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.809592 12324 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.812952 12509 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:00.809615 12513 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:00.815130 12513 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:00.816041 12513 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.732911 12461 raft_consensus.cc:2764] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:00.764299 12496 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:00.752629 12458 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:00.820220 12458 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:00.819950 12238 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.821188 12458 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.768714 12470 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.823617  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:00.824249  6666 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:00.764379  6516 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:19:00.824820  6516 pending_rounds.cc:85] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 7
I20260709 13:19:00.825104  6516 pending_rounds.cc:107] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:19:00.825409  6516 raft_consensus.cc:2889] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:00.825847  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:00.826434  7232 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:00.768114  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.871486  6662 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:00.830178  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:00.871767  7236 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:00.875818  6562 leader_election.cc:400] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
I20260709 13:19:00.876390  6562 leader_election.cc:403] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:00.819865  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2413) took 1162 ms. Trace:
I20260709 13:19:00.876551  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:00.831554  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:00.877409  6664 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:00.876920  6665 rpcz_store.cc:276] 0709 13:18:59.657333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.657472 (+   139us) service_pool.cc:224] Handling call
0709 13:19:00.819666 (+1162194us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54000e56d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:00.819807 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141213952}
I20260709 13:19:00.878717  6593 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:00.831265  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:00.885923  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:00.828200  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:00.886924  7234 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:00.887180  7232 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.891129  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.892490  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.895359  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:00.895963  6591 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:00.904125  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:00.904364  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73211.7us
I20260709 13:19:00.905066  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:00.905934  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:00.906275  6670 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:00.728323 12412 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.906400  6664 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:00.907814  7233 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:00.910166  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.782390 12481 raft_consensus.cc:993] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:19:00.827126 12473 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.763793 12245 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.770329 12469 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.914162  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.870800 12237 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.917155  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.888710 12361 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:00.942853 12361 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:00.943497 12361 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:00.944092 12238 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:00.945019 12469 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:00.945632 12469 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:00.883404 12505 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:00.946794 12505 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.946702 12469 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.905151 12340 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:00.949193 12505 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:00.949301 12340 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:00.950585 12340 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:00.928786  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.928915  6666 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:00.921042  7232 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:19:00.888371 12476 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.914951 12271 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.767843 12444 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:00.931938  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:00.932998  6665 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:00.939010 12345 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:01.024631  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:01.025463  6666 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:00.938915  6593 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:00.894523 12206 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:00.968452  6670 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:00.973747  7234 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
W20260709 13:19:01.014117  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.955030 12237 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.020272  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:00.910995 11910 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:01.022281  6664 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:00.774068 12493 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.055411 12237 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:01.056648 12493 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:01.058920 11910 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.058921 12470 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:01.060346 11940 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:01.061795 11940 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.064241 11910 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:01.066012 11940 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:01.066579  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.067981 12361 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.069446  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:01.070899  6668 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:19:01.076903  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.093328  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:00.923349 12412 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.048960 12345 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:00.957543 11856 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.156073  6594 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:19:01.138123  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.089903 12470 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:01.151440  6589 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:01.160312  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.138198  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:01.152604  6597 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:01.153190  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:00.927186 12473 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.046269 12490 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 13:19:01.091913 12482 raft_consensus.cc:1081] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:19:01.200352 11856 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:01.138772 12491 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.010s	sys 0.024s
I20260709 13:19:01.165807  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:01.201560 12491 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.011s	sys 0.025s
I20260709 13:19:01.201788  6595 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:01.203857  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1643) took 1020 ms. Trace:
I20260709 13:19:01.204357  7229 rpcz_store.cc:276] 0709 13:19:00.183338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.183518 (+   180us) service_pool.cc:224] Handling call
0709 13:19:01.203587 (+1020069us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54004e16d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:01.203785 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326493568}
I20260709 13:19:01.028658 12271 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.136878  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.206348  7235 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:01.160975  6591 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:01.185904 12444 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:01.187121 12361 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:01.187908 12412 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:01.154700 12084 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:01.027796 12476 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:01.241297  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 337878us
I20260709 13:19:01.247900  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:01.248898  6518 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.242838  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:01.158918 12408 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.273707  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:01.285655  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:01.286404  6515 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.290302  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:01.291093  6517 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:01.292981  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:01.299921  6516 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
W20260709 13:19:01.226387 12464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.006s	sys 0.015s
I20260709 13:19:01.300479  6516 pending_rounds.cc:85] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 11
I20260709 13:19:01.233664 12084 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:01.300843  6516 pending_rounds.cc:107] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:19:01.307832  6516 raft_consensus.cc:2889] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:01.264200 11910 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.267943 12449 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:01.304906 12464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.007s	sys 0.018s
I20260709 13:19:01.266284  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:01.338425  6517 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:01.324533  6515 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:01.327872  6518 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
W20260709 13:19:01.349560  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 195407us
W20260709 13:19:01.382845  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 116632us
I20260709 13:19:01.383790  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:01.342201  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:01.412297  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 70170.1us
I20260709 13:19:01.412770  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:01.413656  6522 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:01.263135 12361 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:19:01.413417  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:01.319257 12340 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:19:01.423009  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 308328us
W20260709 13:19:01.426976  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 275575us
I20260709 13:19:01.427856  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:01.428200  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:01.458552  7228 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:01.418279 12464 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:01.301252 12345 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.566521  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.423099 12496 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:01.338089 11910 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.341902 12449 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.567407  6671 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:01.568127  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:01.414593 12444 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:19:01.429924 11940 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:01.558501 12322 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:01.565905 12458 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.567991 12464 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:01.636749 12522 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.450s	user 0.001s	sys 0.000s
I20260709 13:19:01.431092 12084 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.571085 12412 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:01.645403 12464 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:01.638319 12322 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.661656 12322 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:01.423770 12340 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.417425 12481 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:01.663043 11669 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:01.653069 12206 consensus_meta.cc:287] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.614s	user 0.011s	sys 0.000s
I20260709 13:19:01.679414  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.706931  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.707557  6664 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 9.
I20260709 13:19:01.656456 12476 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:01.640318 12458 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:01.718653 12525 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 13:19:01.719859 12524 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:19:01.679255 12340 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:01.735260 12529 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:19:01.741688 12340 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:01.743304 12536 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:19:01.745316  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:01.746320  6671 raft_consensus.cc:2368] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15.
I20260709 13:19:01.697642 12084 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:01.679414 12496 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:01.703711 12206 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
W20260709 13:19:01.773149  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.028s	sys 0.075s
I20260709 13:19:01.657344 12412 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:01.773842  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.028s	sys 0.076s
W20260709 13:19:01.785004  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 879046us
I20260709 13:19:01.786307  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.786913  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.716693 12481 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:01.704566 12523 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:19:01.754303 12534 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:01.806805  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.792230  6593 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:01.809568  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:01.785475 12408 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:01.821149  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:01.825871  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.829914  6597 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:01.826807  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:01.820598  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:01.821552  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.827163  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:01.788450 12538 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:01.832346  6596 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:01.849087  6588 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:01.788095 12322 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:01.852293  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:01.852010 12481 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.852820  6596 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:01.855471 12322 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.857533 12322 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:01.862818  6596 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:01.867057 12527 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
I20260709 13:19:01.869556 12496 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.789660 12238 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:01.871008 12238 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.874188 12238 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:01.874576 12458 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:01.875205 11910 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:01.828914 12534 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:01.876616 12412 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:01.877205 12412 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:01.876860 12534 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.794531 12340 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:01.811976 12490 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.061s	sys 0.068s
W20260709 13:19:01.879067 12490 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.062s	sys 0.068s
I20260709 13:19:01.851445  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.879447  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:01.880141  6595 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:01.888794  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.885710 12491 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:01.889470  6671 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:01.892000  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:01.892030 12491 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:01.887585 11910 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.897541  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.898010  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:01.899986  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.894522 12538 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:01.893334 12530 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:19:01.901748 11910 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:01.904048 12412 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:01.910060  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 481915us
I20260709 13:19:01.919672  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:01.910534  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1485) took 1188 ms. Trace:
I20260709 13:19:01.920733  7235 rpcz_store.cc:276] 0709 13:19:00.722331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.723595 (+  1264us) service_pool.cc:224] Handling call
0709 13:19:01.206110 (+482515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.208970 (+  2860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.910280 (+701310us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54004e16d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:01.910462 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109033344}
W20260709 13:19:01.902930  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 468041us
I20260709 13:19:01.906587 12449 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:01.919196 12412 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:01.909559 12245 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.010s	sys 0.020s
W20260709 13:19:01.959275 12245 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.010s	sys 0.021s
I20260709 13:19:01.927695 12538 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:01.976388 12490 raft_consensus.cc:993] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:19:01.977105 12490 raft_consensus.cc:1081] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:19:01.978992 12473 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:01.926178 12491 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.981653 12491 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:01.985740 12503 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:01.970142 12538 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:01.994243 12538 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:01.994566 12498 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.634s	user 0.015s	sys 0.000s
W20260709 13:19:02.021981  6513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.004s	sys 0.024s
W20260709 13:19:02.023172  6513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.005s	sys 0.024s
W20260709 13:19:02.040249  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 120562us
I20260709 13:19:02.042008  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:02.067833 11910 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.048924  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:01.943233 12412 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:02.088624  7236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1468) took 2770 ms (client timeout 1947 ms). Trace:
I20260709 13:19:02.088372 12523 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:19:02.089370  7236 rpcz_store.cc:269] 0709 13:18:59.317665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.318543 (+   878us) service_pool.cc:224] Handling call
0709 13:19:02.087984 (+2769441us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400571554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:02.088304 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3384499456}
I20260709 13:19:02.091208 12530 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:02.091538  6663 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:02.092070 12530 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:02.093933 12469 raft_consensus.cc:2764] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:19:02.093973 12530 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.095592 12492 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.096575 12492 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.112012 12449 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.113125 12449 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.125228 12458 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.126147 12458 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.127030 12531 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.127543  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2802) took 2122 ms. Trace:
I20260709 13:19:02.127955 12531 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.128728 12458 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.131067 12531 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.154249 12449 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.155957  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:02.157401 12492 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.158175 12473 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.158535  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.160486  6522 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:19:02.159258  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.162061  6518 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:02.152750  6563 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:02.129437 12458 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.179613  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 70253.5us
I20260709 13:19:02.184062 12326 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:02.185377  6662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 2812) took 1433 ms (client timeout 1816 ms). Trace:
I20260709 13:19:02.185253 12473 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.186523  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2426) took 1299 ms. Trace:
I20260709 13:19:02.128038  6669 rpcz_store.cc:276] 0709 13:19:00.005009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.005191 (+   182us) service_pool.cc:224] Handling call
0709 13:19:00.005639 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.006087 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.006128 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.006702 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.232523 (+225821us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:00.232616 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.232862 (+   246us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.994042 (+1761180us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:02.126995 (+132953us) raft_consensus.cc:1685] finished
0709 13:19:02.127035 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.127365 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6520,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":35690,"spinlock_wait_cycles":5848448,"thread_start_us":160763,"threads_started":1,"wal-append.queue_time_us":265884}
I20260709 13:19:02.203843  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:02.208256  6662 rpcz_store.cc:269] 0709 13:19:00.751963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.752207 (+   244us) service_pool.cc:224] Handling call
0709 13:19:02.184927 (+1432720us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000e51d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:02.185159 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271745792}
I20260709 13:19:02.209586  6668 rpcz_store.cc:276] 0709 13:19:00.886539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.892287 (+  5748us) service_pool.cc:224] Handling call
0709 13:19:02.186136 (+1293849us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54001dced4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:02.186332 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409002240}
W20260709 13:19:02.241273  7198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.011s	sys 0.035s
W20260709 13:19:02.265412  7198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.012s	sys 0.035s
I20260709 13:19:02.238020 11910 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:02.232437 12523 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:02.243014 12458 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.263356 12449 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:02.285334  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46296.1us
W20260709 13:19:02.281315 12477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.747s	user 0.036s	sys 0.045s
W20260709 13:19:02.298133  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.293988  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 189974us
W20260709 13:19:02.309442  7197 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.019s	sys 0.029s
W20260709 13:19:02.297293 12477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.804s	user 0.036s	sys 0.046s
W20260709 13:19:02.312057  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 159460us
I20260709 13:19:02.314699  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:02.312245  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.310577  7197 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.020s	sys 0.029s
W20260709 13:19:02.325589  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 83070.5us
W20260709 13:19:02.314832 11940 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.002s	sys 0.044s
I20260709 13:19:02.319288  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:02.371183  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1143) took 1071 ms. Trace:
I20260709 13:19:02.371830  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1144) took 1056 ms. Trace:
I20260709 13:19:02.375010  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.303217 12361 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:02.323661 12271 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.404287 12449 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.409698 12513 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:02.410502 12513 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.413306 12513 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.391568  6516 rpcz_store.cc:276] 0709 13:19:01.315417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.315559 (+   142us) service_pool.cc:224] Handling call
0709 13:19:01.342894 (+ 27335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.343153 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:01.896050 (+552897us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.366489 (+470439us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.371169 (+  4680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.371250 (+    81us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:02.371313 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.371399 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.371631 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1199,"prepare.queue_time_us":573540,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":29656,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":332,"thread_start_us":906159,"threads_started":3}
I20260709 13:19:02.391212  6520 rpcz_store.cc:276] 0709 13:19:01.302124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.302299 (+   175us) service_pool.cc:224] Handling call
0709 13:19:01.302716 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.303007 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:01.785161 (+482154us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.351220 (+566059us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.358245 (+  7025us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:02.363141 (+  4896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.363271 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.370941 (+  7670us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":595229,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1363,"raft.queue_time_us":31353,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":496,"thread_start_us":1005175,"threads_started":3}
I20260709 13:19:02.428897 12473 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.431699 12549 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:02.432533 12549 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.400063 12523 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.434963 11826 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.435756 12523 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:02.400364 11940 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.005s	sys 0.046s
I20260709 13:19:02.436398 12531 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:02.436875 11940 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:02.438656 11940 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.439234 12458 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:02.431720  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 135619us
I20260709 13:19:02.326867 11910 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.451100 12449 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.463516  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:02.469188 12473 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:02.479137 12473 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.481856 12473 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.482753  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:02.484668  7198 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.662s (SENT) [suppressed 32 similar messages]
I20260709 13:19:02.485543  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:02.486297  6669 raft_consensus.cc:2376] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16. Re-sending same reply.
W20260709 13:19:02.489252 12550 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 13:19:02.496062 12473 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:02.497126 12449 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:02.499189  6486 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:02.500500 12449 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:02.501274 12449 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:02.501507  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:02.435750 11826 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.502800 12449 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.505594 12549 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.498418 12473 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:19:02.509117 11910 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.505633 11826 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.470048 12530 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.517972 12530 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.524988 12245 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:02.525871 12245 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.526644  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:02.529011 12245 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.531810  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.532799  6513 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:02.534567  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.536639  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:02.537380  6513 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:02.544730 12271 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:02.545540 12493 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:02.545513 12271 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.546311 12493 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.548650 12493 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.549047 12271 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.549227  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:02.550197  6522 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:02.550635  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.551728  6515 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:19:02.553802  6517 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:19:02.563274  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.565161  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:02.566367  6591 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:02.568023  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.570046  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.572444  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.570853  6591 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:02.573236  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.574297  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.575569  6594 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:02.576925 12493 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:02.577775 12245 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:02.578547 12245 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.577608 12493 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.580014 12477 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.580829 12477 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.581703 12493 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.584304 12477 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:02.585834  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 84498.7us
I20260709 13:19:02.586937 12473 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:02.587550 12473 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:02.588536  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:02.588897 12473 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.590284  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.591192  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:02.613291  6517 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:02.591246  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.585947 12568 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:02.581372 12245 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.595387 12469 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:02.595602  6662 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:02.593122  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:02.594739 12412 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:02.598143 12464 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.926s	user 0.012s	sys 0.010s
I20260709 13:19:02.602579  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:02.603080  6668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45444
I20260709 13:19:02.627915  6668 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:02.628378  6668 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:02.604761  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.606348  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:02.606853  6671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45444
I20260709 13:19:02.609028  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:19:02.575683  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 174096us
I20260709 13:19:02.629004  6518 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:02.612654  6515 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 10.
I20260709 13:19:02.635095  6514 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:02.615912 12568 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.636341  6671 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:19:02.643198  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54841.2us
I20260709 13:19:02.635391  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.639276  6671 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:02.648782 11573 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:02.649238 11573 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.649556 11573 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:02.650210  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.651170  6667 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:02.646538  6663 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:02.646947  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.641866 12568 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:02.670302  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 185640us
I20260709 13:19:02.671803  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.637043 12464 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.673594  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:02.679548 12458 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:02.679664 12571 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.680141 12458 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.680719 12458 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:02.680543 12571 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.683262 12473 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:02.683142 12531 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:02.684298 12531 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:02.684626 12531 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:02.685588 12531 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.687377 12530 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.689095 12245 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:02.688983 12530 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.689495 12245 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.689582 12412 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:02.689895 12245 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:02.690891  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 82023.5us
I20260709 13:19:02.691854  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:02.692168  6594 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:02.693578 12271 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:02.694921  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:02.683168  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:02.698696 12271 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:02.699481  6515 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:19:02.701774 12490 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:02.703975  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:02.706148  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:02.714793  6514 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:02.704783 12493 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:02.699788  6563 leader_election.cc:400] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
I20260709 13:19:02.704797  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.718521  6517 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:02.716518 11573 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.719727 12549 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.726810 12549 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.728127  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:02.729151 12549 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:02.729607  6563 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:02.729493  6592 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:02.730852 12523 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:02.731206  6518 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:02.731729 12523 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:02.733379 12571 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.723702 12326 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:02.734958 12326 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:02.736172 11573 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.735939 12326 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.739548 12531 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:02.720122 11940 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:02.730906 11826 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:02.747026 11826 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.748373  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:02.749176  7237 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:02.749291 11826 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:02.753463  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:02.754195  7236 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:02.757352  6671 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.716846 12490 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:02.770850  6662 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:02.716692 12493 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.772363  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.746569 11940 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.777920 12449 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.778478 12449 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.780664 12449 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.781243 11940 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:02.783450  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.760586 11826 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.786983 11826 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.789654  6518 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
W20260709 13:19:02.798669  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63939.5us
I20260709 13:19:02.806747  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:02.770004 12549 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.815662  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.777217 12493 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:02.774825 12490 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:02.824231  6668 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.833006  6671 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.799256 12530 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.837401  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.839145  6669 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:02.840662  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:02.837440 12412 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:02.812795 12571 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:02.842036  6665 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:02.837634  6667 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:02.843850  6668 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.770185 12271 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:02.846812  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:02.846563 12412 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:02.850747  6488 leader_election.cc:336] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.505s (SENT)
I20260709 13:19:02.854475  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:02.853461  7231 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:02.841852 12477 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:02.847280  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:02.844451 12469 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:02.859071 12576 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:19:02.830688 12549 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.860114  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.864192  6513 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:02.861267 11573 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:02.857770 12477 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.877548  7235 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:02.863963 12530 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:02.852959 12458 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:02.885686  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.886464  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:02.894742  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:02.884166  7233 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:02.903056  7236 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:02.900408  7234 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:19:02.908596  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 233267us
I20260709 13:19:02.879249 12477 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:02.865643 12571 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.917042  6592 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:02.920495  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.921275  6671 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
W20260709 13:19:02.910254 12572 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:19:02.927663  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:02.928385  7229 raft_consensus.cc:2410] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:19:02.929919  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 110412us
I20260709 13:19:02.879602 12568 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.932870  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.933406  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:02.934980  7236 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:19:02.914166 12464 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.916090 11826 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:02.924425  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.934396  6589 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:02.945504  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:02.947341  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:02.948841 12531 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:02.948896 12568 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.949401 12531 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:02.949888 12531 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:02.951442  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.952092  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.953433  6596 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:02.953790  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:02.954435  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.953457  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.955102  6595 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:02.955196  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.955788  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.955917  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.956557  6595 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:02.957257  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.956391  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:02.956350  6591 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:19:02.962523 12531 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:02.960690  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:02.965202 12531 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.966346 12464 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:02.966845 12464 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:02.967222 12464 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:02.972847  6596 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:19:02.978518  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 45893.4us
I20260709 13:19:02.988346 12571 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:02.989603 12412 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:02.991120 12582 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:19:03.000691  6670 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:19:03.007155  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 74983.1us
I20260709 13:19:03.011332  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:03.014873  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.020795  6589 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:03.020982  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:03.021622  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:02.970116 12493 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.018644  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:03.024310  7235 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:03.024999  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:03.027460  6592 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:03.029271  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:03.033871  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.038695  6663 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:03.044236  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:03.044516  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:03.044953  7235 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:03.045004  6666 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:03.046986  6518 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:03.054335  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.055454 12568 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:03.056031 12568 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.056458 12568 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.061081 12245 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.062271  6668 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:19:03.063303 12458 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.065711  7234 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:03.066919  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.067128  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.067365  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.067940 11826 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.068034  7232 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:03.068426 11826 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.068519  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.068876 11826 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.069172  7230 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:03.070050  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.071067 12549 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:03.073094 12245 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:03.031622 12571 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.078810 12571 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.079258 12571 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.070986  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.068053  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.076227  6663 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:03.070999  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.062297  6662 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:02.977523 12464 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.045172  6519 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:03.082479  7233 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
W20260709 13:19:03.096992  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 115547us
I20260709 13:19:03.097728  7234 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:03.115255  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:03.115675  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:03.116372  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 72643.3us
I20260709 13:19:03.116317  7230 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:03.118836  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.122794  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.123402  7233 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:03.125464  6662 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.113183 12412 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:03.130084  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.127596  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.130331  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.123895  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.131382  7234 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:03.132167  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:03.119304 12589 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:03.120782 12449 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.135726 11573 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:03.135106  7229 raft_consensus.cc:2376] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14. Re-sending same reply.
I20260709 13:19:03.127722 12490 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.141579  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.142798 12490 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.143299  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.144377  7234 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:03.150781 12449 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.153067 12449 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.155773 12531 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:03.156216 12589 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.156828 12477 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.157286 12531 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:03.143591  6596 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:03.159772 11573 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:03.160528  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.161091  7230 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:03.133659 12412 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:03.164706  6667 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 13:19:03.166457 12585 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.147s	user 0.001s	sys 0.003s
I20260709 13:19:03.166554  7234 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:03.168066  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.168812  7234 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:03.169838  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.170367  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.171950 12271 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:03.172638 12271 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.179065 12589 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.179741  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:03.179898  6662 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:03.183852  7232 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:03.140501  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.182376  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.196231  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.196866  7229 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:03.199973  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 74477.5us
I20260709 13:19:03.158775 11910 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:03.201822  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.202250  7234 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:03.208005  7232 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:03.165444 12412 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:03.208901 11573 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.214371  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.215212  7232 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:03.180698 12271 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.195596 12361 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:03.202015 12531 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:03.221323  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 89277.5us
I20260709 13:19:03.204504 11910 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:03.213133 12589 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.223089 11910 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.223788 12589 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.224231 12589 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.224460 11910 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:03.234495  6666 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
W20260709 13:19:03.237056  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 112924us
I20260709 13:19:03.237540  6596 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:03.239598  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.240010  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.240465 12271 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.240797  7235 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:03.241110 12271 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.244448 12271 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.251174  6668 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:03.251382  6418 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 9 to 10, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 10 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.253615  6418 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.257133 12493 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:03.262847  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 59122.4us
I20260709 13:19:03.265503  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.220124 12477 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:03.271574  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.274269  7231 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:03.238336 12531 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:03.284720  7233 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:03.265630 12473 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:03.284106  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.288921  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:03.291706  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.299019  6563 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:03.271020 12449 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:19:03.302760  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:03.308055 12477 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:03.308564 12477 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:03.308961 12477 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.309413 12531 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.310904 12523 raft_consensus.cc:2764] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:03.311712 12523 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:03.316726  7229 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:03.271950 12458 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:19:03.324019  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 54732.7us
I20260709 13:19:03.325253  6418 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 8 to 12, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 12 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:03.336328  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46731us
I20260709 13:19:03.276607 12571 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.338312  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.323650 12464 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.341670  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.285549 12412 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.345515 12551 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:03.362481  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:19:03.362085  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 79894us
I20260709 13:19:03.313305 12568 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.340250 12530 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:03.366323  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 63683.7us
I20260709 13:19:03.383526  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.340706 12569 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:03.391239  7230 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:03.384495  6418 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 3 to 7, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 7 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.392663  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.393582  7233 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.364140 12551 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.395187  7237 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:03.379482 12464 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:03.398247  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.415148  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:03.416674 12569 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.419073 12569 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.344775 11940 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:19:03.420169 12569 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.402074 12473 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:03.420675 12569 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.445179  7233 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:03.440272  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.436761  6418 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 7 to 8, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.402266 12551 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:03.445899  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.453338  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.414837 12464 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.455528  7237 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:03.391603 12530 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.395154 11826 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.472239  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:03.472887  6594 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:03.467461  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.437666 12493 raft_consensus.cc:2764] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:19:03.473173  6594 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:03.452270 11940 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:03.449349 12523 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:03.470760  6418 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.474224  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:03.478288  6588 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:03.478138  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 78501.5us
I20260709 13:19:03.478715  6588 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:03.468155 12464 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:03.484933  6419 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 333) took 27607 ms (client timeout 15000 ms). Trace:
W20260709 13:19:03.485319  6419 rpcz_store.cc:269] 0709 13:18:35.877630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.886505 (+  8875us) service_pool.cc:224] Handling call
0709 13:19:03.483610 (+27597105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:59.623215 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:59.789673 (+166458us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:59.792275 (+  2602us) write_op.cc:435] Acquired schema lock
0709 13:18:59.796200 (+  3925us) tablet.cc:662] Decoding operations
0709 13:18:59.936745 (+140545us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:59.936796 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:59.936821 (+    25us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:18:59.941538 (+  4717us) tablet.cc:701] Row locks acquired
0709 13:18:59.941557 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:18:59.941721 (+   164us) write_op.cc:270] Start()
0709 13:18:59.941834 (+   113us) write_op.cc:276] Timestamp: P: 1783603139941677 usec, L: 0
0709 13:18:59.941857 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:18:59.942466 (+   609us) log.cc:844] Serialized 6326 byte log entry
0709 13:19:00.371655 (+429189us) op_driver.cc:464] REPLICATION: finished
0709 13:19:00.711113 (+339458us) write_op.cc:301] APPLY: starting
0709 13:19:00.712128 (+  1015us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:00.813753 (+101625us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:00.813866 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:02.414435 (+1600569us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:02.426117 (+ 11682us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:19:02.426216 (+    99us) write_op.cc:312] APPLY: finished
0709 13:19:02.427008 (+   792us) log.cc:844] Serialized 148 byte log entry
0709 13:19:02.644014 (+217006us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:02.738486 (+ 94472us) write_op.cc:454] Released schema lock
0709 13:19:02.738587 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:02.739279 (+   692us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:02.739325 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5595653,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":331424,"apply.run_cpu_time_us":128397,"apply.run_wall_time_us":2033620,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70975,"num_ops":16,"prepare.queue_time_us":307022,"prepare.run_cpu_time_us":10228,"prepare.run_wall_time_us":402522,"raft.queue_time_us":308208,"raft.run_cpu_time_us":20231,"raft.run_wall_time_us":332082,"replication_time_us":409731,"spinlock_wait_cycles":1404672,"thread_start_us":853150,"threads_started":5,"wal-append.queue_time_us":284171,"wal-append.run_cpu_time_us":28469,"wal-append.run_wall_time_us":1247281}]]}
I20260709 13:19:03.491032  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.492229 12551 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.496351 12551 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:03.497617 12600 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:19:03.499985 12604 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:03.500430 12604 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:03.501405 12604 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.502965 12473 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.507522 12568 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:03.508018  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:03.508682  7228 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:03.515539  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:03.483225  6594 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.516465  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:03.522552  7233 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:03.523465 12595 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:19:03.480026 11940 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:03.481195 12271 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:03.480188 12523 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:03.522603  6418 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 4 to 8, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.536507 11940 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:03.538667 12596 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:19:03.540831  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56935.4us
I20260709 13:19:03.540395  6418 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 5 to 9, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 9 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.542114  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:03.542158  6418 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 10 to 11, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 11 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.483286 12245 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:03.544123  6418 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 10 to 11, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 11 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.550047 12361 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:03.550259  6418 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:03.527318 12469 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:03.548339 12490 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:19:03.554494 12469 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:03.545333 12523 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:03.556947 12569 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.559103 12589 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.562028 12589 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:03.534806 12604 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:03.563253  6594 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.546689  6588 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:03.546720  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 205110us
W20260709 13:19:03.576602  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.578653  7237 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:03.563246 12412 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:03.581877  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.581900  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62275.2us
W20260709 13:19:03.583621  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 75030.1us
I20260709 13:19:03.554217 12571 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:03.575466 12271 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:03.601889  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:03.604916  6663 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.602111  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:03.599221  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.605813  6668 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:03.617376  6663 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:03.618388 12271 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:03.633126 12606 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.122s	user 0.002s	sys 0.003s
I20260709 13:19:03.639482  6588 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:03.640136  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:03.617560 11940 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.585516 12245 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:03.646796 12245 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:03.648993  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.649276  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.651419 11826 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:03.652829  6588 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:03.653482 12361 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:03.653615 12326 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.659431 11940 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:03.575882 12589 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:19:03.660991 12477 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.672339  7228 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:03.677351  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 152960us
I20260709 13:19:03.677789  6594 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:03.681464  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:03.691327  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:03.699383 12619 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.699720  6663 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:03.699890 12619 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:03.689257 12326 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:03.700263 12619 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.701771 12326 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:03.689536 12361 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.703418  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:03.704444  6668 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:03.678330 12569 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:03.708370  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:03.709184  6668 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:03.717120  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:03.717483  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:03.718101 12603 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:03.717504  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:03.719487  6592 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:03.718132  6588 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:03.720119  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:03.720955  6518 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:19:03.705935 11826 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:03.724628  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:03.724838  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:03.725592  6518 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:03.727136  6513 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:03.733989  6519 raft_consensus.cc:2410] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:19:03.736222  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:03.739559  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 195487us
I20260709 13:19:03.740109  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:03.744221  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:03.744449  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:03.744997  6597 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.744215  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:03.744338  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:03.746263  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:03.746579  6591 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:03.752966  6517 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:03.755074  6666 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:03.721868 12569 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:03.702502 12361 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:03.760411  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.759964  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:03.764964  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 161019us
I20260709 13:19:03.710500 12523 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:03.787946  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:03.789708 12485 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:03.794538  6588 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:03.759418 11826 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:03.795815  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 196667us
I20260709 13:19:03.805552  6589 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:03.790352 12485 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:04.044648 12617 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.294s	user 0.000s	sys 0.000s
I20260709 13:19:03.855911  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:04.172731  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 384869us
I20260709 13:19:04.172924  6517 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 12.
I20260709 13:19:04.173570  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:03.856325 12614 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:04.174271 12551 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:04.056250  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.043053 12569 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:03.956431  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 132880us
W20260709 13:19:04.175969  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.177057  6666 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:04.023358 12612 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.188046 12612 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.163933  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:04.180502 12485 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:03.965545 12464 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.018805 12622 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.357s
W20260709 13:19:04.212745  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 353900us
I20260709 13:19:04.048004 12477 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:04.023197 11826 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.057790 12361 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:04.134399  6592 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:04.174479  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:04.171254  6597 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
W20260709 13:19:04.201323  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 452445us
I20260709 13:19:04.214385  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:19:03.857378 12576 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:19:04.231488  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:04.232162  6514 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:19:04.232394  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 58024.5us
I20260709 13:19:04.235787  6513 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:04.237165  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:04.217644 12612 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:04.242198  6593 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:04.242807  6636 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.947s (SENT)
I20260709 13:19:04.243813  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:04.248464  6514 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:04.233660 12551 raft_consensus.cc:2764] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:19:04.250985  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 504761us
I20260709 13:19:04.252660  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:04.254073  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.255087  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.255718  6592 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 10.
I20260709 13:19:04.257613  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:03.857199 12271 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.382s
I20260709 13:19:04.264396  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.228602 12326 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:19:04.269483  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:04.270082  6588 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.271121  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.273223  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:04.267196 12619 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.000s
I20260709 13:19:04.275993  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:04.279243  6590 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:19:04.279320  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65019.1us
I20260709 13:19:04.281642  6597 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:04.271793 12551 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.283946  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:04.292493  6588 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:04.276513 12477 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:04.277169 12245 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:04.297645  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58811.7us
I20260709 13:19:04.303565 12636 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:04.303678  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.304057 12636 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.304569 12636 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:04.304584  6514 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:04.309742  7198 leader_election.cc:400] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15.
I20260709 13:19:04.310148  7198 leader_election.cc:403] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:04.302790  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:19:04.322070  6636 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 2.023s (SENT)
W20260709 13:19:04.323653  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47755us
I20260709 13:19:04.300393 12549 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:04.326092  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:04.301060 12477 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:04.283461 12619 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.298609 12589 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:19:04.336273 12641 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:04.298365 12551 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:04.338865  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 142495us
I20260709 13:19:04.339447 12624 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15.
I20260709 13:19:04.339907  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.361824  6521 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:04.320793 12629 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:04.326630 12523 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:04.350595 12464 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:04.351327  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.351610  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.375866  6518 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.352670 12551 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:04.330461 12477 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.355234 12490 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:19:04.359835  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.368505  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 43514.4us
I20260709 13:19:04.370435  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:04.332106 12530 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:04.387645  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:04.387674  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45386.8us
I20260709 13:19:04.392081  6514 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
I20260709 13:19:04.392442  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:04.361124 12590 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:04.395639  6671 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:04.363417 12271 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:04.415827  6518 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:04.420622  6667 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:19:04.421092  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 79913.6us
I20260709 13:19:04.423322  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.420182  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:04.396040  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:04.395257 12568 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.433754  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:04.440165  6671 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:04.439875  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:04.446487 12624 raft_consensus.cc:2764] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:19:04.446673 12642 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:04.447307 12642 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:04.448169 12642 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.460584  6588 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:04.462806  6521 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:04.460337 12477 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:04.470206 12477 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:04.444434 12612 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:19:04.441227 12568 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.484581 12568 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:04.404445 12604 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 13:19:04.488951  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49139.9us
I20260709 13:19:04.489924  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:04.491029  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:04.491899 11573 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:19:04.497587 12589 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:04.498091  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.498705  6593 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:04.498806 12589 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:04.494997  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:04.441634 11826 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:04.494236 12590 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:04.505261  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:04.526477  6597 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.526647  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:04.505586  6593 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:04.435514 12641 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:04.511487  6592 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:04.519752 12618 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:04.522146  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.523763 12640 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:04.525586 12620 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:04.434950 12271 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:19:04.537086 12641 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.560009  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.561172  6590 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:04.555001 12640 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:04.562306 12589 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:04.505800 12245 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:19:04.563544 12640 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:04.565080  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.568529  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 73603.4us
I20260709 13:19:04.539309 12473 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.539179 12618 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:04.552800 12620 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:04.541764  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.538872 12590 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:04.544750 12603 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:04.551143 12647 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:04.555402  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 164716us
I20260709 13:19:04.589946  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.590960  6522 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:04.541805  6597 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:04.587548  6516 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:04.593894  6593 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:04.594784  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.594132  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:04.601230  6667 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:04.604552  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.606088  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:04.607175  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:04.607749  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.608372  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:04.573793 12473 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:04.609791  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:04.611037  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:04.569509 12477 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:19:04.606357  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:04.589552 12647 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.617476  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.558840 12636 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:04.587512 11826 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:19:04.628089  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:04.628366  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.629666  6517 raft_consensus.cc:2368] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11.
W20260709 13:19:04.629652  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 77415.3us
W20260709 13:19:04.631350  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.633478  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.634387  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.637557  6593 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:04.637148  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.635413  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:04.636436  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:04.640367  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.640889  7230 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.641484  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.641480  7234 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:04.642089  7229 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:04.641531  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.644038  6517 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:04.619624 12647 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:04.647250  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.650992  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.648577  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.637236  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:04.651866  7232 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:04.651932  7228 raft_consensus.cc:2376] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15. Re-sending same reply.
I20260709 13:19:04.646701  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:04.653090  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.653952  7232 raft_consensus.cc:2368] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:19:04.631369 12485 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:19:04.657603  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.658828  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.659857  7233 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:04.660498  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.661010  7228 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:04.660650  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.661451  7235 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:04.662606  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.663642  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:04.634430 11940 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:04.665680 11940 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:04.666136 11940 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:04.659893  6593 raft_consensus.cc:2368] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:04.669598  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.670393  7231 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:04.672564  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.678884  6592 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:04.678097  6517 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:04.656924 12624 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:04.685359  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 145413us
I20260709 13:19:04.685827  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.687299  7235 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:04.686671  6593 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:04.678108  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.693464  7232 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:04.693437  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:04.695500  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.637607 12603 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:19:04.646109 12551 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:19:04.716355  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.610663 12620 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.719048  6517 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:04.705051  7230 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:04.706871  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:04.676811 12650 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:04.665109 12245 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:19:04.725383  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.726024  6516 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:04.707075  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 60484.9us
I20260709 13:19:04.709705  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.727259  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.711913  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.688599 12473 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:04.727767  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.717432  6513 raft_consensus.cc:2368] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:19:04.688726 12655 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:04.720954  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:04.721571  7237 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:04.720516  6522 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:04.710494  7232 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:04.769433  7236 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:04.723762  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:04.769974  7235 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:04.777293  6515 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:04.705716  7233 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:04.771140  6521 raft_consensus.cc:2368] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17.
I20260709 13:19:04.683344 12464 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:19:04.727934 11940 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.706959 12624 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:04.649681 12469 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:19:04.778595  6522 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:04.801481  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:04.774171 12654 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:19:04.778959  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.696271 12642 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:19:04.722201 12650 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.699175 12628 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:19:04.787766 12485 raft_consensus.cc:2764] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:19:04.803620  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79975.8us
I20260709 13:19:04.807226  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:04.814554 11940 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:04.814314  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:04.682471 12590 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:04.779423  7228 raft_consensus.cc:2376] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16. Re-sending same reply.
I20260709 13:19:04.828179  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:04.832902  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.827417  6521 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:04.836197 11940 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:04.837014 11940 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:04.837399 11940 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:04.836304  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:04.820784 12620 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:04.819710 12619 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:04.843371  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:04.845556  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.816833 12650 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:04.844861  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:04.848222  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:04.835580 12523 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:04.849057  6592 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:04.841168 12603 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:04.854395  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.864181 12647 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:04.865494  7231 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:04.840684 12636 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:04.867296  7231 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:04.868520  6593 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:04.869534  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:04.870180  6522 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:04.882120  7231 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.885434  7231 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:04.866919 12603 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:04.886674 12603 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:04.887142 12653 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:19:04.844430 11826 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:19:04.893504  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:04.895022  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.854861 12361 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.900966 11940 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:04.909011  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 188146us
I20260709 13:19:04.866230 12523 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:04.916422  7231 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:04.881794 12614 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:04.882720 12412 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:04.920315  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:04.921097 11940 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:04.921190  6592 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:04.896165 12473 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:04.923569  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:04.928251 11573 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:04.928516  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:04.928767 11573 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:04.929464 11573 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:04.947567  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:04.950400  6516 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
W20260709 13:19:04.967408  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.969179 12523 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.919566 12449 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
W20260709 13:19:04.973824  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 146142us
I20260709 13:19:04.974128  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:04.971822 11573 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.927361 12473 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:04.981997 11573 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:04.982872  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:04.946188 12568 raft_consensus.cc:2764] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:19:04.983299  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:04.948236 12612 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.985021  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:04.918108 12361 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:04.957952 12549 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.995819  6592 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:05.003217  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:04.975500 12603 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.004992 12666 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:05.005523 12657 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:19:04.882081 12624 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.006829 12603 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:04.976584 12412 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:05.005103 12661 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:05.004505 12665 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:05.013314  6593 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:04.991801 12473 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:05.013288  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 90840.8us
I20260709 13:19:05.003723  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:05.029812  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:05.022531  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.022336  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:05.038348 12614 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:04.995417 12612 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:05.023949 12469 consensus_queue.cc:1243] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:05.039603 12671 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:05.011670 12619 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:05.002125 12549 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.044486 12589 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:05.044822 12412 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.045723 12674 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:05.047230 12676 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:05.021267 12618 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.052425  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.053159  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.054965  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.096370  6589 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:05.055217  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.054883  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:05.054840 12652 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:19:05.097137  6591 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:05.004997 12663 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.215s
I20260709 13:19:05.104454 12663 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:05.105547 12663 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:05.097227  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 74973.4us
W20260709 13:19:05.096621  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 66901.3us
I20260709 13:19:05.092559 12589 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:05.097945  6592 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:05.111449  6636 leader_election.cc:400] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:19:05.111838  6636 leader_election.cc:403] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:05.113601 12675 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:05.114143 12618 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:05.119140 12589 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:05.119998 12612 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.043618 12619 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:05.121871  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:05.106065 12663 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.056056 12660 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:05.094945 11826 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.095650 12674 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:05.094660 12671 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:05.136961 12668 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:05.128686 12641 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.130463 12619 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:05.150216  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:05.151456  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 191958us
I20260709 13:19:05.158968  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:05.157485  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:05.159024 12665 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:05.160881 12685 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:05.165195 12685 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:05.138609 12660 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.166646 12685 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.161283 12642 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:05.137529 11826 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.140621 12674 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:05.170181 12659 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.333s	user 0.000s	sys 0.002s
I20260709 13:19:05.199144  7233 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:05.168819 12660 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.123888 12614 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.202136 12666 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:05.155639 12647 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:05.130498 12650 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:05.207983 12680 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:19:05.208865 12650 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:05.205741  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46858.6us
I20260709 13:19:05.212112 12620 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:05.213668 12620 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:05.213752 12473 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.214737 12612 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:19:05.215974  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.218591 12666 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:05.219213 12666 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:05.205205  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:05.192377 12685 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:05.209612  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 440794us
I20260709 13:19:05.222772  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:05.225594  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:05.234894  6592 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:05.235394  6592 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:05.236439  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.256978  6516 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:05.260210  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:05.261075  6517 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:05.261927  6594 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:05.263947  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.264503  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.264995  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:05.265537  6665 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:05.203054 12688 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:05.267882  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.270310  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:05.272324  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:05.273126  6515 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:05.273919  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.233978 12612 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:05.288410  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.279670 12473 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.280531 12691 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:05.250111 12449 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:05.266193 12361 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.281632  6520 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:05.256811 12549 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.283717  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:05.283972 12681 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.249810 12568 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
W20260709 13:19:05.285917 12689 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:19:05.286332 12671 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.253562 12619 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.290448  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.293004  6592 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.301074  6563 leader_election.cc:400] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11.
I20260709 13:19:05.314158  6563 leader_election.cc:403] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:05.294582 12412 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:05.285468 12683 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:19:05.303687  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:05.303753  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.305233  6665 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:05.305603  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.317386  7233 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:05.276145 12688 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:05.317848  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:05.318502 12688 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:05.319190  6662 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:19:05.290562 12612 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:05.323148 12612 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:05.333264 12612 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.334767  6516 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:19:05.337313 12612 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.340946  6517 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:05.363473 12678 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:05.365494 12693 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:05.366041 12619 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.366653  6671 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:05.298740 12449 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:05.373122  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 82740.5us
W20260709 13:19:05.402822  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 101882us
I20260709 13:19:05.403560  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:05.405407  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.406267  6594 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:05.397181 12603 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:05.396528  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.402638  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.409755  6563 leader_election.cc:400] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17.
I20260709 13:19:05.411808  6563 leader_election.cc:403] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:19:05.370163 12688 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.309082 12681 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.423504 12703 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:05.423871  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.429953  6666 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:05.408828 12449 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:05.373706 12490 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:05.435824  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.436676  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.437575  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:05.417456 12636 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:05.440392  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:05.442502  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:05.443827  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:05.427461 12691 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 10. Current term is 11.
I20260709 13:19:05.445156 11940 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:05.445585 11940 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.445930 11940 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.448609  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:05.449256  7230 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:05.450429  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:05.450752  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:05.455940  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.456698  7236 raft_consensus.cc:2368] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:19:05.455540  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:05.441388 12636 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:05.461292 12703 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.449808 12412 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:05.462435  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.433835 12361 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:19:05.444922 12603 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:05.465756  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.469821  7230 raft_consensus.cc:2376] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15. Re-sending same reply.
I20260709 13:19:05.470506 11940 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.470986  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:05.471077  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.454224 12676 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:05.473003  7230 raft_consensus.cc:2368] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:19:05.474488  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.475495  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.480189 11940 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.475405  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:05.480538  6636 leader_election.cc:400] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:19:05.458328 12705 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17.
I20260709 13:19:05.485643  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:05.485947  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:05.491078  7234 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
W20260709 13:19:05.492587  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.492672  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.494336  6515 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:05.480365 12699 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:05.497975  6563 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:05.485267 12691 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:19:05.481669 12618 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:05.472111 12703 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.499100 12676 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:19:05.516919 12681 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.519228 12688 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.503964  6636 leader_election.cc:400] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:19:05.524286  6636 leader_election.cc:403] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:05.482694 12706 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:05.535686 12705 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.536537 12705 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.540588 12691 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:05.541358 12691 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:05.543951 12704 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:05.544523 12704 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:05.546784 12676 consensus_queue.cc:1243] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 19.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:05.545626 12704 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:05.549216 12704 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.551829 12705 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.553100 12705 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:05.553650 12705 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:05.556452 12705 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:05.558920 12691 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.499724 12619 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:05.565325  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 61426.6us
I20260709 13:19:05.580935  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.524844 12618 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:05.503096 12669 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:05.528158 12706 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:05.564885 12688 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.605561  6597 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:05.581583 12620 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.601859 12703 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.604460 12618 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:05.503113 12699 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.632449  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.614944 12646 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:05.503536 12636 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.634411  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:05.615362 12706 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:05.637151  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 139266us
I20260709 13:19:05.649127  6597 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:05.629935 12705 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:05.659462 12698 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:19:05.661993 12711 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:05.662883 12711 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:05.664247 12711 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:05.669836 12589 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.550s	user 0.006s	sys 0.005s
I20260709 13:19:05.674790 12636 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:05.679291  6515 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:05.681419  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:05.683423  7229 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:05.687426  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:05.688733  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:05.689824  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:05.619922 12688 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:05.701372 12493 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:05.701866 12493 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:05.707353  7233 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:19:05.707048 12493 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.708633 12493 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:05.713284 12663 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.719784 11826 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.720338 12699 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.731279  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.740948 12666 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.742816 12669 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:05.743871 12641 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.000s
W20260709 13:19:05.744653 12674 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.004s
I20260709 13:19:05.744774  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:05.780974  6597 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:19:05.789009  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.781942  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.793529  6597 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:05.796658  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.801931  6670 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:05.803234  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.790817  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:05.805835  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61120.2us
W20260709 13:19:05.790380 12655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.002s	sys 0.010s
I20260709 13:19:05.809231  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:05.815022  6597 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:05.816175  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.816184 12614 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:05.817786  6597 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:05.810472  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:05.791517 12641 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:19:05.670392 12589 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.825670 12614 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:19:05.830833  6667 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:05.829422 12719 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:19:05.831928  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:05.836371 12708 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:19:05.838447 12589 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.838882  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:05.839927  7228 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 13: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:05.841431  7228 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:05.844748  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:05.814746 12655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.002s	sys 0.011s
I20260709 13:19:05.813684 12568 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:05.863773  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.793473 12703 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:19:05.855366 12655 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:05.793790 12674 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:05.901912  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55616.8us
I20260709 13:19:05.824760 12685 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:05.907968 12449 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.927071 12449 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.929009  6589 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:05.955615 12412 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:19:05.962885 12412 consensus_queue.cc:1243] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:19:05.909112 12688 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:05.912942 12691 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:05.914664 12685 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.942411 12589 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:05.907302 12568 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:05.969516 12691 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:05.986466 12726 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:05.971041 12685 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:05.997468  6595 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:19:05.992626 12620 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14.
I20260709 13:19:06.005228 12568 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:05.978199 12688 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:19:05.980182 12589 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.023387 12589 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:06.023883 12721 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:06.027141 12674 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:06.028587 12568 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:06.029136 12568 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:06.030191 12568 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:06.034214  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.043558  6663 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:06.051244  6591 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:06.009841 12666 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:06.063776 12642 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:06.070978  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1513) took 1419 ms. Trace:
I20260709 13:19:06.071527  7231 rpcz_store.cc:276] 0709 13:19:04.642760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.677972 (+ 35212us) service_pool.cc:224] Handling call
0709 13:19:04.691584 (+ 13612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.918040 (+226456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.061602 (+1143562us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400646c54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:06.061837 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399573888}
I20260709 13:19:06.038734 12618 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.080749  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1154) took 1490 ms. Trace:
I20260709 13:19:06.081368  6666 rpcz_store.cc:276] 0709 13:19:04.590853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.602467 (+ 11614us) service_pool.cc:224] Handling call
0709 13:19:06.080520 (+1478053us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b540011dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:06.080666 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881377152}
I20260709 13:19:06.038334 12691 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.092389 12691 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:06.102911  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.103528  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.106662 11573 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:06.107549 11573 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:06.107486  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:06.108016 11573 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:06.108841  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.108806 11573 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:06.109777  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.110194 11573 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:06.056118 12729 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
W20260709 13:19:06.110405  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.187668 12604 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:06.118142  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.203361 12604 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.204239  6663 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:06.205168  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.205732  6663 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:06.206152 12604 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:06.146477 12469 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:06.116089  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1510) took 1474 ms. Trace:
I20260709 13:19:06.188941  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.146979  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2765) took 1252 ms. Trace:
I20260709 13:19:06.151430  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.208153  6593 rpcz_store.cc:276] 0709 13:19:04.894560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.894705 (+   145us) service_pool.cc:224] Handling call
0709 13:19:06.146797 (+1252092us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540023cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:06.146903 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270570880}
I20260709 13:19:06.208595  6662 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:06.129500  7231 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:06.209542  6593 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:06.152693  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:06.148029  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:06.210391  6662 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:06.210827  6597 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:06.148774  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.138711  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.213967  6667 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:19:06.140208  7233 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:06.157253  6594 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:06.164649  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:06.207324 12469 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.081167 12618 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:06.149540  6589 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:06.198005  6592 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.168296  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:06.219623  7235 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:06.220315  6595 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:06.216140  7236 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:06.147126  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:06.125841  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:06.136404 12663 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:06.226164 12663 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:06.071586 12720 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:19:06.207502  7237 rpcz_store.cc:276] 0709 13:19:04.642036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.664640 (+ 22604us) service_pool.cc:224] Handling call
0709 13:19:04.721128 (+ 56488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.774204 (+ 53076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.115837 (+1341633us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b54004de250. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:06.116016 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96465152}
I20260709 13:19:06.110461  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.123594 12706 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:06.231323  6665 raft_consensus.cc:2410] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:19:06.233278 12706 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:06.187225  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.110750 12705 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.237294 12705 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.230319  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:06.238992  7233 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.218720  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:06.239563  6669 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
W20260709 13:19:06.240484  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.242046  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:06.242686  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 166792us
W20260709 13:19:06.246886  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 100232us
I20260709 13:19:06.252185  6516 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:06.254211  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:06.218919 12469 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:06.255594  6666 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:06.275090  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 128124us
I20260709 13:19:06.281190  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:06.277333  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:06.279191  6593 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:06.288249  6521 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:06.288357  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:06.293267 12665 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:06.287948 12607 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.296921 12473 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:06.297032 12607 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.303253  6515 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:06.304617  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.275302 12728 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:19:06.305611 12702 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:06.227181 12663 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.306695 12702 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:06.306951 12663 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:06.307082 12702 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:06.308010 12702 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.308396 12551 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.309074 12551 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.311122 12551 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:06.312777 12702 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:06.317279  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 98658.6us
I20260709 13:19:06.318017  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:06.313247 12705 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:06.315410  6521 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:19:06.315631  6488 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.104s	sys 0.175s
W20260709 13:19:06.319527  6488 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.104s	sys 0.175s
W20260709 13:19:06.319818  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 509595us
I20260709 13:19:06.320761  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:06.293390  6637 leader_election.cc:400] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 10 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 10. Current term is 11.
I20260709 13:19:06.316255  6592 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:19:06.330076  6521 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:06.330652  6662 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:06.332211  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:06.332960  6514 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:06.246889 12618 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:06.325348  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
W20260709 13:19:06.325973  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 77976.2us
I20260709 13:19:06.342947  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:06.341952  7233 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:06.343719  6515 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:06.290287 12604 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:06.338373  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:06.338686  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:06.339342  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:06.345369  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:06.346378  6597 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:06.347349 11573 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:06.326159  6592 pending_rounds.cc:85] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 11
W20260709 13:19:06.336849  6488 connection.cc:256] RPC call timeout handler was delayed by -1.4063816269999734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
I20260709 13:19:06.007288 12655 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:06.492478  6488 connection.cc:256] RPC call timeout handler was delayed by -1.3864794140001777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 13:19:06.493280  6589 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:06.395291 12618 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:06.343377 12728 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:19:06.507570  6665 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:06.457831 12640 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 13:19:06.419636 12629 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.576s	user 0.001s	sys 0.018s
I20260709 13:19:06.395927 12469 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:06.410508  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 58715.7us
I20260709 13:19:06.373250  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:06.412676  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.426069  6517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.003s	sys 0.011s
W20260709 13:19:06.527653  6517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.005s	sys 0.013s
I20260709 13:19:06.395537  6514 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:06.324213 12603 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:06.417416  6667 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:06.528976 12469 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:06.353188 12667 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.530474 12667 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.531255 12469 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:06.532749 12667 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:06.482112  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.006s	sys 0.013s
W20260709 13:19:06.533353  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.007s	sys 0.014s
W20260709 13:19:06.533710  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 611025us
I20260709 13:19:06.539992  6588 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:06.321620 12551 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:06.332710 12477 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:19:06.414925  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57008.1us
I20260709 13:19:06.415630  6593 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:06.493186  6488 connection.cc:256] RPC call timeout handler was delayed by -1.3503484620000563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 13:19:06.547611  6488 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.549s (SENT)
I20260709 13:19:06.548032  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:06.392769 12669 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:06.548959 12669 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:06.549952 12669 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.538311 12568 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:06.500506  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 155196us
I20260709 13:19:06.553843 12607 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:06.554015  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:06.555675 12603 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:06.555886  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:06.373425 12736 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:06.512923  6663 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:19:06.556573 12603 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:06.557480  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2462) took 1319 ms. Trace:
I20260709 13:19:06.557859  6663 rpcz_store.cc:276] 0709 13:19:05.238038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.206454 (+968416us) service_pool.cc:224] Handling call
0709 13:19:06.512385 (+305931us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:19:06.512619 (+   234us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.557036 (+ 44417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.557397 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98061056}
I20260709 13:19:06.503810 12692 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:06.518592  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.508899 12678 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:06.561066 12678 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:06.561556 12678 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.561760 12603 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:06.548038 12618 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:06.564656  7233 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:06.564953  7236 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:06.566129  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:06.566330  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:06.566869  7237 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:06.561116 12660 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:06.566942  7236 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:06.568825 12685 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.000s
I20260709 13:19:06.569221 12685 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.571450  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:06.562163 12603 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:06.572055  7233 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:06.572691 12603 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.573132  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:06.573976  7235 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:06.574753  7230 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:06.573989 12603 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:06.573567 12741 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:06.580366  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2862) took 1362 ms. Trace:
I20260709 13:19:06.580802  6668 rpcz_store.cc:276] 0709 13:19:05.217664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.233135 (+ 15471us) service_pool.cc:224] Handling call
0709 13:19:05.233446 (+   311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:05.254812 (+ 21366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.254862 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:05.263794 (+  8932us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.006596 (+742802us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b18000ad620. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:19:06.063516 (+ 56920us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:06.063554 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.063640 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.579973 (+516333us) raft_consensus.cc:1685] finished
0709 13:19:06.580002 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.580275 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8838,"raft.queue_time_us":6574,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10382,"spinlock_wait_cycles":14478080,"thread_start_us":55768,"threads_started":1,"wal-append.queue_time_us":1143446}
I20260709 13:19:06.582877  6670 raft_consensus.cc:1217] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[14.12-14.12]   Dedup: 14.12->[]
I20260709 13:19:06.584182  6670 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:19:06.492444  6592 pending_rounds.cc:107] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:19:06.585714  6592 raft_consensus.cc:2889] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:06.585656  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:06.586409  7233 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:06.586584  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1292) took 1352 ms. Trace:
I20260709 13:19:06.586957  6592 rpcz_store.cc:276] 0709 13:19:05.234336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.234442 (+   106us) service_pool.cc:224] Handling call
0709 13:19:05.234752 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.197129 (+962377us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b58009626d0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fdd0aa 00007f04b905f447 00007f04b9077fe7 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4
0709 13:19:06.586322 (+389193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.586511 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1458773888}
I20260709 13:19:06.585846  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1162) took 1316 ms. Trace:
W20260709 13:19:06.588486  6669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1161) took 1324 ms (client timeout 1529 ms). Trace:
I20260709 13:19:06.588620  7234 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:06.588625  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:06.588354  6670 rpcz_store.cc:276] 0709 13:19:05.269786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.208636 (+938850us) service_pool.cc:224] Handling call
0709 13:19:06.582708 (+374072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.584590 (+  1882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.585616 (+  1026us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54001e0ad0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:06.585782 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859483008}
I20260709 13:19:06.589594 12469 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:06.589768 12636 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:19:06.589900  6669 rpcz_store.cc:269] 0709 13:19:05.264171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.283450 (+ 19279us) service_pool.cc:224] Handling call
0709 13:19:06.588332 (+1304882us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54002bcd54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:06.588427 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1266841472}
I20260709 13:19:06.590698 12469 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:06.590924 12523 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:06.591234 12469 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.591467 12523 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:06.591722 12469 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:06.592716 12473 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:06.592391 12523 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.593380 12646 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:06.593667 12523 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:06.593755 12646 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:06.595265  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1291) took 1361 ms. Trace:
I20260709 13:19:06.594995 12646 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.596190 12646 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:06.595643  6590 rpcz_store.cc:276] 0709 13:19:05.233366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.233997 (+   631us) service_pool.cc:224] Handling call
0709 13:19:06.148917 (+914920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.149238 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.149263 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.149741 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.499221 (+349480us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:06.499279 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.499398 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.584666 (+ 85268us) raft_consensus.cc:1685] finished
0709 13:19:06.584692 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.595010 (+ 10318us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b540023cdd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:06.595213 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3866,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":2040349056,"thread_start_us":348993,"threads_started":1,"wal-append.queue_time_us":398423}
W20260709 13:19:06.521631 12629 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.682s	user 0.003s	sys 0.023s
I20260709 13:19:06.598716  7232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1516) took 1357 ms. Trace:
I20260709 13:19:06.526386  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:06.590586  6636 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:06.599153  7232 rpcz_store.cc:276] 0709 13:19:05.240922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.241097 (+   175us) service_pool.cc:224] Handling call
0709 13:19:05.241541 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.257321 (+ 15780us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:05.283007 (+ 25686us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.093409 (+810402us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b1800486160. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9
0709 13:19:06.094116 (+   707us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b1800486160. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:19:06.104462 (+ 10346us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b1800486160. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:06.409901 (+305439us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.439017 (+ 29116us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.439211 (+   194us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:06.439250 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.439332 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.598472 (+159140us) raft_consensus.cc:1685] finished
0709 13:19:06.598499 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.598658 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8802,"prepare.queue_time_us":25999,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":16295,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":23205,"spinlock_wait_cycles":49046016,"thread_start_us":315435,"threads_started":1,"wal-append.queue_time_us":1310277}
I20260709 13:19:06.603847  7236 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:06.604552  6669 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:19:06.602192  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:06.608103  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:06.614919  6593 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:06.609397 12661 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:06.616868  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:06.618125  6517 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:06.560357 12692 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:19:06.618718  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:06.618953 12607 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.620174  6521 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:06.621070 12640 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:06.621438  6518 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:06.624969 12704 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:06.659050 12704 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.659543 12704 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:06.665560 12738 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.271s	user 0.004s	sys 0.007s
I20260709 13:19:06.641564 12661 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:06.667057 12661 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:06.675253 12685 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:06.677783  6514 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:06.680650 12678 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.622373  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:06.683588  6519 raft_consensus.cc:2410] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:19:06.623040 12736 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:06.684989 12736 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:06.685786  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:06.609771 12493 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:19:06.686165 12736 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.687959 12736 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:06.623436 11573 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:06.623813  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 76233.4us
I20260709 13:19:06.688421 12607 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.624241 12688 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:06.615597  7235 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
W20260709 13:19:06.690574  6488 connection.cc:256] RPC call timeout handler was delayed by -1.2233125160000782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
I20260709 13:19:06.628507  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:06.630841  7237 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
W20260709 13:19:06.607779 12622 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:19:06.645648  6515 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:06.649338  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 123033us
W20260709 13:19:06.622110  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 75095.8us
I20260709 13:19:06.625476 12326 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:06.651120  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.700229  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:06.654068 12640 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:06.625416 12691 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:06.690038 12748 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:06.651322 12604 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:19:06.702155 11573 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:06.705204  6488 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:06.706658  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.703675  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:06.698103 12607 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:06.714411  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.714329  6515 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:06.711392  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:06.718327  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.716931  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:06.702415 12640 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:06.703820 12326 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:06.701726 12706 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:19:06.709445 12477 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:06.762598  6516 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 11.
W20260709 13:19:06.775990  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.717335 12685 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.721274 12568 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:06.744488 12636 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.719671 12614 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.719410 12660 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:06.752043 12667 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:06.748593 12691 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:06.799000  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:06.800503  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:06.716450 12641 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:19:06.709720 12618 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:19:06.809558  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:06.814847  6590 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:06.776867 12477 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:06.789634 12622 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:06.820482  6515 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 13:19:06.821439  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 73857us
I20260709 13:19:06.826570  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:06.827003  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.751237 12681 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
W20260709 13:19:06.827338  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.802012 12667 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.809976 12728 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:06.825186 12477 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:06.807391 12549 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:19:06.832028 12622 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:06.888053 12622 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:06.846261 12663 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:06.809713 12660 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.891060 12663 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.892279 12660 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:06.893584 12663 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:06.851066 12469 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.895277  6592 raft_consensus.cc:2368] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16.
I20260709 13:19:06.860831  6590 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:06.835552 12568 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:06.835801 12667 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:06.875036  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48565.3us
I20260709 13:19:06.833743 12668 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:06.828797 12678 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:06.795773 12685 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.883704 12641 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:06.808398 12636 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.842424 12640 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.910019  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:06.879243 12691 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.809983 12614 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:06.950042  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:06.952484  6664 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:06.955662  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:06.956656  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:06.912436 12668 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:06.960840 11573 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:06.973671 11573 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:06.974952 12699 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:06.975410 12699 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:06.975572  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:06.976882 11573 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:06.976509 12699 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:06.978088 12699 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:06.952737 12568 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:06.957635 12702 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:06.957590 12604 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:06.984450 12640 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.910158 12326 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.005682  6488 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:19:07.004204  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 287337us
I20260709 13:19:06.986934 12568 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:06.986258  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 373451us
I20260709 13:19:07.012753  7237 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:06.995613 12706 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:07.011019  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.018003 11573 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.020140 11573 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.025614  7233 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:19:07.031370  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 327733us
I20260709 13:19:07.035009  6519 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:19:07.047483  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.048157  7233 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:07.049336  7233 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:07.050575  7233 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:07.012593 12568 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:07.048861  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.056661  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.064043  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.064405  6671 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:07.064594  6667 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:19:07.065433  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.057258  7233 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:07.057627  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.060860  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.069983  7234 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:07.061128  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:07.071123  7229 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:07.075336  6590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56ba1578b97b4faa966168899769e067"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:07.076086  6590 raft_consensus.cc:606] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.076447  6590 raft_consensus.cc:614] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:07.012473 12604 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:07.077482 12604 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.077911 12604 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:07.088243  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.089311  7237 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:07.097735  6590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:07.112288 11573 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.121760  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:07.124868  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:07.122409  6670 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:07.037145 12490 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:07.131377  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.047861 12666 raft_consensus.cc:476] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 13:19:07.037637 12669 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:07.037804 12523 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:07.047675 12748 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.053944 12702 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:07.051870 12629 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:07.061098 12665 raft_consensus.cc:476] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Not starting pre-election -- already a leader
W20260709 13:19:07.122923 12646 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.001s
I20260709 13:19:07.019470 12706 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.134662 12549 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:19:07.134824 12603 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
I20260709 13:19:07.151713  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:07.148737  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 402445us
I20260709 13:19:07.157146  6669 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:07.016655 12668 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.136054 12655 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.135550 12469 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.155140  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.187547  7200 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:07.190821  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:07.138592 12490 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.156591 12750 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:19:07.196981 12753 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:19:07.197484 12618 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:07.177609 11826 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:07.153394 12629 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.145391 12523 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.208408  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:07.150004 12748 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.189399 12668 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:07.187981  7229 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
W20260709 13:19:07.229466  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 74485.5us
I20260709 13:19:07.189119 12685 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:07.201522 12636 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.181730 12551 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:19:07.240065  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.244627  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 2920) took 1097 ms. Trace:
I20260709 13:19:07.149655 12669 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.243908  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.192565 12603 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:19:07.182524 12655 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.212980 12773 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:07.234968 12614 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.201668 12650 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:19:07.201916 12469 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:19:07.239737 12493 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:07.189286 12646 consensus_queue.cc:1175] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "921c423406da4f289884afca51ca198f" responder_term: 18 status { last_received { term: 15 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:07.267568 12655 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.259227  6591 rpcz_store.cc:276] 0709 13:19:06.146978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.279490 (+132512us) service_pool.cc:224] Handling call
0709 13:19:06.279881 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.280157 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:06.325357 (+ 45200us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.244100 (+918743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.244177 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:07.244245 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.244357 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.244535 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19707,"prepare.queue_time_us":73417,"prepare.run_cpu_time_us":1087,"prepare.run_wall_time_us":20552,"raft.queue_time_us":30861,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":12651,"thread_start_us":894947,"threads_started":2,"wal-append.queue_time_us":677274}
I20260709 13:19:07.274708  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:07.277458 12757 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:19:07.302752  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:07.285772  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:07.287631  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47837.5us
I20260709 13:19:07.303591  6514 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:19:07.303553  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 292593us
W20260709 13:19:07.295706  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 241221us
I20260709 13:19:07.297389  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:07.305573  6518 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:07.210923 12766 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
W20260709 13:19:07.274299 12661 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.005s
I20260709 13:19:07.282836 12523 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:07.275416 12758 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:19:07.275815 12759 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:19:07.274973 12762 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:19:07.279497  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:07.282465 12704 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.000s
I20260709 13:19:07.298318 12622 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:07.280616 12477 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:07.279829  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.295256 12751 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 13:19:07.271894 12748 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.260389 12636 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.343273  6670 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
W20260709 13:19:07.310175  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.306144 12493 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:07.294889 12641 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:07.325204 12768 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:07.356323 12641 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:07.356494 12768 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:07.276823 12669 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:07.277892 12763 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:19:07.357388 12768 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.357399 12641 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.353127 12622 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.362978  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.363971  6669 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:07.318790  7230 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:07.274366 12771 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:19:07.270596 12604 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.365497  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.366956  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:07.367605 12604 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.314252  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:07.367942  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:07.368683 12748 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:07.326068  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1183) took 1011 ms. Trace:
I20260709 13:19:07.369119 12748 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:07.369675 12748 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:07.337786  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.338899  6486 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:07.341245  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.301084 12705 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:07.278084 12490 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:07.344550  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.279033 12706 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.372560  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.367615  7230 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:07.373165  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 59158.1us
I20260709 13:19:07.369220  6513 rpcz_store.cc:276] 0709 13:19:06.314181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.315733 (+  1552us) service_pool.cc:224] Handling call
0709 13:19:06.316051 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.323401 (+  7350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.323428 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.329689 (+  6261us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.530237 (+200548us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b18004b0dc0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:19:06.953999 (+423762us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:06.954123 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.954213 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.325723 (+371510us) raft_consensus.cc:1685] finished
0709 13:19:07.325749 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.325998 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6711,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":10166,"spinlock_wait_cycles":3692416,"thread_start_us":423576,"threads_started":1,"wal-append.queue_time_us":777846}
I20260709 13:19:07.384486  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.323525 12629 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.353878 12523 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:07.394330  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.364876 12622 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.371135 12780 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.380671 12712 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.310261 12661 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.401113  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:07.320086 12704 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.382737 12771 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:07.384424 12768 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:07.383769 12604 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:07.384130 12490 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.380982 12766 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:07.373093 12636 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:07.417079  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 246051us
I20260709 13:19:07.417352  6513 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
W20260709 13:19:07.417950  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 209626us
I20260709 13:19:07.305104  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.424546  7230 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:07.395613 12773 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:07.429565  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:07.429744 12773 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:07.431272 12773 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.432066 12775 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:07.432668 12775 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:07.432881  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:07.433617 12775 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.434852 12779 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:07.435895 12779 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:07.436307 12779 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:07.436735 12779 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:07.436529  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.437938  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:07.438429  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 133433us
I20260709 13:19:07.438997  6513 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:07.439541  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.440297 12699 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:07.437513  7230 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:07.382213 12705 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.443326  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.422941 12604 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.456028  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:07.402691 12523 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.442561 12669 raft_consensus.cc:2764] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:19:07.462594  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.416563 12771 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:07.452770  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.481169 11573 raft_consensus.cc:2764] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:19:07.442266 12490 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.455065 12766 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.454500 12551 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:19:07.401178 12706 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:19:07.485484  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:07.413053 12780 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:07.480880 12736 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.792s	user 0.011s	sys 0.002s
W20260709 13:19:07.479636 12751 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:19:07.506227 12622 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:07.514734  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62086us
I20260709 13:19:07.483323 12705 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.514964  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.519695  6519 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
W20260709 13:19:07.509199 12778 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:19:07.508352 12604 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:07.525698  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.461174 12650 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
W20260709 13:19:07.533125  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 70583.1us
I20260709 13:19:07.460417  7200 leader_election.cc:400] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16.
I20260709 13:19:07.418642 12712 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.544570  7230 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
W20260709 13:19:07.545812  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 102516us
I20260709 13:19:07.559800  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.561578  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:07.480343 12493 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.519141 12702 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:07.514171 12771 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:07.556097  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.549742  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:07.547662  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.572674  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:07.566152 11573 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:07.573915 11573 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.574277 11573 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:07.538880 12665 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:07.567108  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 106872us
I20260709 13:19:07.578353  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:07.559882 12705 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.592128  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1694) took 1373 ms. Trace:
I20260709 13:19:07.585386  6519 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 11: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:07.585772  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.592798  7231 rpcz_store.cc:276] 0709 13:19:06.218384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.229362 (+ 10978us) service_pool.cc:224] Handling call
0709 13:19:06.229704 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:06.296118 (+ 66414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.296150 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:06.296715 (+   565us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.706147 (+409432us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:06.743876 (+ 37729us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.749846 (+  5970us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.759738 (+  9892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.759811 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.083158 (+323347us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b18003431e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:07.084276 (+  1118us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b18003431e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:07.591835 (+507559us) raft_consensus.cc:1685] finished
0709 13:19:07.591868 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.592051 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170980,"prepare.queue_time_us":90644,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":124120,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":22878,"spinlock_wait_cycles":66490368,"thread_start_us":409024,"threads_started":1,"wal-append.queue_time_us":666307}
I20260709 13:19:07.545229 12622 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:07.581260  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 202287us
I20260709 13:19:07.602942 11573 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.656026  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.656194 11573 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.584239 12699 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:07.657766  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:07.581632 12607 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:19:07.660005  6637 connection.cc:256] RPC call timeout handler was delayed by -1.0138396240001839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
I20260709 13:19:07.590322 12665 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:07.578318 12477 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.666752  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:07.667301 12523 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.668361  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.668881 12661 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.667335  6662 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:07.668957  6665 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:07.616111 12769 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:19:07.582746 12412 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:07.675748  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.677407 12779 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.673107  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.662029 12691 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:07.687283  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:07.636186 12780 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.660768 12699 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.679399 12523 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:07.692967  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.694271  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:07.698323  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.697470 12699 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:07.674407 12607 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:07.702044 12607 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.704911  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.705773  7234 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:07.692617 12636 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.692907 12692 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:07.707430 12692 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:07.701084 12640 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.695s	user 0.005s	sys 0.006s
I20260709 13:19:07.721542  6513 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:07.701764  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:07.716605  6637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.129:34121 timed out after 1.645s (SENT) [suppressed 32 similar messages]
I20260709 13:19:07.598671 12705 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.737298  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:07.744525 12667 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.823s	user 0.010s	sys 0.001s
I20260709 13:19:07.687879 12704 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.734068  6662 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:07.767304  6590 raft_consensus.cc:606] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:07.770432  6590 raft_consensus.cc:614] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:07.745708 12667 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.706583 12622 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.693423 12691 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:07.774801 12798 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:19:07.775228 12746 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.982s	user 0.001s	sys 0.004s
I20260709 13:19:07.784559 12766 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.708123 12629 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.696444 12785 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:07.810559 12785 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:07.769536 12568 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.637s	user 0.002s	sys 0.010s
I20260709 13:19:07.794456 12705 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:07.814994  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 234383us
I20260709 13:19:07.816248  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2859) took 2599 ms. Trace:
I20260709 13:19:07.818888  6516 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:07.805748 12629 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:07.807548 12810 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:19:07.810082  6669 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:07.816617  6664 rpcz_store.cc:276] 0709 13:19:05.216698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.232373 (+ 15675us) service_pool.cc:224] Handling call
0709 13:19:05.268369 (+ 35996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.957378 (+1689009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.816044 (+858666us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54000cbf54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:07.816196 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3872470912}
I20260709 13:19:07.798672  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.797313 12691 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:07.847204  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 130624us
I20260709 13:19:07.866910  7236 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:07.840112 12804 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:07.844426 12704 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.827893 12568 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.874871 12787 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:19:07.875352 12811 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:19:07.883531 12803 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:19:07.891083 12785 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.892755 12771 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.893236 12712 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.897342  7234 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:07.899092  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.899170  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.900105  7228 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:19:07.902982  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:07.903944  6662 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:07.870491 12779 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.872325 12804 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.911773  6517 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:19:07.917958  6515 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:19:07.919838  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.827320 12665 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.910832 12712 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:07.844830 12692 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.897867 12704 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:07.919595 12804 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:07.898396 12699 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.928306 12712 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.954195 12629 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.930559 12749 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:19:07.959156  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:07.979507  7236 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:07.968750  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:07.982460  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 415525us
I20260709 13:19:07.984661 12728 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:07.985074 12728 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:07.985724 12568 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.986976  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:07.989272  6516 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:07.990116 12814 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:19:07.990418 12604 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:07.992233 12802 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.357s	user 0.001s	sys 0.001s
I20260709 13:19:07.992897  6522 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:19:07.993856  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:07.995571 12704 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.998077 12604 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.998955 12773 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:07.935959 12667 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:07.935134 12771 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:07.999449 12773 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.000105 12773 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:08.002290 12748 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.000s
I20260709 13:19:08.003175  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:08.004125 12699 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.004745 12706 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:07.985935 12728 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:07.933982 12736 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:07.963063 12819 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:19:07.955468 12712 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:08.014986  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.050945  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.052129  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.053465  6669 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:08.054303  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.025206  6518 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:08.032618  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.033470  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.033599  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.034133  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.038280  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:08.057111  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.042012  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.005162 12704 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:19:08.045934  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.058789  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.968557 12809 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 13:19:08.048869  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.061230  6513 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:08.050042 11826 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:08.049856  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.062486 11826 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:08.051146  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:08.061272  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.059777  6563 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:08.063679  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.062482  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.065912  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.067869  6515 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:08.069058  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.069775  6515 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
W20260709 13:19:08.072614  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.062850 11826 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:08.077396  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:08.043640 12706 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.079089  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.084743  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.079281  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.086160  7231 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.072062 12641 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:08.092561 12641 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.093006 12641 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:08.046943 12646 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:08.101982  6515 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:08.095738  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.085366  6513 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:19:08.105633  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.106127  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.043483 12748 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.118141  6515 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:08.086293  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:08.119423 12748 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:08.119378  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:08.120090 12748 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:08.120450 12748 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:08.120787 12748 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:08.121058 12622 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.097539 12766 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:08.122960 12766 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:08.123193  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:08.124255 12668 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:08.125656 12668 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:08.126006 12668 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.126322 12668 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.115890 12691 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.111500 12804 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.130722 12646 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.131218 12728 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:08.131512 12804 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.123355 12766 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:08.132845 12646 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:08.106367  6513 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:08.134764  6517 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:08.135762  6517 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:08.136703 12412 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:08.136900  6517 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:19:08.137184 12412 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:08.137852 12691 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.094952 12706 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:08.103601  6516 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:08.145431  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:08.146028  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 57764.2us
I20260709 13:19:08.143146  7231 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:08.138892 11826 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.150733 11826 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:08.106755 12744 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 13:19:08.210978 12744 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:08.211351 12744 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:08.212163 12744 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.213310 12744 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:08.215647 12473 consensus_queue.cc:1175] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64b95375974b48d18c596a23c9cac75c" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 13 index: 12 } }
I20260709 13:19:08.225553 12748 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.232736 12748 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:08.168733 12568 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.247761  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.256000  6519 raft_consensus.cc:2393] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:08.183943 12745 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.262421 12745 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:08.270931 12820 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:19:08.276496 12412 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:08.292484  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.292508  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.293041  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.293726  7237 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:08.267014 12787 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:08.307009  7232 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:08.307077  7231 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:08.308143  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.309314  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.310148  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.256640 12568 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.315344  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:08.321712 12812 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.690s	user 0.002s	sys 0.000s
I20260709 13:19:08.329296 12641 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.333424 12622 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:08.310075 12766 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.373248 12766 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:08.375283  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.368714 12712 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.339519  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.362716  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.340286  6593 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 13:19:08.377832  7237 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:08.378166  6593 pending_rounds.cc:85] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 8
I20260709 13:19:08.378432  6593 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:19:08.378752  6593 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:08.345449 12668 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.377877  6595 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:08.380594  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:08.338460  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.336792 12706 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.340819  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.381624  6591 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:08.382073  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:08.382634  6592 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:08.381073 12668 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:08.347174  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.359979 12629 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:08.384424  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.369513  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.351392  6519 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:08.356340  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45976 (request call id 39) took 1269 ms. Trace:
I20260709 13:19:08.384922 12767 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:08.382355  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:08.385421  6517 raft_consensus.cc:2376] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17. Re-sending same reply.
I20260709 13:19:08.385784  6591 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:08.385687  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.294492 12646 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:08.384099 12618 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:19:08.386758 12646 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.387004 12493 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:08.385251  6596 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:08.388888 12646 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.343029 12412 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:08.341598  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.390064  6597 raft_consensus.cc:2410] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:08.389788  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:08.390275 12650 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:08.365815 12655 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.390914  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.391502 12650 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:08.391314 12655 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.345616 12568 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.341276  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:08.385707 12767 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:08.392760  6589 raft_consensus.cc:2368] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15.
I20260709 13:19:08.368830  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.393534  6588 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:08.393644 12640 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:08.393873 12655 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.389837 12618 consensus_queue.cc:1243] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:08.395293 12667 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:08.395678 12667 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:08.396415 12667 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.396873  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.391996  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:08.397476  6664 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:08.398286  6671 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:08.379249  6593 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:19:08.399443  6593 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:08.285255 12773 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.384542 12629 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:08.315866 12787 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.413648  7234 raft_consensus.cc:2368] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:08.391902 12650 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:08.394018 12640 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:08.376384 12691 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:08.400269  6592 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:08.341113  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:08.431900  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54566.1us
I20260709 13:19:08.385223  6590 rpcz_store.cc:276] 0709 13:19:07.087105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.097557 (+ 10452us) service_pool.cc:224] Handling call
0709 13:19:08.356124 (+1258567us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540019e6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b907b853 00007f04be04960a 00007f04b75cf7d4 00007f04b75cf762 00007f04b75cf691 00007f04b75cf60c 00007f04b75ce8c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:08.356239 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538025856}
I20260709 13:19:08.401084  6596 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:08.396885  6662 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:08.402074  6595 raft_consensus.cc:2368] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:08.368917 12745 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.346514 12678 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:19:08.408202  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.444819  6517 raft_consensus.cc:2376] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18. Re-sending same reply.
I20260709 13:19:08.434584  6519 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:08.402473  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:08.436184 12640 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.437429 12650 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.438663 12629 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.461513  6522 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:08.463020  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 64833.8us
I20260709 13:19:08.355806 12771 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.441910  6592 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:08.382817 12706 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.463569 12744 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:08.397526 12667 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:08.440485  6518 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:08.468438  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.480146  7237 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:08.404436 12766 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.480419  6591 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:08.344750 12490 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.481793  6594 raft_consensus.cc:2368] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14.
I20260709 13:19:08.453562 12681 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:08.496737  7232 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:08.495975  6596 raft_consensus.cc:2410] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:19:08.482329  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 290730us
I20260709 13:19:08.498346  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:08.484262  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:08.484525  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.484928  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.485747  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:08.439424  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.422889 12622 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:08.482452  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.508450  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:08.566133  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:08.566339  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58004.6us
W20260709 13:19:08.566169  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 126808us
I20260709 13:19:08.571727  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:08.567251  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.509321  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.568325  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.499828  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.487553 12744 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.573433  6670 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:19:08.576958  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.490523 12629 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.461490 12691 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.499562  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:08.575310  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.575314  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.495136 12650 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:08.585680  6666 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:19:08.569715  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 71033us
I20260709 13:19:08.500087  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.493722 12640 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:08.578738 12607 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:08.581598  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:08.598323  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:08.582907  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.594033  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.584201  6596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:08.611083  6596 raft_consensus.cc:606] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.611440  6596 raft_consensus.cc:614] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:08.603309  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:08.605486  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.606601  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:08.607373  6665 raft_consensus.cc:2368] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13.
I20260709 13:19:08.567816 12702 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:08.567643 12604 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:08.618665  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.570946 12779 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:08.627473  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46381.6us
I20260709 13:19:08.630208  6597 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:08.619371  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.577517 12744 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:08.486969 12771 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.481010 12692 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 13:19:08.588143 12745 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:08.638856  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.638994  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.496456 12706 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
W20260709 13:19:08.633644  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50843.3us
I20260709 13:19:08.654105  6488 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.654322  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.655140  6522 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:08.655368  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.636325  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.632557  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.571321 12622 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.497337 12766 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.656708  6521 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.658762  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.486229 12775 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.123s
I20260709 13:19:08.663214 12771 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:08.667484  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.670024  6488 leader_election.cc:400] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15.
I20260709 13:19:08.627264 12629 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:08.587915 12728 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:19:08.679167  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:08.616196 12549 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:08.643973 12699 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:08.628559 12767 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:19:08.687175  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72069.5us
I20260709 13:19:08.578785 12655 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:19:08.690732  6488 leader_election.cc:403] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:08.638948 12749 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.627526 12804 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:08.699590  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:08.687517 12806 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:08.656999 12641 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:08.692119 12797 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:08.632661 12704 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:19:08.640450 12412 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:19:08.643601 12779 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:08.698894 12706 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:08.694439 12702 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:08.645226 12790 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:19:08.733323  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:08.705696 12830 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:19:08.689954 12768 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 13:19:08.750511  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 80662.2us
I20260709 13:19:08.764186  6521 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:08.764201  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:08.765170  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.766417 11573 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15.
I20260709 13:19:08.694139 12549 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.737749 12706 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:08.813546  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:08.705484 12749 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.766142 12712 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:08.779175 12850 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:08.779500 12629 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:08.780840  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:19:08.831837  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 51129.3us
I20260709 13:19:08.781582 12473 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:08.779610 12826 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 13:19:08.771487  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 138375us
I20260709 13:19:08.880237  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:08.823065  6521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:19:08.756656 12849 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:08.834890  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:08.881042  6521 raft_consensus.cc:606] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:08.881479  6521 raft_consensus.cc:614] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:08.881388  6513 raft_consensus.cc:2376] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18. Re-sending same reply.
W20260709 13:19:08.823926 12784 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.192s	user 0.000s	sys 0.006s
I20260709 13:19:08.812012 12669 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.898556  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:08.831151 12629 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:08.831529 12850 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:08.878098 12604 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:08.966261  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 67801.8us
I20260709 13:19:08.967535 12604 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:08.972136 12549 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:08.972818 12629 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:08.959194 12850 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:08.916167 12669 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:08.980198 12829 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 13:19:08.980697 12799 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:19:08.981576 12846 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:19:08.901877 12712 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:08.991432 12836 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:19:08.908411 12768 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:19:08.993820  7236 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:19:08.995018  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2623) took 1986 ms. Trace:
I20260709 13:19:08.997802  6522 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:08.999485  6488 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:09.000020 12780 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:09.002295 12691 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.015151  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.020155 12699 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:08.915182 12704 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:09.034052  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:09.035013  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.055028  6594 raft_consensus.cc:606] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:09.055378  6594 raft_consensus.cc:614] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:09.055668  6563 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:09.056579  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.036069  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.060501  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.062058 12691 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:09.062770 12691 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:09.063162 12691 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.063570 12691 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:09.065054 12779 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.066783 12712 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:09.066970 12705 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:09.066829 12704 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:08.868561 12745 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:09.068849 12745 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:09.070091 12840 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.366s	user 0.002s	sys 0.000s
I20260709 13:19:09.071288  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:09.072012  6516 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:09.072903  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.047515  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:09.073834  6522 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:09.011497 12833 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 13:19:09.050827  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.037950 12622 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.080931  6669 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:19:09.052285  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 52889.4us
I20260709 13:19:09.081851  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.067495 12712 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:19:09.096383 12842 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:19:09.096740 12780 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:09.098273 12744 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.099632  6516 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:09.100411 12744 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.045222  6520 rpcz_store.cc:276] 0709 13:19:07.008707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.008864 (+   157us) service_pool.cc:224] Handling call
0709 13:19:07.009273 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.011879 (+  2606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.011912 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.013117 (+  1205us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.081378 (+ 68261us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b1800351a00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:19:07.508081 (+426703us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:07.508137 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.508224 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.729306 (+1221082us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:08.994787 (+265481us) raft_consensus.cc:1685] finished
0709 13:19:08.994809 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.994956 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2204,"raft.queue_time_us":2855,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":1761,"spinlock_wait_cycles":76094720,"thread_start_us":426473,"threads_started":1,"wal-append.queue_time_us":585997}
I20260709 13:19:09.077322  6487 leader_election.cc:400] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:09.102157  6487 leader_election.cc:403] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:19:09.050750 12768 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:09.058532  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.080441  6522 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:09.083406  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.093070  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.093567 12853 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:09.093708 12669 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.094584  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.105916  7234 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:09.104568  6488 leader_election.cc:400] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:09.105916  7236 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
W20260709 13:19:09.106431  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 47963.2us
I20260709 13:19:09.109127  6522 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:09.109992  6520 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:09.110728  6522 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:08.945822 12706 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.051265 12766 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.100492 12691 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.114728  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.116170  6522 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:09.116770  7236 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:09.122291  6522 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:19:09.113098  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:09.139071  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.139022  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:09.109213 12744 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:09.144498  6563 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:09.141701 12853 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:09.142278 12622 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:09.141520 12667 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.650s	user 0.011s	sys 0.000s
I20260709 13:19:09.177515  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:09.177749  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.178593  6521 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:09.178114  7236 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:09.174167 12704 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:09.173137 12669 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.143229 12766 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:19:09.195444 12848 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:19:09.196926 12853 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.139662 12604 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:09.199524  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 103020us
I20260709 13:19:09.231972  6488 leader_election.cc:400] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14.
I20260709 13:19:09.233132  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.174782 12681 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:09.207962 12749 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.210690 12669 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:09.212678 12831 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.192s	user 0.002s	sys 0.000s
I20260709 13:19:09.213783 12854 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:09.214322 12850 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.242192  6594 raft_consensus.cc:606] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.215785 12691 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.216707 12704 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:09.243651 12704 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.181778 12622 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:09.246809  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.247247 12622 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.251051  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2878) took 1023 ms. Trace:
I20260709 13:19:09.251420  6514 rpcz_store.cc:276] 0709 13:19:08.227993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.228137 (+   144us) service_pool.cc:224] Handling call
0709 13:19:08.232003 (+  3866us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:08.233555 (+  1552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.233587 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.234015 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.483405 (+249390us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:08.602656 (+119251us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.250514 (+647858us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b18001282c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287
0709 13:19:09.250646 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.250716 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.250755 (+    39us) raft_consensus.cc:1685] finished
0709 13:19:09.250866 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.250995 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59510,"prepare.queue_time_us":117586,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":41017,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":11468,"spinlock_wait_cycles":5095168,"thread_start_us":267964,"threads_started":2,"wal-append.queue_time_us":445636}
I20260709 13:19:09.218787 12787 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:09.219417  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.227317  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 165712us
I20260709 13:19:09.256085  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.237789 12773 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:09.238009 12568 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:09.261184  6561 leader_election.cc:400] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13.
I20260709 13:19:09.261529  6561 leader_election.cc:403] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:09.240002 12804 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:09.241529 12669 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:09.242599  6594 raft_consensus.cc:614] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:09.269026  7236 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:09.270925  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.192342 12828 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:19:09.272622  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.279342 12787 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.269989 12804 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:09.290946  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 594333us
I20260709 13:19:09.293133  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1530) took 2175 ms. Trace:
I20260709 13:19:09.294162  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.292495 12767 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:09.297292 12490 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:09.297842 12767 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.298477  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:09.299376  6592 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:09.285794 12748 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:09.304150 12858 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.304306  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.304710 12650 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.305280 12650 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.306125 12640 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:09.302382 12847 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:09.307811 12847 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:09.308912 12655 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:09.308638 12847 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.309432 12655 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.309898 12847 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:09.310144 12667 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:09.310750 12728 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.311636 12728 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.257908 12681 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.310793 12667 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.310882 12568 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.269989 12691 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:09.313393 12568 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.313962 12667 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.304719 12858 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.314766 12804 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.316545 12804 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:09.317134 12850 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.293664  7233 rpcz_store.cc:276] 0709 13:19:07.110290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.123802 (+ 13512us) service_pool.cc:224] Handling call
0709 13:19:07.124122 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.143936 (+ 19814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.143994 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.010843 (+866849us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b2c00047568. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:19:08.011212 (+   369us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.440415 (+429203us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:08.440448 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.440519 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.211816 (+771297us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:09.285229 (+ 73413us) raft_consensus.cc:1685] finished
0709 13:19:09.285251 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.285376 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23489,"raft.queue_time_us":19977,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9627,"spinlock_wait_cycles":62062976,"thread_start_us":428935,"threads_started":1,"wal-append.queue_time_us":882890}
I20260709 13:19:09.234843 12766 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:09.320168  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.295992  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.325680  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:09.327412  6590 raft_consensus.cc:2410] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:09.328321  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.333559  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.331357  6592 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:09.335793  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:09.336921  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.295928 12646 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:09.338414  6593 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:09.341410  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 112498us
I20260709 13:19:09.342356  6589 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:19:09.345700  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.347280  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.348136  6514 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:09.349895  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.351941  6593 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:19:09.340785 12493 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:09.344410 12641 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:19:09.339342 12681 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.339371 12655 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.341804 12748 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:09.345785 12691 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:09.340582 12618 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:09.344525 12650 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.349540 12850 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:09.361126  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:09.341168 12774 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:19:09.358213 12568 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.339514 12640 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:19:09.369733  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2890) took 1033 ms. Trace:
I20260709 13:19:09.374015  6588 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:09.375496  6590 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:09.380671  6519 rpcz_store.cc:276] 0709 13:19:08.336539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.385457 (+ 48918us) service_pool.cc:224] Handling call
0709 13:19:09.369540 (+984083us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b5400355a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:09.369673 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110427392}
I20260709 13:19:09.389421 12629 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:09.380703 12650 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:19:09.391119 12774 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:09.391598 12774 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:09.391939 12493 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:09.392360 12493 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:09.392472 12774 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.393231 12493 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.393859 12774 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:09.400156  7233 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:09.384681  6588 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:19:09.423533  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:09.424543  7233 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:09.380013 12850 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:09.353199 12790 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:19:09.383149 12728 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.374329 12655 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:19:09.381371 12618 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:09.434412  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:09.435604  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.437575  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.442833  6591 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:09.377202  6592 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:09.423817 12858 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:09.375115 12766 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.426991 12748 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:19:09.475106 11826 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:09.475550 11826 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:09.476265 11826 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.477167 11826 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.454545 12750 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.478727  7233 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:09.481324  6593 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:09.483142  6588 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:19:09.497762  6592 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:09.512375  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:09.517424  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.520895  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:09.488379 12750 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:09.533851  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 175995us
I20260709 13:19:09.541438  6588 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:09.508249 12850 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.544530 12641 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:09.545684 12641 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.546903 12655 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:09.549600 12641 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.549546 12655 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.551671 12655 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.556519 12750 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:09.519958 12618 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.567754 12493 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:09.560679 12861 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:19:09.589512 12870 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:09.590075 12870 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.592089 12870 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.592792  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:09.594336  6516 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:19:09.598548 11826 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:09.599097 11826 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.599387 11826 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:09.576618 12622 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.613547  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.621563  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.616279 11826 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.620095  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2644) took 1002 ms. Trace:
I20260709 13:19:09.624356  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.625041  6596 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:09.625866  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.638650  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:09.639259  7233 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.639739 12668 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:09.640149  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.641584 12804 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:09.646283 12862 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:19:09.647467  7233 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
W20260709 13:19:09.652529 12865 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:19:09.622397  6517 rpcz_store.cc:276] 0709 13:19:08.617670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.617786 (+   116us) service_pool.cc:224] Handling call
0709 13:19:08.618130 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:08.649860 (+ 31730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.649899 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:08.650418 (+   519us) log.cc:844] Serialized 52 byte log entry
0709 13:19:08.900650 (+250232us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:08.900692 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.582867 (+682175us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b5400044a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:09.582940 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.582991 (+    51us) raft_consensus.cc:1685] finished
0709 13:19:09.583024 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.620005 (+ 36981us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14149,"raft.queue_time_us":3739,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":14933,"spinlock_wait_cycles":72172416,"thread_start_us":182650,"threads_started":1,"wal-append.queue_time_us":360409}
W20260709 13:19:09.653779 12866 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:19:09.621289 12775 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.655647  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.655905  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.655601  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2486) took 1167 ms. Trace:
I20260709 13:19:09.692772  6669 rpcz_store.cc:276] 0709 13:19:08.488007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.488107 (+   100us) service_pool.cc:224] Handling call
0709 13:19:08.500390 (+ 12283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.081342 (+580952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.655383 (+574041us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54000c5dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:09.655547 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334350208}
I20260709 13:19:09.659543  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.694089  6594 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:09.660617 12750 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.661149  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:09.661677 12667 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:09.664852  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.665902  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:19:09.665796  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.668418  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:09.675791  7229 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:09.676975  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.678510  7235 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:09.706486  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.707167  7235 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:09.678519  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.679896  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.708105  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.688316  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:09.689239  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.690647  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:09.709774 11826 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:09.709884  6663 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:09.695624  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.711347  7236 raft_consensus.cc:2410] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:09.712320  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:09.686684 12787 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13.
I20260709 13:19:09.700371  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:09.719184  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 50874.6us
I20260709 13:19:09.722939  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.723681  6593 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:09.724568  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:09.710192  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.698599  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:09.725713  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.715070  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:09.726740  6518 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:09.705351  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.716439  6662 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:09.718502 11573 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:09.728346 11573 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.728747 11573 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:09.692528 12847 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.726359  6593 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:09.730892  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.759016  6593 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:19:09.764645  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:09.781563  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.783529  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.765549  6519 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:09.788926  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.759018  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:09.698720 12622 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.712440 12773 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:09.811777 12749 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:09.804332 12787 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.815605 12869 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:19:09.816104 12749 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:09.816704 12749 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.817046 12787 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.702337 12775 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.819651 12749 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.789274  6589 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:09.820437 12775 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:09.821002 12706 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:09.821331  6596 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:09.821328 12787 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.822540  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:09.825315  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:09.825604 12568 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:09.826117 12568 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.826638  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:09.828208  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:09.828438  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:09.829690  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.830103  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.832204 12622 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:09.832403 12867 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:19:09.703728 12667 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:09.795202  6663 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
W20260709 13:19:09.807951  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 75532.9us
I20260709 13:19:09.838007  6594 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:09.839993 11826 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:09.844887 11826 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.845039  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:09.811118  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 100989us
I20260709 13:19:09.845878  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.849466  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.716135 12852 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:09.791524  6593 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:09.842427  6590 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:09.791239  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:09.853698  7229 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:19:09.854967  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.856403  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:09.858748 12773 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.859681  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.828210 12568 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.881408  7232 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:09.881446  6595 raft_consensus.cc:2376] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15. Re-sending same reply.
I20260709 13:19:09.883701 12870 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.885136  6666 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:09.838727 12859 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:19:09.897779  6419 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 15 to 17, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 17 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:09.903661  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.904686  6596 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:09.906535 11826 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:09.713690 12750 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.812000 12669 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.863842 12704 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:09.868745  6668 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:09.870754  6518 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:09.884305 12870 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:09.872617  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:09.919781  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47250.5us
W20260709 13:19:09.874363  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.870813 12604 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:09.911320  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.930184  6593 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:09.789249 12847 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.871922 11573 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:09.917349  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:09.937232  6595 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:09.879660 12852 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:09.888684 12650 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.861874  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.948817  6419 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 16, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 16 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:09.936816  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.952721  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.953909  6593 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:19:09.960672 11573 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:09.966817  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.967772  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:09.969986  6594 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:09.970845  6419 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 17, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 17 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:09.972007  6593 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:09.903862 12748 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:19:09.972743  6419 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:09.975063  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:09.977020  6593 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:09.977627  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:09.978472  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:09.982803  6667 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:09.763684 12607 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.184s
I20260709 13:19:09.979013  6589 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:19:09.934814 12641 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:09.984602  6589 pending_rounds.cc:85] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 10
I20260709 13:19:09.988173  6589 pending_rounds.cc:107] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:19:09.988607  6589 raft_consensus.cc:2889] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.999696  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.000602  6666 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:10.001394  7232 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:09.947050 12604 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:10.006544  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.017159  6590 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:10.019055  6589 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:10.011021  6595 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:09.928052 12669 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.023129  6593 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:10.024438 12604 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.025187  6589 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:09.950636 12873 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:19:10.027529 11573 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:10.002411 12681 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:09.958424 12650 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.041209  6589 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:10.042939  6589 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:10.044360  6589 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:09.937839 12667 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.046324  6419 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 14, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 14 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.037895 12641 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:10.057329  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 365167us
I20260709 13:19:10.060827 12629 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:10.063203 12750 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.063648 12773 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.064371 12641 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.068463  7232 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [13.10-15.11]
W20260709 13:19:10.088696  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.057516 12681 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:10.059298 12871 consensus_queue.cc:1175] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64b95375974b48d18c596a23c9cac75c" responder_term: 15 status { last_received { term: 13 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:10.119647 12870 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:10.121547 12847 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:10.093942 12668 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:10.123818 12650 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:10.123932 12852 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.125376 12852 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:10.125485 12669 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.133313  6597 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:10.140806  7232 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
W20260709 13:19:10.143465  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.075169 12774 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:10.144078 12774 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.145078  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:10.146113 12774 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:10.146346 12745 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:10.146757 12745 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:10.147097 12745 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:10.150729  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.154028 12493 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:10.111351 12622 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:10.156088 12847 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:10.156626 12847 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:10.146138 12816 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:19:10.157953 12847 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.110961 12681 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.160984 12681 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.162272 12668 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:10.163501 12847 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.174326  6419 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 10 to 16, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 16 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.176451  6419 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 12, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 12 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.177837  6419 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.185333  6419 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:10.186807  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 427849us
W20260709 13:19:10.187806  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 357967us
I20260709 13:19:10.171509 12850 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:10.190511  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 295913us
I20260709 13:19:10.190963  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:10.194950  6419 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 14, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 14 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.202476  6419 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 13 to 15, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 15 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:10.204159  6419 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 16, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 16 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:10.205932  6419 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.171176 12493 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:19:10.186919 12650 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:19:10.198388 12774 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:19:10.206652 12775 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:10.235195 12775 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.236650 12745 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.209450 12607 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:10.240857  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:10.241233 12745 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.207695 12850 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.242762  6589 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:10.244329 12850 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.209856  6419 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 18, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 18 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:10.211916  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 232593us
I20260709 13:19:10.212610  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:10.199919 12871 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:19:10.224550 12667 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:10.225481  6418 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 328) took 49310 ms (client timeout 15000 ms). Trace:
I20260709 13:19:10.246783  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:10.247299  6515 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:10.247823  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.248829  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:10.248929  6419 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 12 to 17, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.258828  6419 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 14, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 14 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.262738 12850 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:10.264221  6419 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 13, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 13 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.270303 11826 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:10.270524  7235 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:10.270772 11826 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:10.271368 11826 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:10.266407  6419 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 10, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 10 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:10.273895  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:10.275498  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:10.276104  7230 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:10.276559  6591 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:10.279326  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:10.285529  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:10.286835 12745 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:10.287526 12745 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:10.288813 12745 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.305135  7235 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:10.273895  6419 catalog_manager.cc:5697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
W20260709 13:19:10.306977  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.310698  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 117110us
I20260709 13:19:10.311980  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:10.263253 12850 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:10.319793  6419 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:10.322141  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:10.326860  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:10.256048  6418 rpcz_store.cc:269] 0709 13:18:20.915386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.915714 (+   328us) service_pool.cc:224] Handling call
0709 13:19:10.225388 (+49309674us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:04.292487 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:04.474982 (+182495us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:04.475053 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:19:04.475078 (+    25us) tablet.cc:662] Decoding operations
0709 13:19:04.476293 (+  1215us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:04.476417 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:04.476447 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:19:04.477542 (+  1095us) tablet.cc:701] Row locks acquired
0709 13:19:04.477589 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:19:04.477774 (+   185us) write_op.cc:270] Start()
0709 13:19:04.477932 (+   158us) write_op.cc:276] Timestamp: P: 1783603144477710 usec, L: 0
0709 13:19:04.477957 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:19:04.606064 (+128107us) log.cc:844] Serialized 4364 byte log entry
0709 13:19:05.116666 (+510602us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b907ce28 00007f04bcab7581 00007f04bcab6d58 00007f04bcabb9f1 00007f04bcabb9a9 00007f04bcabb939 00007f04bcabb901 00007f04bcabab2d 000000000038f354 000000000038f199
0709 13:19:05.168973 (+ 52307us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:05.683522 (+514549us) op_driver.cc:464] REPLICATION: finished
0709 13:19:06.001573 (+318051us) write_op.cc:301] APPLY: starting
0709 13:19:06.010297 (+  8724us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:06.377046 (+366749us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:06.377095 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:06.618289 (+241194us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:06.625249 (+  6960us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:19:06.635563 (+ 10314us) write_op.cc:312] APPLY: finished
0709 13:19:06.677962 (+ 42399us) log.cc:844] Serialized 105 byte log entry
0709 13:19:07.046721 (+368759us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:07.060730 (+ 14009us) write_op.cc:454] Released schema lock
0709 13:19:07.112121 (+ 51391us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:07.154694 (+ 42573us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:07.157923 (+  3229us) write_op.cc:454] Released schema lock
0709 13:19:07.323221 (+165298us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b180034fae0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
Metrics: {"mutex_wait_us":379333,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":300271,"apply.run_cpu_time_us":77359,"apply.run_wall_time_us":1147367,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50085,"num_ops":11,"prepare.queue_time_us":708001,"prepare.run_cpu_time_us":46895,"prepare.run_wall_time_us":841206,"raft.queue_time_us":266171,"raft.run_cpu_time_us":20962,"raft.run_wall_time_us":169383,"replication_time_us":1205346,"spinlock_wait_cycles":85395456,"thread_start_us":1026447,"threads_started":5,"wal-append.queue_time_us":671709,"wal-append.run_cpu_time_us":78484,"wal-append.run_wall_time_us":2805664}]]}
I20260709 13:19:10.327461  6663 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:10.328768  6665 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:10.329165  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:10.329603 12858 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.329692  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.330611  6665 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:10.331667  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.331987  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.332743  6664 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:10.329723  6666 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:10.333774  6670 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:10.334118  6670 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:10.334878  6670 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.335850  6670 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:10.340348  6666 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:10.341660 12767 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:10.346155  6670 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:10.328379  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.348732 11826 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.348121  6419 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:10.350412 11573 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.351056 11573 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.350253  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:10.354044  6667 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:10.309562 12828 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:10.362015  6513 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:10.330116 12858 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.322527  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:10.357043  6419 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 14, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 14 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:10.342279 12767 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:10.365422  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53534.8us
W20260709 13:19:10.368640  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46201.4us
I20260709 13:19:10.368573  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.369602  6419 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 12, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 12 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:10.317674 12850 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:10.378010  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:10.383970  6515 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:10.384107  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:10.387042  6419 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.389418  6419 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 8 to 11, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 11 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:10.390374  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:10.391618  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:10.392212  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.396003 12870 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:10.396564 12655 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:10.397022 12655 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:10.397531 12655 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.422791  6668 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:10.427259  6589 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:10.366063 12828 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.429235  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.436376  6592 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:10.378831 12749 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:10.369903 12692 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:10.433918  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:10.401098  6419 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.442386  6419 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 7 to 14, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 14 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:10.444059  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.449227  6419 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 6 to 12, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 12 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.451078  6419 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:10.441059  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:10.399930 12847 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:10.424140 12774 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.498636  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.378633 12629 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:19:10.503762  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.508603  6591 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:10.508800  6594 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:10.444998 12828 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:10.499433  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:10.510721  6597 raft_consensus.cc:2376] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15. Re-sending same reply.
I20260709 13:19:10.445725 12618 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.399425 12894 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 13:19:10.460109 12816 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:10.459048 12775 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.493464 12853 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:10.495657 12749 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:10.391882 12871 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 13:19:10.437616 12523 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.096s
W20260709 13:19:10.550788 12882 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:19:10.518641 12692 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.580510 11573 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:10.507071 12607 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:19:10.445443 12870 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:19:10.495460 12702 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:19:10.536378 12816 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:10.589644  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:10.598590  7235 raft_consensus.cc:2393] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:10.591282  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:10.630400  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.634991  7231 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:10.576140 12853 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:10.503592 12767 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.583309 12607 consensus_queue.cc:1243] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:10.583309 12773 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:10.627214 12790 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.013s	sys 0.010s
I20260709 13:19:10.645375  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:10.649019  7229 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:10.628254 12816 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:10.596572 12775 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:10.545935 12618 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.661262  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:10.676542  7229 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:10.585472 12692 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:10.683313  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 226670us
W20260709 13:19:10.685560  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 299786us
W20260709 13:19:10.649984 12790 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.013s	sys 0.011s
I20260709 13:19:10.698753  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:10.709314  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:10.711833  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:10.724088  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:10.657058 12893 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:10.697453 12858 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:10.735868  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.677341 12775 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.744959  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:10.745798  6521 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:10.657892 12773 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.731642 12894 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:10.736035 12790 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:10.739701 12887 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 13:19:10.678337 12749 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.759037  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:10.760455 12790 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:10.772328  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:10.778018  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:10.735545 12893 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.762116 12669 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:10.782272  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 51767.8us
I20260709 13:19:10.774480 12790 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.764272 12910 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:19:10.783174 12669 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:10.740090 12853 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.779745 12847 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:19:10.791392 12919 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:10.820227 12907 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:19:10.821014 12901 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:19:10.809302 12669 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:10.864220  7230 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:19:10.822324 12898 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:19:10.822975 12896 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
W20260709 13:19:10.826232  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.826989 12913 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.827095 12902 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:19:10.872013 12913 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.835222  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.841733 11826 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:10.821645 12897 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:19:10.847016  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53068.3us
I20260709 13:19:10.874640 12913 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.876240  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:10.876379  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.877135  6596 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
W20260709 13:19:10.888351  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 176597us
I20260709 13:19:10.888566  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1194) took 1067 ms. Trace:
I20260709 13:19:10.891047  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:10.891309  6670 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:10.889962  6669 rpcz_store.cc:276] 0709 13:19:09.821403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.825007 (+  3604us) service_pool.cc:224] Handling call
0709 13:19:09.825363 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.000897 (+175534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.000930 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.001422 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.492755 (+491333us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:10.492823 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.581651 (+ 88828us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400160154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:10.581712 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:10.875706 (+293994us) raft_consensus.cc:1685] finished
0709 13:19:10.875741 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.888500 (+ 12759us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22171,"raft.queue_time_us":7011,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":20652,"spinlock_wait_cycles":339137536,"thread_start_us":422217,"threads_started":1,"wal-append.queue_time_us":620756}
I20260709 13:19:10.891598  6663 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.890420  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:10.893304  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:10.893885  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.898224 11573 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.898882 11573 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.899271  6665 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:10.900384  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:10.903801 12667 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.648s	user 0.011s	sys 0.000s
I20260709 13:19:10.904464 12667 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:10.904906 12667 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:10.906812 12618 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:10.907055  6671 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:19:10.907480  6671 pending_rounds.cc:85] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 11
I20260709 13:19:10.907714  6671 pending_rounds.cc:107] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:19:10.907852 12873 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:10.908401 12873 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:10.908816 12904 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:19:10.907956  6671 raft_consensus.cc:2889] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:10.905833 12667 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:10.913278 11826 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.914183  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:10.917177  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:10.920598  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.921654  6594 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:10.847566 12917 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:10.923487  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:10.876255 12871 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:10.930626  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:10.931660  6663 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:10.931900 11573 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.942687  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.902061 12828 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:10.897701 12899 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:19:10.933981  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:10.943713  6663 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:10.907977 12775 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:10.959573  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:10.963621 11826 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:10.909329 12873 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:10.929159 12917 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:10.966228  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:10.869912 12922 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:19:10.971702  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:10.937121 12871 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:10.987668  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1568) took 1316 ms. Trace:
I20260709 13:19:10.987869  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:10.979025  7234 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:10.948342 12852 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.946826 12692 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:11.000528  7233 rpcz_store.cc:276] 0709 13:19:09.670835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.671325 (+   490us) service_pool.cc:224] Handling call
0709 13:19:09.671701 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:09.935060 (+263359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:09.935111 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:09.935632 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.383942 (+448310us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:10.383986 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.532900 (+148914us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400575dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:10.532964 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:10.987268 (+454304us) raft_consensus.cc:1685] finished
0709 13:19:10.987290 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.987592 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8189,"raft.queue_time_us":2088,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":18639,"spinlock_wait_cycles":604488064,"thread_start_us":376590,"threads_started":1,"wal-append.queue_time_us":661852}
I20260709 13:19:10.961433 12926 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:10.957402 12748 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.924872 12915 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:19:10.975292 12912 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.009749  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:11.010991  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 95406.9us
I20260709 13:19:11.013134  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:10.989699 12910 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:10.990027 12775 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:19:10.988602 12917 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:11.040000 12766 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:11.040592 12766 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:11.014024 12905 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:19:11.048009 12766 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:11.059760 12920 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:19:11.037672 12912 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.061885  6667 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:11.005875 12852 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.063083 12912 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:11.064988 12852 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:11.065568 12774 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.001s
I20260709 13:19:11.066087 12774 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:11.066757 12871 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.068137 12871 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:11.073719 12893 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:11.074815 12910 consensus_queue.cc:1175] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "921c423406da4f289884afca51ca198f" responder_term: 14 status { last_received { term: 14 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 14 index: 8 } }
I20260709 13:19:11.006786 12926 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:11.055501 12906 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:19:11.038960 12927 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:11.059060 12916 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
W20260709 13:19:11.059613 12909 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:19:11.200104  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:11.007408 12692 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:11.051086  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:11.033905 12641 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:11.078299 12655 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.004s
I20260709 13:19:11.089731  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:11.092619  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.009151 12873 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:11.158143  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:11.208930  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 119286us
I20260709 13:19:11.159822  6520 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:11.020784 12915 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:11.216892  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:11.202407  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:11.221244  6594 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:11.221727  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:11.233358  6517 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:11.084081 12934 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:11.017474 12748 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.237980  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:11.146826 12926 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:11.240461  6519 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:11.249953  6667 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:11.249996  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.250986  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.251843  6663 raft_consensus.cc:2410] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:19:11.251622  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.210705 12922 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:11.220532 12668 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:11.220852 12622 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.201776 12912 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:19:11.224730 12928 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:11.239346 12934 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:11.271282  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.232784 12828 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:11.272405 11573 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:11.273866  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2901) took 1572 ms. Trace:
I20260709 13:19:11.274938  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.299453 11573 consensus_queue.cc:1243] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:11.299947  6667 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:11.275581  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:11.301218  6663 raft_consensus.cc:2368] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18.
I20260709 13:19:11.278498  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:11.217222 12655 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.228672 12692 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.256727 12932 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:11.263724 12668 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.299506  6671 rpcz_store.cc:276] 0709 13:19:09.702218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.702433 (+   215us) service_pool.cc:224] Handling call
0709 13:19:09.702760 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.911123 (+1208363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.273674 (+362551us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000c58d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:11.273819 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2768550656}
W20260709 13:19:11.309670  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.270205 12706 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.332581  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.355351  6667 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:11.333585  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.337030  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2815) took 1168 ms. Trace:
I20260709 13:19:11.342963  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1584) took 1074 ms. Trace:
I20260709 13:19:11.267367 12927 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:11.294224 12493 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:11.360361  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.357105  7237 rpcz_store.cc:276] 0709 13:19:10.268675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.272751 (+  4076us) service_pool.cc:224] Handling call
0709 13:19:10.273023 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.273325 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.273350 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.284252 (+ 10902us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.696949 (+412697us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:10.697008 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.697098 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.342645 (+645547us) raft_consensus.cc:1685] finished
0709 13:19:11.342681 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.342879 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1512,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":12642,"thread_start_us":350034,"threads_started":1,"wal-append.queue_time_us":461082}
I20260709 13:19:11.356529  6595 rpcz_store.cc:276] 0709 13:19:10.168755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.168884 (+   129us) service_pool.cc:224] Handling call
0709 13:19:10.169164 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:10.169479 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.169503 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:10.169988 (+   485us) log.cc:844] Serialized 52 byte log entry
0709 13:19:10.595173 (+425185us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:10.595219 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.595294 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.336706 (+741412us) raft_consensus.cc:1685] finished
0709 13:19:11.336740 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.336948 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1040,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":8664,"thread_start_us":424807,"threads_started":1,"wal-append.queue_time_us":589759}
I20260709 13:19:11.299050 12934 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:11.364710 12706 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.303284 12745 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:11.334888 12618 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.270540 12922 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.359910 12668 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:11.364046 12493 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:11.356581 12937 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.293151 12622 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.377601  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:11.341060 12858 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:19:11.392158 12816 consensus_meta.cc:287] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.002s
W20260709 13:19:11.422575  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.380143 12948 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:11.427776  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50283.6us
I20260709 13:19:11.428836  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:11.431252  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.431914  6663 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:11.434427  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.435602  6667 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:19:11.359707 12932 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:11.425362 12790 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.002s
W20260709 13:19:11.442646  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.425374 12850 consensus_meta.cc:287] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.001s	user 0.014s	sys 0.000s
I20260709 13:19:11.420442 12852 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:11.453105 12493 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.454576 12790 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:11.455132 12935 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:19:11.436373 12950 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.456614 12939 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:11.384231 12618 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.379506 12745 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.480419  6663 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:11.475832  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:11.529117  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53421.6us
W20260709 13:19:11.455921 12936 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 13:19:11.482049  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 94038.8us
I20260709 13:19:11.424808 12946 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:11.432593 12816 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.533850  7198 leader_election.cc:400] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18.
I20260709 13:19:11.538749  7198 leader_election.cc:403] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:11.540594  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:11.488281  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 409561us
W20260709 13:19:11.523833  7200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.004s	sys 0.007s
W20260709 13:19:11.478161 12924 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.379s	user 0.001s	sys 0.001s
W20260709 13:19:11.485708  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47971.8us
W20260709 13:19:11.543512  7200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.004s	sys 0.007s
I20260709 13:19:11.544009  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:11.544032  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 611752us
I20260709 13:19:11.531538  7230 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:19:11.431365 12912 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:19:11.547497  6663 pending_rounds.cc:85] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 10
I20260709 13:19:11.547947  6663 pending_rounds.cc:107] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:19:11.548264  6663 raft_consensus.cc:2889] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.366773 12927 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.551626 12927 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:11.552805 12927 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.553594 12927 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:11.525173 12931 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.426s	user 0.000s	sys 0.002s
I20260709 13:19:11.417718 12937 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.580448 11826 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:11.588483 11826 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:11.588912 11826 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:11.594267  6667 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:11.483062 12850 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:11.601686  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 308512us
I20260709 13:19:11.537410 12749 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:11.616216  6521 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:11.618508  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 71509.8us
W20260709 13:19:11.625296  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 83237.4us
I20260709 13:19:11.631381  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:11.637985 12568 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:11.640733 12568 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.646714 12568 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:11.637262  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 103489us
I20260709 13:19:11.652304 11573 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:11.652964 11573 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.656407 11573 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:11.669508 12767 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:11.643074 12891 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:11.649303 12925 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.634610 12928 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:19:11.642154 12893 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:19:11.681165 12667 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:11.483598 12852 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.436143 12948 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:11.704763  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 74791.9us
I20260709 13:19:11.710631  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:11.747007 12952 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 13:19:11.577363 12946 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.712031  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:11.537674 12958 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:11.579871 12493 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:11.592022 12939 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.723944  6668 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:11.604562 12858 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.597309 12956 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:11.746786  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:11.615231 12748 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:11.525437 12950 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.699721 12749 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:11.808094  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 264144us
I20260709 13:19:11.817790  6520 raft_consensus.cc:2393] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:11.839612  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:11.710845 11826 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:11.822664  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.742247 12655 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:11.742761 12773 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:11.800685 12766 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:11.791989 12935 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:11.791352 12692 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:11.843349  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.799273 12523 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:11.821419 12931 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.097s	user 0.000s	sys 0.004s
I20260709 13:19:11.812593  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:11.851308  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.853577  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:11.807438 12960 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:11.843530 12749 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:11.867275 11826 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:11.898483  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44997.4us
I20260709 13:19:11.739933 12568 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
W20260709 13:19:11.903787  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 76284.6us
I20260709 13:19:11.751096 12925 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.903432  7229 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:11.819567 12706 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:11.908682  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 96156.6us
I20260709 13:19:11.746649 12891 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:11.914873 12897 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:19:11.954607  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:11.954922 11826 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.977838 11826 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.895821 12958 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:11.951987 12896 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:19:11.838155 12858 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:11.953804  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:11.983428  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2664) took 1072 ms. Trace:
I20260709 13:19:11.913869 12622 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.002650  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:12.004458  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 159099us
I20260709 13:19:11.978595  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:11.998534  6516 rpcz_store.cc:276] 0709 13:19:10.911210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.912086 (+   876us) service_pool.cc:224] Handling call
0709 13:19:10.912403 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.966272 (+ 53869us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:10.970258 (+  3986us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.587862 (+617604us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.596336 (+  8474us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.596520 (+   184us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:11.596558 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.596627 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.983165 (+386538us) raft_consensus.cc:1685] finished
0709 13:19:11.983194 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.983345 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3547,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":225658,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":26908,"thread_start_us":670522,"threads_started":3,"wal-append.queue_time_us":285772}
W20260709 13:19:11.973054 12669 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.086s	user 0.008s	sys 0.004s
I20260709 13:19:11.993279  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2509) took 1094 ms. Trace:
W20260709 13:19:12.008651  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54920.8us
I20260709 13:19:12.017920  7200 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:12.011443  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46606.7us
I20260709 13:19:11.965106 12947 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:12.015126  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:12.015120  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:11.958164 12922 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:11.991776 12873 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.764s	user 0.013s	sys 0.000s
I20260709 13:19:12.017398  6664 rpcz_store.cc:276] 0709 13:19:10.898444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.901246 (+  2802us) service_pool.cc:224] Handling call
0709 13:19:10.901538 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.901879 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.901908 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.902370 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.974825 (+ 72455us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:11.002521 (+ 27696us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.008412 (+  5891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.008527 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.685845 (+677318us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b1800394840. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1
0709 13:19:11.687686 (+  1841us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b1800394840. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:11.993065 (+305379us) raft_consensus.cc:1685] finished
0709 13:19:11.993088 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.993215 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10616,"prepare.queue_time_us":27563,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":9214,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":2223,"spinlock_wait_cycles":51673728,"thread_start_us":72044,"threads_started":1,"wal-append.queue_time_us":153377}
I20260709 13:19:11.955597 12850 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:11.974568 12894 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.988739 12929 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:11.831063 12939 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:11.990404 12706 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:11.891780 12766 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.011518  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.022179  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:12.021504  7229 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
W20260709 13:19:12.025679  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47145us
I20260709 13:19:12.024510  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:11.836769 12956 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.045941  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.046248  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.046980  6595 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:12.047384  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:11.974320 12523 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:19:11.979911 12927 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.007529 12790 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.076799  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.078130  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.080470 11826 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.092434 11826 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:12.093019 11826 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:12.088271  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 66141us
I20260709 13:19:12.095707  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:12.045279  6668 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:12.084293  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.085429  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.087751  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:12.089378  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.097353  6595 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:12.098457  6596 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:19:11.986019 12773 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:19:11.994534 12913 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:19:12.097361  6591 raft_consensus.cc:2376] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18. Re-sending same reply.
I20260709 13:19:12.038944 12704 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:12.020474 12964 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.025029 12932 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:11.989034 12933 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.244s
I20260709 13:19:12.042272 12816 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.111680 12977 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.112727 12986 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 13:19:12.113637 12978 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:19:12.113814 12937 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:12.114248 12941 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.507s	user 0.001s	sys 0.001s
W20260709 13:19:12.115339 12971 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:19:12.116531 11826 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.118973 12948 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.071667 12745 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.119552 12889 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:12.113031 12930 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.357s
I20260709 13:19:12.122965 12956 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.124249  6664 pending_rounds.cc:85] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 8
I20260709 13:19:12.237627  6664 pending_rounds.cc:107] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:19:12.237970  6664 raft_consensus.cc:2889] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:12.124876 12926 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.859s	user 0.009s	sys 0.000s
I20260709 13:19:12.125721 12766 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.126574  6668 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:12.025655 12669 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.127466  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.071883 12894 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.132941  6590 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:11.920174 12960 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.081993  6667 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:12.140902  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:12.141567  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46005.2us
I20260709 13:19:12.080945 12893 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:12.143343  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:12.154584  6666 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.007s	sys 0.012s
W20260709 13:19:12.252219  6666 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.007s	sys 0.012s
I20260709 13:19:12.252288  6591 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:12.164322  6588 pending_rounds.cc:85] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 9
I20260709 13:19:12.253091  6588 pending_rounds.cc:107] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:19:12.253455  6588 raft_consensus.cc:2889] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:12.253846  6588 pending_rounds.cc:107] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:19:12.254129  6588 raft_consensus.cc:2889] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:12.179908  6562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.004s	sys 0.001s
W20260709 13:19:12.255220  6562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.004s	sys 0.001s
W20260709 13:19:12.180639  7231 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.001s	sys 0.005s
W20260709 13:19:12.258528  7231 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.001s	sys 0.005s
I20260709 13:19:12.080955 12947 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:12.183178  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.127197 12858 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.189340  6662 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:12.197705  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.144641 12790 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:19:12.164764 12969 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.340s	user 0.004s	sys 0.000s
I20260709 13:19:12.120491 12988 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:12.233644 12852 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.241878 12928 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:12.235292  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.247687  7237 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:12.248958  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:19:12.245798 12984 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.128s	user 0.001s	sys 0.007s
I20260709 13:19:12.110463 12604 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.128s
I20260709 13:19:12.264462  6596 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:19:12.241689 12704 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.258684 12665 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:12.264948  6513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.010s	sys 0.014s
I20260709 13:19:12.265239  7236 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
W20260709 13:19:12.265172  6563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.004s	sys 0.033s
I20260709 13:19:12.232157 12618 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.270941  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2513) took 1334 ms. Trace:
I20260709 13:19:12.243122 12937 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.297683  6669 rpcz_store.cc:276] 0709 13:19:10.936218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.938662 (+  2444us) service_pool.cc:224] Handling call
0709 13:19:10.939094 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.939529 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.939576 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.940184 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.253204 (+313020us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:11.253250 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.253316 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.270716 (+1017400us) raft_consensus.cc:1685] finished
0709 13:19:12.270741 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.270871 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":206,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":11987,"thread_start_us":312675,"threads_started":1,"wal-append.queue_time_us":498484}
I20260709 13:19:12.270592  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.272181  7232 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:12.272835  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:12.262881 12919 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.023s	sys 0.044s
I20260709 13:19:12.246323 12706 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.319638 12706 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.320731  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:12.319145  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:12.321699 12706 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.322299  6662 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:19:12.318463 12871 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.250s	user 0.008s	sys 0.004s
W20260709 13:19:12.319227 12919 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.024s	sys 0.044s
I20260709 13:19:12.315858  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:12.233963  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 91532.4us
I20260709 13:19:12.295992 12681 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:12.331952  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:12.332561  6662 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
W20260709 13:19:12.292516  6513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.011s	sys 0.015s
W20260709 13:19:12.292694  6563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.004s	sys 0.033s
I20260709 13:19:12.294603  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.290498  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.296731  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:12.300156  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:12.304075 12960 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.304829 12891 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:12.307585 12976 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.063s	user 0.003s	sys 0.000s
W20260709 13:19:12.308120 12962 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:19:12.307993  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.310287 12947 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.283037 12790 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:12.203680 12964 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.249646 12926 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.312107 12669 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.312574 12937 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.291512 12852 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.291400 12629 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.286226 12748 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:12.352924 12925 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:12.315074  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.315932  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.316461  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2503) took 1425 ms. Trace:
I20260709 13:19:12.204994 12977 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:12.339480  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49068us
I20260709 13:19:12.332095 12681 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:12.283396 12958 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.324890 12917 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.247s	user 0.006s	sys 0.007s
I20260709 13:19:12.313733 12979 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:12.283802 12523 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:12.291465 12692 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.286897 12974 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.319746 12939 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:12.358992  7230 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.018s	sys 0.006s
W20260709 13:19:12.350087 12668 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.934s	user 0.010s	sys 0.001s
I20260709 13:19:12.294809 12704 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.288264 12946 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:12.363499 12667 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.573s	user 0.002s	sys 0.008s
W20260709 13:19:12.369120 12934 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.989s	user 0.001s	sys 0.011s
I20260709 13:19:12.290742 12928 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.368290 12748 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.368353 12998 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:12.375911 12790 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.377319 12893 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:12.378139 12767 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.641s	user 0.002s	sys 0.007s
I20260709 13:19:12.381672 12937 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.383612 12681 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:12.389397 12692 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.318181  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.488956 12692 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.454430  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:12.455111  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:12.491343  6514 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.492096  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:12.315775  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.460366  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:12.494589 12607 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:12.340175  6591 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:12.495307 12607 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:12.496906 12607 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.342499  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.499078 11573 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:12.499393 12692 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:12.499583 11573 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:12.499697  6594 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:12.500030 11573 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:12.464965  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.500140 12692 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.401367 12704 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.465646  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.507095  6592 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:12.369200  6666 rpcz_store.cc:276] 0709 13:19:10.890969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.892541 (+  1572us) service_pool.cc:224] Handling call
0709 13:19:10.892897 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.893222 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.893249 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.915521 (+ 22272us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.242896 (+327375us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:11.361745 (+118849us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.252587 (+890842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.252662 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.316251 (+ 63589us) raft_consensus.cc:1685] finished
0709 13:19:12.316275 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.316407 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138290,"prepare.run_cpu_time_us":1202,"prepare.run_wall_time_us":4549,"raft.queue_time_us":25196,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":26920,"thread_start_us":1267908,"threads_started":3,"wal-append.queue_time_us":1187309}
W20260709 13:19:12.386034  7230 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.018s	sys 0.006s
W20260709 13:19:12.380851 12493 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.550s	user 0.004s	sys 0.004s
I20260709 13:19:12.367983  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:12.480139 12604 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:12.481637 12655 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.369844 12629 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.511765 12604 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.512050 12655 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.513772 12629 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.388798 12668 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.388031 12523 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.514533 12604 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:12.516666  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 148648us
W20260709 13:19:12.470005  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.369324 12852 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.402683 12977 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.516832 12523 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.386552 12917 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.541124 12668 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.406014 12929 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18.
I20260709 13:19:12.543977  7233 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:12.411787 12926 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:12.398783 12928 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:12.392565 12934 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.404404 12950 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.385717 12958 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.556659 12958 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.397125 12964 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:12.467815  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.389245 12974 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.560856 12974 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.399326 12998 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:12.563277 12998 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.415907 12988 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:12.569389  7232 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:19:12.569211  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:12.572680 12934 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.573879 12973 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:12.575997 12973 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:12.575770 12917 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.576578 12973 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:12.584632  6591 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:12.525403 12775 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:12.585112 12681 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:12.532087 13009 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:19:12.588944  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:12.593014 12655 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.552183 12852 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.551147  6514 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:12.522552 12692 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.596400 11573 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.609457 11573 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:12.573274 12749 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.008s
I20260709 13:19:12.617187  6671 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:12.612588  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1207) took 1324 ms. Trace:
I20260709 13:19:12.617969  6665 rpcz_store.cc:276] 0709 13:19:11.270554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.296328 (+ 25774us) service_pool.cc:224] Handling call
0709 13:19:11.296681 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.549344 (+252663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.549390 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.549447 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:11.549488 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.594234 (+1044746us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540012a054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:12.594307 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.594572 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580530432}
I20260709 13:19:12.628937 12913 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.629822 12934 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.629909 12913 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.630935 12934 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.623600 12749 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.633275 12934 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:12.634282 12982 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 13:19:12.635421 12934 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:12.635694  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.637477  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.634471 12749 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.635739 12972 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:12.635429 12853 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:12.659372 12975 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:12.659907 12975 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:12.659762 12853 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.661363 12975 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.653882 12972 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.665392 12972 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.650854  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:12.649614 12681 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.677734 11826 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:12.678131 11826 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:12.678594 11826 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:12.681787  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 57732.1us
I20260709 13:19:12.683404  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:12.685067  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:12.686075 12773 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:12.686902 12773 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:12.695822  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:12.697505  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.696269  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.698227  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.699261  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.699611  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.674105 12975 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:12.713135  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.714054  6591 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:12.715142  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.716058  6592 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:12.719101  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.678057 12853 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.736855  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.737160  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.738456  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:12.740149  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.741189  7230 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:12.736889  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:12.742307  6592 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:12.742132 11826 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.742832  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.743558  7236 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:12.743950  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.744334  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:12.746024 11826 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.744611  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:12.747129  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.746039  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.746361  7229 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:12.749963  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.751950  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2666) took 1839 ms. Trace:
I20260709 13:19:12.752754  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:12.755396  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1558) took 2113 ms. Trace:
I20260709 13:19:12.755533  6670 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:19:12.710186 12913 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.752379  6521 rpcz_store.cc:276] 0709 13:19:10.911924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.913715 (+  1791us) service_pool.cc:224] Handling call
0709 13:19:10.914015 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.629139 (+715124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.751354 (+1122215us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b5400044a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:12.751721 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614326144}
I20260709 13:19:12.756454  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2507) took 1864 ms. Trace:
I20260709 13:19:12.759577  6670 rpcz_store.cc:276] 0709 13:19:10.892151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.892280 (+   129us) service_pool.cc:224] Handling call
0709 13:19:10.903295 (+ 11015us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:12.754929 (+1851634us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54000689a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:19:12.756059 (+  1130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.756401 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726696704}
I20260709 13:19:12.705945 12934 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:12.749445  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 98712.6us
I20260709 13:19:12.763886  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.764760  6597 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:12.766455 13013 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:19:12.767331 12925 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.768427  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 132931us
I20260709 13:19:12.770190 12910 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:12.755788  7231 rpcz_store.cc:276] 0709 13:19:10.641431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.641780 (+   349us) service_pool.cc:224] Handling call
0709 13:19:10.642174 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.765330 (+123156us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:11.388898 (+623568us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b1800113080. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf
0709 13:19:11.389111 (+   213us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.258868 (+869757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.258920 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:12.258960 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.754866 (+495906us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400646c54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:12.754959 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.755179 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47336,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":84239,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":286919808,"thread_start_us":916962,"threads_started":3,"wal-append.queue_time_us":509815}
I20260709 13:19:12.772580  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1592) took 1495 ms. Trace:
I20260709 13:19:12.779101  7233 rpcz_store.cc:276] 0709 13:19:11.277168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.280896 (+  3728us) service_pool.cc:224] Handling call
0709 13:19:11.281230 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.544530 (+1263300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.772239 (+227709us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540027d8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:12.772495 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2902758144}
I20260709 13:19:12.787833  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.766856 12748 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:12.801282  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.802829  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:12.804241 12973 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.805917 12970 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:12.806511 12970 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:12.807142 12970 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:12.807773 11573 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:12.805226 12568 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:19:12.813062 12894 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.813966 12894 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:12.814530  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.815048 12641 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:12.815517 12641 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.815701 12493 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:12.815920 12641 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:12.816332 12493 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:12.816923 12493 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:12.819276  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 116843us
W20260709 13:19:12.821959  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48162.6us
I20260709 13:19:12.824685  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:12.824841  6595 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:19:12.825981 12655 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:12.826691 12655 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.787003  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.826354  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:12.780928 12910 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.840807 12977 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:12.841030  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.841358 12977 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.841817  6597 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:12.842777  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.842998  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.843930  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:12.844566  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.845954  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.841826 12977 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:12.849740 13017 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
I20260709 13:19:12.787178 12979 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:12.851483  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.814414 12894 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:12.861702  6597 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:12.844628  6590 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:12.877056  6522 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:12.878301  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.878700  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.879572  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.880787  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.881172  7237 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:19:12.881935  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.883248  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.839170 12910 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:12.887292  6521 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:19:12.886234  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:12.890398  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.890846 12655 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:12.891316  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.895648  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:12.896358  6662 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:12.897292  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.863819 12894 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:12.901271  6670 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:12.901523  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:12.902295 12568 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:12.903172  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.902761 12568 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.903903 12568 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:12.904587  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.902400  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.909279  7229 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:12.786586 12925 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:12.951370  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:12.906418 12853 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:12.918205 12774 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:12.954066  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.958182  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:12.957437  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:12.967157 12933 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:12.967510  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 141316us
I20260709 13:19:12.968487  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:12.969094  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46132.3us
I20260709 13:19:12.969871  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:12.969875  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:12.970808 12607 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:12.970950  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.971258 12607 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:12.971715 12607 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:12.972380  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.973133  6669 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:12.974488 12889 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:12.975085 12889 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:12.975185 12774 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:12.976114  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.976171 12889 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:12.977536  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:12.964891  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:12.980362  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 81014.6us
I20260709 13:19:12.976869  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:12.985740  6671 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:12.960075 12692 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:12.984525  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:12.974488 12975 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:12.994882  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 75399.6us
I20260709 13:19:12.999612 12914 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:12.999980  6521 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:13.001976  6671 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:13.002820  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:13.008709 13021 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:19:12.981513 12921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.006s	sys 0.016s
I20260709 13:19:13.001708  7236 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:13.025820  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:13.008227  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:13.006140  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:12.989398 12692 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.043514  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:13.012239 12975 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:13.039726 12921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.007s	sys 0.016s
I20260709 13:19:13.062352  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:13.057019  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:13.044441 12692 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:13.064721  7231 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
W20260709 13:19:13.069226  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53837.2us
I20260709 13:19:13.072041  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:13.043382 12965 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:13.017401 12853 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.058720 12975 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:13.065359  7234 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:13.077621  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:13.077435  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:13.088119  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:13.152693  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 89790.6us
I20260709 13:19:13.062391 12925 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:13.155658  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 83926.5us
I20260709 13:19:13.162549  6663 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:13.166126  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.160324  6488 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:13.180836  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 185613us
I20260709 13:19:13.090476 12775 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:13.066112 12964 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.253s
I20260709 13:19:13.188944 12774 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:19:13.186374  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:13.192273 12871 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:13.201112  7228 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:13.092815 12493 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.211828 12745 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:13.198393 12912 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:13.206528  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.220995  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:13.221632  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61429.6us
I20260709 13:19:13.193279 12655 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.222831 12655 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.224198 12973 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:13.227507 12891 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:13.227979 12891 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:13.228420 12891 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:13.231155 12976 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:19:13.233223 12847 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:13.233834 12847 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:13.215583 12965 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.249431 12965 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:13.251039 13026 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.292s	user 0.000s	sys 0.002s
W20260709 13:19:13.262805  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 256977us
I20260709 13:19:13.263043 12934 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.264884  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.267012 12998 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:13.221949  6520 raft_consensus.cc:2410] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:13.184729 12977 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.287204 12977 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.250391 12948 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:13.280498  6669 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:13.275731  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:13.277835  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 71418us
I20260709 13:19:13.278916  6590 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:13.296140  6593 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:19:13.267505 12912 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.285472 12607 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:13.301436  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 96627.6us
I20260709 13:19:13.301570  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:13.301971  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:13.302369  6519 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:13.303437  6517 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:13.302354 12607 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:13.304136 12989 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:13.304898 12989 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:13.305281 12607 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:13.306002 12607 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:13.292889 12948 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:13.309034 12948 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:13.314452  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:13.314962 12607 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:13.315366  6520 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:13.315843  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:13.316154  7233 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:13.316764  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:13.317190  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.318300  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.318531  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:13.319330 12873 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:13.319365 12667 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:13.318467 12925 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:13.319954 12873 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.320083 12667 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:13.303725  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:13.318300  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:13.334403  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:13.334674  6517 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:13.320415 12925 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:13.340139  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:13.315217  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:13.339154  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.341601  6518 raft_consensus.cc:2410] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:13.297161 12977 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:13.339882 12925 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:13.359073 12915 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:13.359817 12915 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:13.359490 12925 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.360885 12925 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:13.360874 12915 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.328027 12667 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:13.328406 12873 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:13.352191  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48590.9us
I20260709 13:19:13.306113 12989 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.353616 12937 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:13.321902 12912 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:13.350873 12568 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.371558  6517 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:13.364585  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.365628  6663 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:13.217345 12871 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:13.385344 12641 consensus_meta.cc:287] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.004s
I20260709 13:19:13.383957 12937 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:13.433722  6520 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
W20260709 13:19:13.422794 12970 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.008s
I20260709 13:19:13.433085  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:13.419526 12977 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:13.439944  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:13.443533  7232 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:13.444044  7237 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:19:13.465289  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:13.465451  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.466388  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:13.430281 12641 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.452986  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.433892 12975 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.446314 12977 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.490583  6662 raft_consensus.cc:2368] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15.
I20260709 13:19:13.443085 12970 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.489285 12977 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:13.483700 12853 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.484556 12749 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:13.506384  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:13.516781  6588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.002s	sys 0.033s
W20260709 13:19:13.517354  6588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.002s	sys 0.033s
I20260709 13:19:13.518059  7228 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:13.520720 12910 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.000s
I20260709 13:19:13.521162 12910 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.521229  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:13.526436  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:13.502023 12493 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:13.530570  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 109220us
I20260709 13:19:13.531386  6595 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:13.532595  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:13.544296  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:13.538089 13029 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.070s	user 0.001s	sys 0.001s
I20260709 13:19:13.476351 12847 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:13.549623 13030 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:19:13.550840  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.545840 12871 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.535331 12568 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.553961  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:13.540728 12963 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:13.562093  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:13.563602  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:13.552057 12958 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:13.558220  7228 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:13.583439  6595 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.566895 12704 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:13.572824 12910 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.571153 12970 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.580629 12958 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:13.565444 12871 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:19:13.529595 12937 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.572577 12692 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.590258 12847 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:13.605257  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:13.605846  6563 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.591100 12958 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:13.591512 12975 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:13.604027 12894 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.009s
I20260709 13:19:13.619450  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.604285 12641 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.631070  6596 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:13.636338 12704 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:13.651225  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:13.653231  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 116159us
W20260709 13:19:13.653409  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 235835us
I20260709 13:19:13.657085  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.657220 12622 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:13.643553 12894 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.664026  6665 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:13.665084 12894 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.667613 12704 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:13.668675 12704 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:13.643034 12853 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:13.671183 12853 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.674208 12853 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.675130 12692 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:13.671733 13024 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.462s	user 0.002s	sys 0.005s
I20260709 13:19:13.670145 12704 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:19:13.658087 12622 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.707692 12692 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:13.727226  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.727844  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.752853  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.728044  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:13.758356 12891 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.003s
I20260709 13:19:13.799292 12891 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.801676 12891 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.701618 12958 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:13.803733 12958 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.803797 12622 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.807968 12667 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.809182 12948 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:13.825251  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 168066us
W20260709 13:19:13.815306 13022 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.821s	user 0.005s	sys 0.000s
I20260709 13:19:13.810431 12667 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.829500 12622 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.830128  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:13.859192  7233 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:13.824379 13040 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:19:13.864907  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 361673us
I20260709 13:19:13.855702  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:13.866963  7237 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:13.872113  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.874167  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.874219  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:13.871937  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 214907us
I20260709 13:19:13.877107  6663 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:13.880439  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 183112us
I20260709 13:19:13.881278  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.883168  6670 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:13.884327 12655 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.884935 12655 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:13.888475  6591 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:13.889972  6666 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:19:13.866811 13039 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:19:13.896723  6663 raft_consensus.cc:2468] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:13.830742 12622 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.866022 13040 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:13.848863 12915 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:13.899108  7237 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:13.877368 13043 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:13.904121 13041 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:13.903465  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.917467  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.919685  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.921094  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2931) took 1326 ms. Trace:
I20260709 13:19:13.930385 12622 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:13.931336 12912 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.001s
I20260709 13:19:13.925544 13046 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:13.931155 13041 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:13.932149 12912 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.932520 13050 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:13.933285  6592 raft_consensus.cc:2368] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19.
I20260709 13:19:13.933565 13041 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:13.933971 13050 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:13.933985  6595 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:13.907184 12915 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:13.934111 13041 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:13.913326 13043 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.901877 12655 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:19:13.948855 13041 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.949940  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:13.951853 13041 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.921995  6664 rpcz_store.cc:276] 0709 13:19:12.594758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.684485 (+ 89727us) service_pool.cc:224] Handling call
0709 13:19:12.685133 (+   648us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:12.774155 (+ 89022us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:12.980641 (+206486us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.191552 (+210911us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.274987 (+ 83435us) log.cc:844] Serialized 52 byte log entry
0709 13:19:13.294821 (+ 19834us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:13.295197 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.295442 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.296889 (+  1447us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.354991 (+ 58102us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b1800009b80. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:13.919823 (+564832us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b540006b6a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff6dbe 00007f04b9001930 00007f04b90018d8 00007f04b9001858 00007f04b9001813 00007f04b9000a9c 00007f04beeec626 00007f04beee6dd8 00007f04b903d7cb 00007f04b90402c0
0709 13:19:13.920766 (+   943us) raft_consensus.cc:1685] finished
0709 13:19:13.920788 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.920918 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100183,"prepare.queue_time_us":73970,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":95967,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":6683,"spinlock_wait_cycles":36052608,"thread_start_us":389714,"threads_started":2,"wal-append.queue_time_us":214097}
I20260709 13:19:13.959470 13050 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:13.960254 12910 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:19:13.960773 12873 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 13:19:13.961225 12873 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.961915  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:13.962816 12910 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:13.963380 12910 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:13.963480 12873 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.963778 13051 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:13.975409  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:13.940025 12915 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:13.973896  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:13.966459  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:13.938339 13046 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:13.946557 13043 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.978739 12948 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:13.976156 13040 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.998338 13040 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:13.988400 12745 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:13.999818 12745 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.000792 12745 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.002023 12745 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:13.982473 13041 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:14.003858 13041 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:14.005209 13041 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:13.964459 12910 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.011610  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:14.015187 13051 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:14.016157 13051 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:14.016964 13051 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:14.017347 13051 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:14.020622  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:14.029660  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 67791.5us
I20260709 13:19:14.026742 13050 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:14.031649  6561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.500s (SENT) [suppressed 14 similar messages]
I20260709 13:19:14.031975 13050 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:14.034457 13028 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.812s	user 0.001s	sys 0.003s
I20260709 13:19:14.034914  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:14.035952 12934 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:14.036412 12934 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:14.036641  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2930) took 1445 ms. Trace:
I20260709 13:19:14.036828 12934 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.037086  6667 rpcz_store.cc:276] 0709 13:19:12.593035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.593201 (+   166us) service_pool.cc:224] Handling call
0709 13:19:12.593769 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.767827 (+174058us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:12.780353 (+ 12526us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.983084 (+202731us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.989898 (+  6814us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.996733 (+  6835us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.996939 (+   206us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:12.996989 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.997080 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.968366 (+971286us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b1800289de0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f622a6 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f
0709 13:19:13.970211 (+  1845us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b4800220158. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f 00007f04be098f37 00007f04be0ad55c 00007f04be0ad508 00007f04be0ad488 00007f04be0ad443 00007f04be0ac6ec 00007f04bcfd6a96
0709 13:19:13.998600 (+ 28389us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b1800289de0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:14.033802 (+ 35202us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:14.036308 (+  2506us) raft_consensus.cc:1685] finished
0709 13:19:14.036335 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.036474 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173227,"prepare.queue_time_us":79471,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":3663,"raft.queue_time_us":188248,"raft.run_cpu_time_us":7819,"raft.run_wall_time_us":983543,"spinlock_wait_cycles":158744960,"thread_start_us":213850,"threads_started":2,"wal-append.queue_time_us":427879}
I20260709 13:19:14.039480  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:14.040107  6515 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:14.043879 12912 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.047628 13050 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.049942  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.050966  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:14.052837  6668 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:14.054750  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.055188  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.056919  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.057430  6670 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:14.057256 13043 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.060489  6521 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:13.985450 13046 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.065852  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2940) took 1174 ms. Trace:
I20260709 13:19:14.066286  6665 rpcz_store.cc:276] 0709 13:19:12.891779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.894776 (+  2997us) service_pool.cc:224] Handling call
0709 13:19:12.895081 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.664426 (+769345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.063736 (+399310us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b54001cacd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:14.065783 (+  2047us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767914240}
I20260709 13:19:14.068611  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:14.069553  6517 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:14.070554  6669 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
W20260709 13:19:14.066918  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 555019us
I20260709 13:19:14.067622 12970 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:14.076166  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:14.080468  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.081691  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.082705  7236 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:14.083689  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.085135  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.085709  7235 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:14.086570 12745 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:14.087118 12745 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.089546  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:14.090260  6664 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:14.069270 12937 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:14.068158 12749 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:14.103806  7198 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:14.018342 13051 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:14.115753  6515 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
W20260709 13:19:14.105757  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.088959 12889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.005s	sys 0.031s
I20260709 13:19:14.106278  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1355) took 1252 ms. Trace:
I20260709 13:19:14.093371 12937 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.122253  6593 rpcz_store.cc:276] 0709 13:19:12.848249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.850579 (+  2330us) service_pool.cc:224] Handling call
0709 13:19:13.294986 (+444407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.298508 (+  3522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.100729 (+802221us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54001b6f50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:14.100888 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1021034752}
W20260709 13:19:14.137423 13037 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.500s	user 0.002s	sys 0.013s
I20260709 13:19:14.137756  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.137986 12934 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:14.137471  7235 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:14.145960  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.147746  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.149245  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.088129 12745 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:14.152763  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 74973.5us
W20260709 13:19:14.124684 12889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.005s	sys 0.032s
I20260709 13:19:14.109115 12692 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:14.163167  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 131534us
I20260709 13:19:14.170143  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.129659 12937 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:14.125352 12998 raft_consensus.cc:2804] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:14.108316 12975 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:19:14.168534  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:14.167701  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 93474.3us
I20260709 13:19:14.105652 12979 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
W20260709 13:19:14.214123  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45699.2us
W20260709 13:19:14.228763  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 77566.6us
W20260709 13:19:14.229476  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.218173 12979 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:14.220077  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:14.220775 12977 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.660s	user 0.010s	sys 0.002s
I20260709 13:19:14.221341  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.240782  7232 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.224802  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:14.225519 12847 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.607s	user 0.006s	sys 0.006s
I20260709 13:19:14.172221 12940 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.216504  7235 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:14.230919  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.237679  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:14.239018  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.248999  6594 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:14.215045 13051 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:14.216019 12998 raft_consensus.cc:697] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:14.247038 12775 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:14.172219 12692 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:14.282756  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:14.292586  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.240087 12979 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:14.260392 12748 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:14.297420  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 69989.2us
I20260709 13:19:14.304792  7235 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:14.305577  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.236848 12853 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:19:14.308374  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:14.313427  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:14.312256 12889 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:14.315032  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.315089 12889 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:14.317463  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.317631  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.318116 13060 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:14.318270  7236 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:14.318727 13060 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.320708 13050 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:14.321308  6597 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:14.321277 13050 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:14.322888  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.322839 12775 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:14.324110 12910 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.324505 13050 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:14.257764 12977 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.324618 12910 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.327152 12977 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.330932  7236 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:14.326972 12910 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.326910 12748 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:14.339183  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1605) took 1366 ms. Trace:
I20260709 13:19:14.340282  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1362) took 1030 ms. Trace:
I20260709 13:19:14.341043  6666 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:14.303560 12622 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:14.345078  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1365) took 1034 ms. Trace:
I20260709 13:19:14.348261  7234 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:14.339596  7229 rpcz_store.cc:276] 0709 13:19:12.973298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.986208 (+ 12910us) service_pool.cc:224] Handling call
0709 13:19:12.996674 (+ 10466us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:13.195469 (+198795us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:13.217608 (+ 22139us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.481398 (+263790us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.486885 (+  5487us) log.cc:844] Serialized 52 byte log entry
0709 13:19:13.487031 (+   146us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:13.487258 (+   227us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.741542 (+254284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.741670 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.973317 (+231647us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b1800240520. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:14.060439 (+ 87122us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b1800240520. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04bcf5558f 00007f04b5f94571 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd
0709 13:19:14.136478 (+ 76039us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:14.338992 (+202514us) raft_consensus.cc:1685] finished
0709 13:19:14.339010 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.339129 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":562990,"prepare.queue_time_us":22111,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":3221,"raft.queue_time_us":31828,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":12938,"spinlock_wait_cycles":29708160,"thread_start_us":443301,"threads_started":2,"wal-append.queue_time_us":298530}
I20260709 13:19:14.350946  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:14.352139  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.352669  6668 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:14.353420  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:14.353884  6668 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:14.345489  6597 rpcz_store.cc:276] 0709 13:19:13.311184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.312622 (+  1438us) service_pool.cc:224] Handling call
0709 13:19:13.975192 (+662570us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:14.344916 (+369724us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400081454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:14.345025 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777121280}
I20260709 13:19:14.357304  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.358135  6670 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:14.340757  6592 rpcz_store.cc:276] 0709 13:19:13.310080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.311737 (+  1657us) service_pool.cc:224] Handling call
0709 13:19:13.316596 (+  4859us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:14.340111 (+1023515us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b540019e6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:14.340229 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424792064}
W20260709 13:19:14.362128  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 101446us
I20260709 13:19:14.266768 12940 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.247781 12749 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.372560  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.362927 12975 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:14.378341 12940 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:14.368302  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.366547  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.313086 13043 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:14.364293  6662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1223) took 1453 ms (client timeout 1775 ms). Trace:
I20260709 13:19:14.384871  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.385393  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.391263  6594 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:14.395895  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:14.395424  6662 rpcz_store.cc:269] 0709 13:19:12.893560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.897044 (+  3484us) service_pool.cc:224] Handling call
0709 13:19:14.338601 (+1441557us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54000db1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:14.346756 (+  8155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358850304}
I20260709 13:19:14.401901  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.402573  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.402603  6670 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:14.403755  7228 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:14.396420  6667 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:14.314173 12607 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:19:14.406651  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1360) took 1089 ms. Trace:
I20260709 13:19:14.409735  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:14.407051  6594 rpcz_store.cc:276] 0709 13:19:13.308843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.312061 (+  3218us) service_pool.cc:224] Handling call
0709 13:19:14.230722 (+918661us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b58001a0d98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:14.397980 (+167258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11868544}
I20260709 13:19:14.411458  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.415128  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.415130  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.415884  6670 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:14.416417  6590 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.418514  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.419080 12704 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:14.419237  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.419727  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.419961  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.420321  6667 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:14.420547  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.421274  6669 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:14.422180  6669 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:14.422757  6662 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:19:14.424590  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.424821  6668 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:14.420585  6663 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:19:14.428774 13058 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:19:14.429826  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.430430  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.431030  7237 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:14.434763 12998 consensus_queue.cc:237] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.439268 12607 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:14.439843 12607 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.441069 12915 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.444200 13060 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.440914 12607 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.458288  6669 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:14.449450  7237 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:14.397228 13040 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:14.450783  6665 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:14.388175 12934 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:14.451853  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.416118  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:14.462252  7236 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:14.464361  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:14.480909  6590 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:14.401408 13064 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:14.486181  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:14.486558  6671 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:14.488547  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.488572  7234 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:14.498605  7231 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:14.507745  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:14.504559  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 196247us
I20260709 13:19:14.512882  7198 leader_election.cc:400] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15.
I20260709 13:19:14.513315  7198 leader_election.cc:403] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:14.455032 12940 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:14.518468  7228 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:14.521324  6666 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:14.523720  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.415207 12958 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:19:14.525252  6669 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:14.532943  7233 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:14.538151  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 85128.5us
I20260709 13:19:14.539347  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:14.501144 13050 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:14.505472 12977 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:14.505087 12910 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:14.558346  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.544118  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.474988 12937 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:14.519111 13064 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.520790 12607 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:14.545118 12894 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:14.518323 13040 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.457608 12979 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.560204 12889 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.566046 13050 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:14.566263 12977 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:14.599540  6664 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:14.600828  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.604667  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.605763  6663 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:14.558445 12940 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.595650 12915 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.597420 12894 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:14.615710  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.637071  7232 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:14.606559 12847 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:14.649552  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:14.650766  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 109712us
I20260709 13:19:14.651759  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:14.652735 12915 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:14.656610  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:14.658110 12915 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:14.663967  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:14.665221  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:14.666061  6597 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:14.666702  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.667270  6594 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:14.667578  6522 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:14.667795  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:14.669018  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:14.669790  6592 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:14.672168  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:14.672485  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:14.672741  6515 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.673040  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:14.673491  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:14.674274  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:14.676544  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:14.677269  6522 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:14.685004  6522 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:14.685802  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.686486  6668 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:14.686481  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.687796  6664 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:14.686980  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:14.690171  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:14.691350 13068 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.691790 12915 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:14.690251  6515 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:14.692328 12915 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.693029 12915 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:14.611416 12692 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:14.700556  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:14.703365 13050 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.707813  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:14.708499  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:14.713285  6594 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:14.719048  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.645961 12989 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15.
I20260709 13:19:14.724378  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.726843  7229 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:14.722074 12948 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:14.748299  6664 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:14.749790  6663 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:19:14.718441 13082 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:14.691883 13068 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:14.752923  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:14.754289  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 338217us
I20260709 13:19:14.723692 12933 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:14.759907  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:14.707085 12963 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:19:14.747361 12948 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.707509 12847 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:14.790087  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.792222  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.719748 12681 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:19:14.816143  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:14.810439  7229 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:19:14.802527  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:14.743678 12970 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:14.821076  7232 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:19:14.768358 13059 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:14.817304  6515 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 11.
I20260709 13:19:14.767647  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:14.819171  6665 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:14.723268 12775 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:19:14.762069 12910 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:14.816238  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:14.826368  6522 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:14.831490  6522 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.831089  6518 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:19:14.822290 13034 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:14.813040 12894 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:14.851133 13075 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:19:14.851631 12940 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:14.852005 12970 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:14.853366 12910 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.855254 13043 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:14.874842  7233 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:14.849303  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 83784.2us
I20260709 13:19:14.861850 12948 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.863319  6594 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: [16.11-17.12]
I20260709 13:19:14.817976 12933 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.877403  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:14.883059  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.883674  6590 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:14.884369  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.888557  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.889178  6595 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:14.843806 12937 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:14.753502 13064 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:14.891119 13067 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:19:14.892393 13068 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:14.895260 13074 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:14.869683 13059 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:14.899750  6518 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:14.903221  6590 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:14.889621 13040 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:14.767176 12977 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.924979  7232 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:14.895454  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 447967us
I20260709 13:19:14.904527 13043 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:14.933624  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:14.899070  6522 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.947471  6590 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:14.947585  6522 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.934105  6595 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:14.949043  6595 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:14.938294  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.950193  6588 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:14.943473  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.951612  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:14.916353 13079 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:14.952280  6594 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:14.929065 12948 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:14.959345  6595 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:14.970681  6590 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:14.921408 12963 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:19:14.972411  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.925279 12891 consensus_queue.cc:1059] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:19:14.935225 12568 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:19:14.989514  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:14.987653  7236 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:14.990345  6594 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:14.994398  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:14.981945  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:14.994316  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:14.998680  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:14.998699  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:14.998855 13078 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:14.999660  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:15.000298 13050 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:15.000631 13076 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:14.995756  6595 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:15.001543 13076 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:15.004586 12748 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:15.007169 13070 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:19:15.006822 13043 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.007741 12915 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:14.948889 12972 consensus_queue.cc:1175] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "53fee9894731461a821a131be24dbbca" responder_term: 15 status { last_received { term: 15 index: 13 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:15.000798 13050 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:15.010540  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.011246  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:14.994848 13051 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.002s
I20260709 13:19:15.012440  6515 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:19:14.969923 13079 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:15.012542  6594 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:15.014696  6670 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:19:15.012217  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:15.017715  7231 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 13:19:14.982605 12934 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:15.021065  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.084914  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:15.085623  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64667.7us
I20260709 13:19:15.086524  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:15.022944  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:14.981271 13082 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:15.053824  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.087283  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64411.5us
I20260709 13:19:15.087555  6522 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:15.023466 13060 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:15.091042  6513 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:15.071280  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:15.072182 13073 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:19:15.017261 13091 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:15.110530  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:15.017738 13050 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.968406 13041 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:19:15.077661  7231 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
W20260709 13:19:15.078223  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 66074.8us
I20260709 13:19:15.120747  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.072624  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:15.022505 13072 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:19:15.079994 12891 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:15.127206  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:15.127705  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:15.125468  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:15.128489  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:15.119585  6520 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:15.056496  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:15.009971 12915 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:15.131037 13082 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.133860 12950 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:15.123641 12937 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:15.101292 12934 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.119211 13091 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:15.137952  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.139078 12972 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:15.139701  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.140633  6515 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:15.205520 12970 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:15.143308  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:15.148651  6588 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:15.152645  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:15.155494  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.156710  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.102164 13051 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:15.158476  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:15.159439  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49009us
I20260709 13:19:15.186088  6521 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:15.154719 12774 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:19:15.185909  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:15.189586  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:15.190347  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.252995  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.190904  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:15.191812  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:15.189121  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:15.190438  6666 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:15.254057  7231 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:15.254123  6664 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:15.145584 12891 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:15.140663 13079 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:14.996124 12933 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.188694 13090 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:15.213215  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.213696  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.145150 13046 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:19:15.057102 13076 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.072849 13059 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.142967  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.273664  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:15.255743  6665 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:19:15.254467  6666 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:15.127295 13089 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:15.261983  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 109442us
W20260709 13:19:15.262003  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 104535us
I20260709 13:19:15.261137 12891 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:15.306687 12889 raft_consensus.cc:3055] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:15.307281 12889 raft_consensus.cc:740] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:15.308173 12889 consensus_queue.cc:260] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.308637  7237 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:15.309098 12889 raft_consensus.cc:3060] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:15.316958  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:15.317405  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:15.319106 12774 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:15.319509 12774 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:15.266513  7231 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:15.246814 12937 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:15.337630 13051 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:15.337827 12937 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:15.338107 13051 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.251910 13091 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:15.268385  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55257.4us
I20260709 13:19:15.250053 12972 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:15.245679  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:15.236835 12704 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:15.359958  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:15.265669 12915 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:15.292109 13094 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:19:15.299180 13092 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 13:19:15.273331 13098 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:15.291057  6666 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.265556 13041 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.300643 12934 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:15.282694 13005 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:15.305912 13089 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:15.373510  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 104248us
W20260709 13:19:15.306041 12607 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.000s
I20260709 13:19:15.306589 13108 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:15.360074 12972 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:15.267011  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 141614us
I20260709 13:19:15.335572 12774 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.337188 12622 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.339512  6662 raft_consensus.cc:1240] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:19:15.361424  6664 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:15.341383  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:15.361675  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 272448us
I20260709 13:19:15.338485 13051 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:15.340870 12749 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:15.360241 12704 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:15.383095  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:15.389973  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.392982 12704 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.394134 12704 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:15.386019 12749 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:15.399585 12749 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.399904 13089 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.400980 12749 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:15.372836 12847 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:15.405556 13071 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.580s	user 0.002s	sys 0.002s
I20260709 13:19:15.410164  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:15.368065 12748 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:19:15.410661 12934 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:15.411178 12934 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.410214 12607 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:15.411535 12934 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:15.412153 12607 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:15.413028 12607 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:15.413391 12607 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:15.420526  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.421641  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:15.422310  6590 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:15.376657 13046 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:15.424486 13098 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:15.425060 13098 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.427660  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:15.431185 13098 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:15.431598  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:15.432286  6515 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:15.435149  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:15.435642  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49986.2us
W20260709 13:19:15.436379  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.438534  6517 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:15.439610 13043 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:15.414343 12607 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.440409  7235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1628) took 1028 ms. Trace:
I20260709 13:19:15.441092 12607 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:15.440088 13110 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:15.441890 13110 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:15.442870 13110 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.444159 13110 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:15.386718 12622 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.441152 13043 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:19:15.447431 12622 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:15.374123 13041 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.449882 13108 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.450928  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 61392.1us
I20260709 13:19:15.451977  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.453217  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:15.454669  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 85728.2us
I20260709 13:19:15.457651  6670 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:15.440807  7235 rpcz_store.cc:276] 0709 13:19:14.412406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.412541 (+   135us) service_pool.cc:224] Handling call
0709 13:19:14.412846 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.417949 (+  5103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.417981 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.418484 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.609058 (+190574us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:14.609111 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.609193 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.404835 (+795642us) spinlock_profiling.cc:243] Waited 7.22 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:15.436718 (+ 31883us) raft_consensus.cc:1685] finished
0709 13:19:15.436740 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.440349 (+  3609us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4953,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":90,"spinlock_wait_cycles":16616704,"thread_start_us":190182,"threads_started":1,"wal-append.queue_time_us":325544}
I20260709 13:19:15.357826 13050 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.449901 13041 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:15.454457 13108 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.466952 12934 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.467420  6666 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:15.454305 12915 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:15.473418  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.476143  6515 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:15.476320  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:15.476838  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.482906  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:15.483621  6590 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:15.471503 12915 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.504168  6515 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:19:15.505434  6663 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:19:15.505460  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:15.507730  6588 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:15.509387  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:15.509989  6517 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:15.474295 12748 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:15.536126  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.543668  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 158518us
I20260709 13:19:15.567222  6671 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:19:15.570266  6590 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:15.573109  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:15.490263 12889 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:15.490852 12913 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:15.569208  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:15.572813 12622 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.583657  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.487998 12891 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.590548  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:15.521606 13043 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.542790 13096 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.545300 12745 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:15.585825 12937 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:15.615327  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46203.7us
I20260709 13:19:15.604873 12622 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.625921  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.626834 13076 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:15.626838 13099 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:19:15.627456 13076 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:15.627514  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:15.627787 13106 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:19:15.636548  6513 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:15.632813  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.629057 12950 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:15.649235  6594 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:15.640537  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:15.629197 13076 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:15.667862  6666 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:19:15.671429  6666 pending_rounds.cc:85] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 13
I20260709 13:19:15.597330 12998 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:19:15.630240 12889 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:19:15.695039  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 173638us
I20260709 13:19:15.671765  6666 pending_rounds.cc:107] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:19:15.697736  6666 raft_consensus.cc:2889] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:15.699401  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 66713.4us
I20260709 13:19:15.617842 13043 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.678750 12692 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:15.606662  6590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:15.726541  6590 raft_consensus.cc:606] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.727087  6590 raft_consensus.cc:614] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:15.690069 12889 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:15.703356 12847 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:15.744215  6591 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:19:15.744666  6591 pending_rounds.cc:85] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 10
I20260709 13:19:15.744918  6591 pending_rounds.cc:107] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:19:15.745193  6591 raft_consensus.cc:2889] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:15.680476 13106 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.005s
I20260709 13:19:15.731987 13121 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:15.738366 12891 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:15.739862 13091 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.633772 13096 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.742101 13051 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:15.700012 13099 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:19:15.708997  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:15.705430 12894 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.012s	sys 0.028s
I20260709 13:19:15.713891 12692 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:15.715864 13041 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.071s
W20260709 13:19:15.800612  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.804260  6518 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:15.804926  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 96131.6us
I20260709 13:19:15.734216 12889 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:15.805516  6667 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:15.786155  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 240911us
I20260709 13:19:15.799144 12847 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:15.807889 12894 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.012s	sys 0.029s
I20260709 13:19:15.805922 12704 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:15.839114  6595 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:15.810019 12692 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:15.933490  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 95153.6us
I20260709 13:19:15.934829  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:15.814330 13050 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:15.799450 13121 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.954305 13115 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:15.814824 12622 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:15.965808 13117 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 13:19:15.977922 13123 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 13:19:15.981609 13116 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:19:15.967960 13115 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:15.992125 13112 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
W20260709 13:19:15.993191 13126 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:19:15.993754 12847 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:15.965591 12950 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:16.015072  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.028604  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:16.011905 12607 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.000s
I20260709 13:19:16.034986 13115 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:16.035573 13043 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.037322 12979 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:16.039510  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.015089 12937 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:16.045336  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:16.050446  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:16.043996 12950 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:16.052161 12972 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.662s	user 0.001s	sys 0.010s
I20260709 13:19:16.063457  6592 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:19:16.056881 13110 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.004s
I20260709 13:19:16.075656  6592 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:16.055083 12937 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:16.079031  6596 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: [16.11-17.12]
I20260709 13:19:16.080552  6597 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:16.059422 12704 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:16.082914 12934 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:16.083624 13125 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:19:16.057514 12622 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:19:16.087375 13121 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.087913 13051 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:16.088999  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.089689 12889 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.061757 13091 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.095896  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2941) took 1160 ms. Trace:
I20260709 13:19:16.096308  6518 rpcz_store.cc:276] 0709 13:19:14.935032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.935356 (+   324us) service_pool.cc:224] Handling call
0709 13:19:14.962435 (+ 27079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:15.811851 (+849416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.095118 (+283267us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b540040bda0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b9084a02 00007f04b9080dd6 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:19:16.095634 (+   516us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b540007e4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:16.095841 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940392960}
I20260709 13:19:16.097990  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2962) took 1373 ms. Trace:
I20260709 13:19:16.098374  6667 rpcz_store.cc:276] 0709 13:19:14.725084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.725207 (+   123us) service_pool.cc:224] Handling call
0709 13:19:14.743435 (+ 18228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:15.812643 (+1069208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.097722 (+285079us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000c58d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:16.097925 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2433539456}
W20260709 13:19:16.101929  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.104383 12889 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.105484  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:16.106011  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:16.106884  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:16.072476 12972 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.107667  6521 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:16.108744  6593 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:16.109047 12972 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.109436  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:16.110688 13114 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.111263 13114 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.113116  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:16.114917  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 340795us
I20260709 13:19:16.115146 13081 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.115370 12989 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:16.115962  7200 leader_election.cc:400] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19.
I20260709 13:19:16.115679 13081 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:16.116261  7200 leader_election.cc:403] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:19:16.116413 12989 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:16.116822 12989 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:16.117681 13081 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:16.117827  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 304624us
I20260709 13:19:16.118067 13088 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19.
I20260709 13:19:16.117653 12989 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.119084 13099 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:16.119889 12989 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:16.123411  7235 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:16.124080 13099 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:16.124481 13099 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:16.126128 13045 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:16.125651 13099 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.127341 13099 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.129576 12989 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:16.108647  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:16.117672  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:16.147212  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:16.152092  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:16.161165 12972 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:16.161722 12972 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:16.162187 12972 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:16.161535  6663 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:16.163144  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:16.164376  6591 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:16.164539  6597 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:16.120092 13136 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.161309  6666 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:16.130112 12989 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.165980  6666 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:16.172902  7228 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:16.174089  6667 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:16.114145 13114 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.094128 13076 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.179497  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:16.182308  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:16.183375  6518 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:19:16.183458  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65852.8us
I20260709 13:19:16.180997  6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835ac063488a40878ccb02b662d3898b"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:19:16.184337  6515 raft_consensus.cc:606] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:16.184671  6515 raft_consensus.cc:614] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:16.149991 12748 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:16.182699  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.182175  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:16.188436  6665 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:16.189113  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:16.191636  7229 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:16.165511 13136 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.189776  6670 raft_consensus.cc:2376] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18. Re-sending same reply.
I20260709 13:19:16.206496  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:16.206800  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:16.177750 13110 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:16.264050  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 57334.6us
I20260709 13:19:16.279349  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:16.182261 12970 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:16.284452  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:16.284648  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:16.285522  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:16.285900  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.286586  6591 raft_consensus.cc:2410] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 21 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:16.142657 13045 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.292546  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:16.293025  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:16.293193  6595 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:16.200129 13096 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.207245 12894 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:16.301059  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 67257us
I20260709 13:19:16.185243 13043 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 13:19:16.309181  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.311738  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:16.328819  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:16.330935  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:16.284015 12998 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:16.339517 12692 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:16.340342  7235 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:16.340761 12989 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:16.342031 13060 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.343189 13060 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.344887 13045 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:16.347748  6594 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:16.336501 12894 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:16.312517 13121 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:16.287505 12655 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:19:16.329376 13091 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.300714 13110 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:16.209586 12748 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:16.313457 13071 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:16.392961  7228 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:16.303423  6670 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:16.370908 13059 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:19:16.399554  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:16.400220 13060 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.400420 12692 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:16.377322 13045 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:16.375293 13064 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:19:16.401443 12681 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:16.405650  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:16.310979 12493 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:19:16.329710 12970 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:16.422456  7237 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:16.310952 13005 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:19:16.376555 12745 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:19:16.429811 13138 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:19:16.431135 12894 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.297451 13046 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 13:19:16.377043 13121 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.419165 13091 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.447610 12681 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:16.442289  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 134061us
W20260709 13:19:16.451733  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 172103us
I20260709 13:19:16.452700 12692 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.454185  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:16.469758  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 135721us
W20260709 13:19:16.469990  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.469861  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.473888  7235 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:16.458174 13040 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:16.463645 13121 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:16.481515  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 152804us
I20260709 13:19:16.464560 13045 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:16.472003 12975 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.501961  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:16.507472  7233 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:16.515573 12970 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:16.525619  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2966) took 1598 ms. Trace:
W20260709 13:19:16.528146  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.526088  6671 rpcz_store.cc:276] 0709 13:19:14.927152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.927583 (+   431us) service_pool.cc:224] Handling call
0709 13:19:14.942828 (+ 15245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:15.567717 (+624889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.525379 (+957662us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54001900d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:16.525561 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435325312}
I20260709 13:19:16.529704 12681 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.530176  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:16.530918 12681 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.531507  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:16.535228  6591 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:19:16.538734  6671 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:16.539687  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:16.496456 13076 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.475920 12692 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.554342 12704 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:16.547032  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:16.491470 13145 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:19:16.477051 13040 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.567207  6589 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:19:16.526594 13136 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:16.499071 13060 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:19:16.569250  6519 raft_consensus.cc:1275] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:19:16.580360  6519 pending_rounds.cc:85] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 13
I20260709 13:19:16.580605  6519 pending_rounds.cc:107] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:19:16.504061 13068 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:19:16.580830  6519 raft_consensus.cc:2889] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.581303 13040 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.525918 13064 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:19:16.583452 13091 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.583855 12915 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:16.585112  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:16.575008 12972 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.579231 12748 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:16.509362 13142 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:19:16.591130  6515 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:16.592324  6522 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:16.593331  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:16.609863  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:16.502574 13045 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.599505  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.476524 13110 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.624794  6516 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:16.625876  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:16.626770  6522 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:16.605425  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:16.611250  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:16.629305  6596 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:16.509765 12975 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.633270 12975 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:16.615062  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:16.620172  6590 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:16.585736 13108 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:16.602344 12894 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:16.604285  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.608971 13050 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:16.647627  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.648723  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.648150  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.655902  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:16.662097  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.664028  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:16.664836  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:16.665160  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:16.666251  6522 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:19:16.663582 13089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.045s	sys 0.057s
I20260709 13:19:16.627172 13142 consensus_queue.cc:1243] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:16.642949 12774 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:16.608439 13121 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:16.698258  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 197766us
I20260709 13:19:16.654191 13050 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:16.706925  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:16.708065  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 93120.4us
W20260709 13:19:16.698444 12979 consensus_meta.cc:287] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.646s	user 0.006s	sys 0.005s
W20260709 13:19:16.700605 13129 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:19:16.712168  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:16.718822 12970 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:16.722363 13150 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:19:16.727464  6515 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:19:16.698697 13089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.046s	sys 0.057s
I20260709 13:19:16.704569 13005 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:19:16.750144  7232 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:16.753516  6519 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:16.757642  6590 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:16.760246  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48149.9us
I20260709 13:19:16.761265  7234 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:16.813275  6521 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:16.762354  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 134353us
I20260709 13:19:16.785979  6588 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:19:16.709285 12704 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:16.851200  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:16.725368 13050 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:16.924280  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.763935 12940 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:16.701634 13107 consensus_queue.cc:1048] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 13:19:16.833220  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:16.923460  6513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1266) took 1819 ms. Trace:
I20260709 13:19:16.824255 12975 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.813041 12681 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.824497 13136 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:16.826470 12748 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:16.708416 13099 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.124s
I20260709 13:19:16.946848  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:16.932940  6521 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:19:16.958231  6521 pending_rounds.cc:85] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 13
I20260709 13:19:16.958562  6521 pending_rounds.cc:107] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:19:16.958899  6521 raft_consensus.cc:2889] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.933141  6513 rpcz_store.cc:276] 0709 13:19:15.104224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.104406 (+   182us) service_pool.cc:224] Handling call
0709 13:19:15.104689 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:15.644572 (+539883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.831936 (+1187364us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400354b54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:16.923394 (+ 91458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222675712}
W20260709 13:19:16.763198  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 177882us
I20260709 13:19:16.823748  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:16.961746 13155 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 13:19:16.962297 13154 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 13:19:17.005069  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:17.006196  6664 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
W20260709 13:19:17.012476 13158 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:19:17.014914  6663 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:17.030810 12937 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.031420 12937 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.032109  6521 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:16.930292 13162 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.037727  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:16.932304 12940 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:17.070989 13040 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:17.071957 12989 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:16.956781 13136 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.075083 13136 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.072522 12989 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:16.944583 12681 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.085705 12681 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.068856  6514 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:16.938194 12975 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.087136  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.087785  6519 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:17.090240 12975 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.069656  6517 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:19:17.037029 13122 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:19:17.060521 13068 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:17.071130 12940 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:17.093606 13050 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.060284 12972 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.067605 13045 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.033579 12937 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:17.081980  6487 leader_election.cc:336] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.585s (SENT)
I20260709 13:19:17.082930  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:17.083487 13121 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.100196 12704 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:17.100606 12989 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.100626 13121 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.101863  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.114413 12972 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:17.115789  6487 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.651s (SENT)
I20260709 13:19:17.102188 13050 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.105760 12940 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.107331  6662 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:17.201898 12940 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:17.209357  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.078841 13162 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.215879  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:17.193571 12937 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.217785  7236 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:17.216113  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:17.219767  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:17.221302  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.243211  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.243927  6517 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
W20260709 13:19:17.247397  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 423684us
I20260709 13:19:17.249632  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.240409 13050 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.255812  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:17.252027  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:17.259022  6517 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:17.223838 12937 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.262290  6516 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:17.263671  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:17.271371  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:17.276127  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:17.261384 13127 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.261586 12975 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.307228  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:17.308672  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.305580  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:17.315582  7235 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:17.266074 12704 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:17.320571  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:17.322710  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.323534  6592 raft_consensus.cc:2410] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:19:17.283524 12622 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:17.285085 12972 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.326857  7237 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:17.267134 13050 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.328439  6595 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:17.334041  6517 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
W20260709 13:19:17.327610  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 209377us
I20260709 13:19:17.340829  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.342008  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
W20260709 13:19:17.342144  6487 leader_election.cc:336] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.835s (SENT)
I20260709 13:19:17.355198  6591 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:17.363163  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.315202 13127 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.371531 12970 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.372112  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.372771 12970 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.330580 12704 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:17.318079 12975 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.379443  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.352874 12889 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.380472 13169 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:19:17.412982  6595 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:17.417392 12975 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.419143 13127 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.419989 12891 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:17.423729  7234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1798) took 2444 ms (client timeout 2043 ms). Trace:
W20260709 13:19:17.424201  7234 rpcz_store.cc:269] 0709 13:19:14.979068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.979179 (+   111us) service_pool.cc:224] Handling call
0709 13:19:17.423508 (+2444329us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400576a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:17.423624 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4014495488}
I20260709 13:19:17.426048  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.426412  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.426792  6517 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:17.426982  6516 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:17.437466  6516 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:17.446534 12975 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:17.447724 12975 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:17.448330  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:17.449182 12975 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:17.449771  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:17.450088 13107 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:17.451319 13170 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:17.451720 13170 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:17.451959 13107 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:17.452440 13107 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.453454 13179 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:17.452857 13170 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.454030 13171 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:17.454552 13171 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:17.456172 13136 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:17.455646 13171 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.456589 13136 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:17.457288 13171 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.457566 13136 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.459470 13136 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.452813 13107 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:17.466411 13143 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:17.467056 13143 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.469309 13143 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.471925 13165 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:17.472458 13165 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:17.475677  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 512988us
W20260709 13:19:17.475926  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:17.476765  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.462061  6666 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.338207 12972 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.384413 13059 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.485427 13045 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.487823 12847 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.506083 12847 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.506675 12972 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.507750 12937 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.508564 13162 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.508920 12847 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.509631 13045 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.510079 13050 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.512207 13045 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.514182 12977 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:17.514657 12977 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:17.516727  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2569) took 1346 ms. Trace:
I20260709 13:19:17.517117  6671 rpcz_store.cc:276] 0709 13:19:16.170179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.540646 (+370467us) service_pool.cc:224] Handling call
0709 13:19:16.540896 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.541087 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:16.926746 (+385659us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.516381 (+589635us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.516439 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:17.516483 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.516553 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.516669 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":336124,"prepare.queue_time_us":393554,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":41,"raft.queue_time_us":268232,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"thread_start_us":592057,"threads_started":2}
W20260709 13:19:17.520040 13175 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 13:19:17.518417  6668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2554) took 2373 ms (client timeout 2043 ms). Trace:
I20260709 13:19:17.521461 13115 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.521992 13115 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.523264 13114 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:17.524091 13115 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:17.525270 13142 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:17.525388 13089 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:17.521258  6668 rpcz_store.cc:269] 0709 13:19:15.144996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.145278 (+   282us) service_pool.cc:224] Handling call
0709 13:19:17.518204 (+2372926us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400160154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:17.518335 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4026163968}
I20260709 13:19:17.523406 13172 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.526923 13172 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:17.528313 13124 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.486s	user 0.007s	sys 0.009s
I20260709 13:19:17.529155 13172 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.529961 13107 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.532050 13107 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.525389  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2562) took 1356 ms. Trace:
I20260709 13:19:17.536479  6662 rpcz_store.cc:276] 0709 13:19:16.168602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.405363 (+236761us) service_pool.cc:224] Handling call
0709 13:19:17.525211 (+1119848us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54000cba54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:17.525332 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515698944}
I20260709 13:19:17.540479 13059 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.542760 12704 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:17.543074 13059 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.543903 12704 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.522301 12894 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:19:17.549687  6666 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:17.568383 13165 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.569870 12692 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.569896 12970 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.576121 12934 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:17.576562 12934 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:17.577309 12934 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.577333  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2879) took 1433 ms. Trace:
I20260709 13:19:17.580085  6594 rpcz_store.cc:276] 0709 13:19:16.144065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.153747 (+  9682us) service_pool.cc:224] Handling call
0709 13:19:16.180676 (+ 26929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.348762 (+168086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.577121 (+1228359us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002fb7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:17.577280 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380874112}
I20260709 13:19:17.584074 12977 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.585618 12977 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:17.488655  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:17.590027  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 101525us
I20260709 13:19:17.593958 13127 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:17.594481 13127 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.594914 13127 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:17.591063 13121 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.607640 13121 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.610090 13121 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.611915 12692 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:17.612623 12692 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:17.500223  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.591041  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:17.454064 13179 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:17.626024  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.632241  6594 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:17.631029 12989 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.643894  6666 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:17.646484 12692 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.646173 12989 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.646963 13069 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.647598  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:17.647449 13069 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.648159  6518 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:17.648438 12989 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:17.649564 13069 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.634718 12998 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:17.657366  6518 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:17.657613 12692 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:17.486881 12889 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.691670  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.692240  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.693053  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:17.690497  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 311251us
I20260709 13:19:17.693857  6517 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:17.623752 13121 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:17.699967  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.701121  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2985) took 1555 ms. Trace:
I20260709 13:19:17.701648  6666 rpcz_store.cc:276] 0709 13:19:16.145904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.146078 (+   174us) service_pool.cc:224] Handling call
0709 13:19:16.161178 (+ 15100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.461654 (+1300476us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54005607a0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fff06f 00007f04b9077f60 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62
0709 13:19:17.645807 (+184153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.701060 (+ 55253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308527744}
I20260709 13:19:17.658737 12998 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.700698  6589 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:17.672864 13081 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.714257  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.728382 13147 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.635046 12940 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 13:19:17.691164 13005 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.709957 13045 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.741534  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:17.756304  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:17.758100  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:17.730284 12989 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:17.746081  6520 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.048s	sys 0.087s
I20260709 13:19:17.763171  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:17.712544 12975 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.730013 13107 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.735631 13165 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.712677 13136 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.769004  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:17.737751 13103 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.709970 12692 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.739511 12998 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:17.780462  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:17.780730  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:17.743624 13099 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:17.739801 13147 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.783566 13103 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.743391 12681 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.750036 13005 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.769788 13078 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:17.737910  6589 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:17.742314  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.712792 13121 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.787465  6671 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
W20260709 13:19:17.805790  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 63542.2us
I20260709 13:19:17.782377  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:17.789951  6521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.043s	sys 0.021s
I20260709 13:19:17.784698 12940 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:17.792135  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:17.767131 13091 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:17.792963  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:17.794902  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:17.738881 13081 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.793540 12871 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:17.811476 12568 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:17.785156 13165 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.816138 13081 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:17.789746 13136 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.817977 13165 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:17.819571 13136 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:17.820341  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:17.806921  6521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.043s	sys 0.022s
I20260709 13:19:17.807852  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.821411 13068 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:17.805663 13078 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.822938 13078 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:17.823161  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.813838 13147 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.824962  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:17.812445  6597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.009s	sys 0.013s
W20260709 13:19:17.826581  6597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.009s	sys 0.013s
I20260709 13:19:17.824270 13192 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:17.805677 13108 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:17.837599  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.779879 13107 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.761972 13045 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.773644 12989 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.839186  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2989) took 1344 ms. Trace:
I20260709 13:19:17.785877 12975 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.809371 13114 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.809516 13115 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.830134 12940 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:17.826713 13089 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:17.846798  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 370407us
I20260709 13:19:17.795341 13099 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.846357  6487 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:17.848542  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.838804 13064 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.837967 13091 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:17.834872 12847 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:19:17.881954 13091 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.868556  6589 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:17.836083 13163 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:17.805889 12681 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.887723 12681 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.879521  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:17.891907 13064 raft_consensus.cc:2764] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:19:17.892831 13099 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.892957  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.893903 13064 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.840307 13108 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:17.895308  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.895342 13108 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.844132  6664 rpcz_store.cc:276] 0709 13:19:16.495050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.007157 (+512107us) service_pool.cc:224] Handling call
0709 13:19:17.007539 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.007781 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:17.008530 (+   749us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.838838 (+830308us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.838895 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:17.838943 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.839011 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.839129 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":469237,"prepare.queue_time_us":263,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":507391,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":359977,"threads_started":1}
I20260709 13:19:17.903116  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:17.911332  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.899420  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.850108 13170 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.913050  6589 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:17.897357 12745 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.851517 13192 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.838115 13165 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 13:19:17.938158  7229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1657) took 1629 ms (client timeout 2014 ms). Trace:
W20260709 13:19:17.949466  7229 rpcz_store.cc:269] 0709 13:19:16.308547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.308804 (+   257us) service_pool.cc:224] Handling call
0709 13:19:17.937933 (+1629129us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540078e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:17.938074 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3524453248}
W20260709 13:19:17.919173  6562 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.003s	sys 0.020s
I20260709 13:19:17.923767 13176 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.924908 12692 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.945158 12975 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.927506 12889 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.852461 13114 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.928412  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:17.929420 13163 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.929922 13050 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:17.930097 12950 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:17.932292  6589 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:19:17.933904  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.935410  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.936667  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:17.788690 13179 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:17.920164 13108 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:17.951421  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:17.999648  6593 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:17.948452  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.003062  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.941515  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:18.004509  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 63120.1us
W20260709 13:19:17.951215  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56003.2us
I20260709 13:19:18.010800 12704 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:18.011220 12704 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.011584 12704 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:18.014906 12937 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:17.975108  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.017944  6520 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:18.018846  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:18.019601  6520 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:18.020352  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:18.021306  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.022214  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.875020 13115 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:17.977584  6562 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.004s	sys 0.020s
I20260709 13:19:17.937465 13107 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:17.938687 13121 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:18.023160  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 767828us
I20260709 13:19:18.024550 13115 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:17.926048 13005 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.028930  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.029634  7235 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:17.981673  6519 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:18.030306  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.030771  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.031168  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.031409  7232 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:18.031985  6519 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:18.032819  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.033816  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.034816  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:17.950909 13165 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:17.943369 13103 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:17.983642  6515 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:17.920034 12915 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:19:17.965029 13090 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:17.985265  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.962803 12847 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:17.987862  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:17.990185  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 248708us
I20260709 13:19:18.056938  6516 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:17.992225  6595 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:18.057539  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:17.996848  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 68534.5us
I20260709 13:19:17.939740 12989 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.031214  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.038331  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:18.037112  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.040220  7233 raft_consensus.cc:2410] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:19:18.044665  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:17.990103 13163 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:18.036536  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:18.044906  6488 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:18.051488  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.051946  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:18.043767 13064 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:18.033782 12894 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:18.062251  7229 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
W20260709 13:19:18.276676  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 214967us
I20260709 13:19:18.058969  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:18.277838  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 218947us
I20260709 13:19:17.997571 13050 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.034807 12950 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:17.984946 13176 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.047664 13165 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.045928 13182 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 13:19:17.998160 12940 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.105086 13107 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:18.052970 12972 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 13:19:18.254091  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 190136us
I20260709 13:19:18.263231 13076 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:18.254252  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.048633 12607 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
W20260709 13:19:18.254985  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.049396 12774 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.261806  6516 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:18.300828 12975 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:18.266419  6513 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:18.075563 12847 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:18.270649  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 225777us
I20260709 13:19:18.064616 13090 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.271580  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.264150  6589 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
W20260709 13:19:18.283380  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.278477  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:18.284425  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:18.291159  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.421962  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 143561us
W20260709 13:19:18.276077 13193 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.213s	user 0.003s	sys 0.007s
I20260709 13:19:18.103446  6520 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:18.268944 13127 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.673s	user 0.006s	sys 0.011s
I20260709 13:19:18.277199 13162 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:18.433722  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.330778  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.329857  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.332580  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.436178  7231 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:18.437062  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.289243 13051 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:18.435206  7232 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:18.435676  7233 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:18.263047 13103 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:19:18.261183 13180 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.449141  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:18.284055 12889 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:18.300661 13107 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:18.380501  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.264199  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 227708us
W20260709 13:19:18.410599  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.426988  6515 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:18.422868  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:18.383919 12745 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:19:18.432330  6519 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:18.432750  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.351768  7234 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:18.445273  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.432698 13121 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:18.449826  7232 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:18.367920 13089 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 13:19:18.380199 12774 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.433352 13162 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.386454 13045 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.471135 12889 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:18.430754  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.492398 13190 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.447s	user 0.000s	sys 0.001s
I20260709 13:19:18.485864  6519 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85490b85caca40d19a25af108b1da46a"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:19:18.510586  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:18.553306  6519 raft_consensus.cc:606] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:18.553723  6519 raft_consensus.cc:614] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:18.440022 13105 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.487080  6520 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:18.487814  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.555485  6515 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:18.447623  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.440294 13051 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:18.477905 13107 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:18.339902 13076 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.426360 13148 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:18.418661 12975 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.584554  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2890) took 1824 ms. Trace:
I20260709 13:19:18.595130  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.439361 13127 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.372635 13196 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.211s
W20260709 13:19:18.542176 13202 log.cc:927] Time spent T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.251s	user 0.001s	sys 0.001s
I20260709 13:19:18.552500 12889 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:18.550033 12607 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:18.563234 13105 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.568748 12692 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:18.643430 12934 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:18.541523 13176 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:18.574484 12998 consensus_meta.cc:287] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.779s	user 0.006s	sys 0.002s
I20260709 13:19:18.644721 13105 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.644676 12998 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.477371 13180 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.647053 12998 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.647647 13180 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.550225 13114 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:18.533695 13195 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:18.552285 13192 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:18.649343 13192 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:18.650068 13078 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.827s	user 0.007s	sys 0.002s
I20260709 13:19:18.549650 13191 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:18.482362 13099 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 13:19:18.607828  6597 rpcz_store.cc:276] 0709 13:19:16.759939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.760113 (+   174us) service_pool.cc:224] Handling call
0709 13:19:16.813257 (+ 53144us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.243328 (+430071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.243376 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.248764 (+  5388us) log.cc:844] Serialized 26 byte log entry
0709 13:19:17.827045 (+578281us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:17.827090 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.827192 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.584342 (+757150us) raft_consensus.cc:1685] finished
0709 13:19:18.584363 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.584494 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11457,"raft.queue_time_us":141418,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":963236,"threads_started":2,"wal-append.queue_time_us":717469}
W20260709 13:19:18.602792  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 92465.1us
W20260709 13:19:18.564101 13204 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.433s	user 0.001s	sys 0.000s
I20260709 13:19:18.561309 13051 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:18.583956 13090 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.622414 13076 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:18.622717 13198 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:19:18.624315  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 336890us
I20260709 13:19:18.602991 13208 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:18.670820 13208 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.670962  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:18.671254 13208 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:18.672286  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.673015  6519 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:18.626055 13209 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:19:18.676235  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.676965  6517 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:18.678200  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.636507 13127 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.638101 12975 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.638583  7235 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:18.639287 12607 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.641067 12774 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:18.643501 12692 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:18.575758 12847 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:18.582594 13171 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.125s	user 0.012s	sys 0.002s
I20260709 13:19:18.542668 13162 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:18.589021 13108 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.000s
W20260709 13:19:18.614116 12704 consensus_meta.cc:287] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.003s
I20260709 13:19:18.658483 13192 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.603752 13165 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.662951  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:18.665804 13076 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:18.666760 13176 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:18.630231 12889 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:18.666752 13205 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.091s	user 0.003s	sys 0.000s
I20260709 13:19:18.691514  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2891) took 1929 ms. Trace:
I20260709 13:19:18.650609 13078 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:18.759922  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 68955us
I20260709 13:19:18.794409 13045 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:18.767931 13115 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.768438 13196 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:18.768885 12915 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:18.769313 13103 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.770634 13127 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:18.772958  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:18.774703  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:18.614940 13148 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:18.692265 13163 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.000s
I20260709 13:19:18.807689  6592 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:18.852667  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:18.853253  6666 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:18.656191 13195 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.796602 12692 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:18.693264 13050 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.858635  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:18.859751  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:18.807188  7236 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:18.651094 12950 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:18.817510 13203 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.234s	user 0.000s	sys 0.002s
I20260709 13:19:18.820845  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:18.818315  6596 rpcz_store.cc:276] 0709 13:19:16.761713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.761910 (+   197us) service_pool.cc:224] Handling call
0709 13:19:16.762348 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:16.762753 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:16.762779 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:16.815484 (+ 52705us) log.cc:844] Serialized 26 byte log entry
0709 13:19:17.549145 (+733661us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b18004bb740. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:19:17.963242 (+414097us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:17.963299 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.963390 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.677507 (+714117us) raft_consensus.cc:1685] finished
0709 13:19:18.677528 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.691410 (+ 13882us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53378,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":1930,"spinlock_wait_cycles":2693376,"thread_start_us":413878,"threads_started":1,"wal-append.queue_time_us":1668450}
I20260709 13:19:18.688433 12998 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.120s
W20260709 13:19:18.823351  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 160443us
I20260709 13:19:18.880656  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:19:18.880779  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 457957us
I20260709 13:19:18.824260  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:18.698194 13005 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.117s
I20260709 13:19:18.661010 13191 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.853302  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:18.853448  6592 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 12: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:18.855346 13163 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.885902 12607 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.886052 13195 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:18.886678 13163 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.886533 13090 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:18.809558 12704 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.887276 12977 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:18.888155  6592 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:18.888453 13078 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.889137 12704 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:18.849154 13196 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.889725 12704 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:18.852469 13103 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.890141 13005 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:18.890645 13005 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:18.890970 13196 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:18.850889 13091 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:18.901548 13005 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.903241 13005 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:18.903861 12950 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:18.905545 13163 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:18.907495 12704 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:18.909013 13050 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:18.909449 13050 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:18.808771 13208 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:18.909582  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.847466 13115 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.918708  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:18.875929  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 477543us
W20260709 13:19:18.919461  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 95281.8us
I20260709 13:19:18.923223  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:18.920145  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:18.927629  6670 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:18.929761  6596 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:18.817880 13076 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.820081 13176 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.931134  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.875481 13099 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.889678 12998 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:18.847666 13045 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:18.918670 13211 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:19:18.947456  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:18.949111 13103 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:18.953570 12692 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.956511 13076 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.958643 13176 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:18.900288 13091 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.851228 12915 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:18.855430 13127 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.963467  6670 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:18.966254 12979 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:18.910544 13050 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.967988 13051 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.969976  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:18.970211  6521 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:18.970932 12937 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:18.971518  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48418.6us
I20260709 13:19:19.029039  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.005848 13060 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:19:18.973218 13005 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.008173 13099 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:18.980880 13191 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:19.024132 12979 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.982462 12889 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:19.080354  6590 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:19.081467  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:18.928526 12989 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:18.987226 13090 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:19:18.988531  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 107929us
I20260709 13:19:18.988984  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3009) took 1334 ms. Trace:
I20260709 13:19:18.990049 13148 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:19.028535 12937 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:19.001669  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:19.003654  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.004047  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 59361.3us
I20260709 13:19:19.005689  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:19.005524 12998 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:19.027501  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.105762  6670 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:19:19.106863  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.107513  6670 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:19.108377  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:19.108914  6670 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.030685  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:18.971740  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.018705 13091 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.113588  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.032135  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:19.032768  7231 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.028s	sys 0.079s
I20260709 13:19:19.033094  6593 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:19.023761 13121 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:19.129447  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 98854.8us
I20260709 13:19:19.050697 13176 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:19.093132  6591 rpcz_store.cc:276] 0709 13:19:17.654632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.654771 (+   139us) service_pool.cc:224] Handling call
0709 13:19:17.655144 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:17.655565 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.655602 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:17.656289 (+   687us) log.cc:844] Serialized 52 byte log entry
0709 13:19:17.977182 (+320893us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:18.476812 (+499630us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.482182 (+  5370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.482278 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.625005 (+142727us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4800061ed8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f 00007f04be098f37 00007f04be0ad55c 00007f04be0ad508 00007f04be0ad488 00007f04be0ad443 00007f04be0ac6ec 00007f04bcfd6a96
0709 13:19:18.816861 (+191856us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:18.988674 (+171813us) raft_consensus.cc:1685] finished
0709 13:19:18.988709 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.988906 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5033,"prepare.queue_time_us":564061,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":10684,"raft.run_cpu_time_us":1597,"raft.run_wall_time_us":139102,"spinlock_wait_cycles":111024384,"thread_start_us":740030,"threads_started":2,"wal-append.queue_time_us":926457}
I20260709 13:19:19.130529  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:19.047788 13182 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:18.933542 13172 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:19.027504  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3014) took 1323 ms. Trace:
I20260709 13:19:19.071017  7228 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:19.076273  6560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.026s	sys 0.034s
W20260709 13:19:19.050966 13170 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.129s	user 0.010s	sys 0.002s
W20260709 13:19:19.140194  6560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.026s	sys 0.035s
I20260709 13:19:19.094985  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1833) took 1065 ms. Trace:
I20260709 13:19:19.082755 13114 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.000180 13115 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:19.093441  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.115551  6592 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:19.116526  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 110949us
I20260709 13:19:19.117676  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.088526 12979 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.144536  6593 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:19.141057  7236 rpcz_store.cc:276] 0709 13:19:18.029888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.030005 (+   117us) service_pool.cc:224] Handling call
0709 13:19:19.094797 (+1064792us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54006cb6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:19.094922 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785112960}
W20260709 13:19:19.117841  7231 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.029s	sys 0.079s
I20260709 13:19:19.107116  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.125644  6670 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:19.114588  6665 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:19.150043  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.128852 13041 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:19.102383 13050 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.151705  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.129434  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:19.036705 13212 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.462s	user 0.005s	sys 0.006s
I20260709 13:19:19.131453  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.093327 13090 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:19.139810  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 2974) took 1131 ms. Trace:
I20260709 13:19:19.161129  6562 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.161195  6521 rpcz_store.cc:276] 0709 13:19:17.986577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.002815 (+ 16238us) service_pool.cc:224] Handling call
0709 13:19:19.118286 (+1115471us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54003f70d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:19.118389 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222684032}
I20260709 13:19:19.153780 13231 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:19.139524  6588 rpcz_store.cc:276] 0709 13:19:17.703995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.704135 (+   140us) service_pool.cc:224] Handling call
0709 13:19:17.732966 (+ 28831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.733165 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:17.986401 (+253236us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.490417 (+504016us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.604595 (+114178us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b1800445c00. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:19.027008 (+422413us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.027081 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:19.027145 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.027330 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.027440 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62974,"prepare.queue_time_us":794434,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":90,"raft.queue_time_us":144612,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":32656,"spinlock_wait_cycles":55137152,"thread_start_us":1202319,"threads_started":3,"wal-append.queue_time_us":808340}
I20260709 13:19:19.157099  7230 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:19.160096  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.162289  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:19.164043  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.118132 12937 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:19.164875  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.167860  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.172245  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:19.155268 12692 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:19.084623 13005 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:19.189306  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 269202us
I20260709 13:19:19.177110  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.179144  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.179385  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.192380  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:19.192950  6519 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:19.194732  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:19.196961  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.198020  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.199230  6592 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:19.199126  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.142417 13176 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:19.139153 13182 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:19.174172  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.182646  6514 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:19.165664 13171 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.221369  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.182436  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:19.187268  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.188293  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:19.159118 13107 consensus_meta.cc:287] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.000s
W20260709 13:19:19.204269  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.204190  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.205145  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:19.178915 13060 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:19.208288  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.211506  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:19.255368  6666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2575) took 1502 ms (client timeout 1658 ms). Trace:
I20260709 13:19:19.206318  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.275455  7234 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:19.276616  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.277549  7234 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:19.177035 13172 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:19.244998  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:19.246179  7237 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:19.281035  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.281590  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.282409  7237 raft_consensus.cc:2410] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:19.282876  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:19.284873  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.285996  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.248558 13182 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:19.249517 13005 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:19.249696  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.288568  6592 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:19.250370 12934 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:19.250871  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:19.296209  7230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1834) took 1266 ms (client timeout 1532 ms). Trace:
I20260709 13:19:19.296708  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:19.250423 13171 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.251447  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:19.253207 13107 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.253693  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:19.210364 13176 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:19.254480 13045 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:19.255352 13210 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.671s	user 0.000s	sys 0.014s
I20260709 13:19:19.254830 13090 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.317840  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:19.257685 13127 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:19.176510 12998 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.258279  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.330230  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.274022 13060 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:19.259112  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 69115.1us
I20260709 13:19:19.337567  7235 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16.
I20260709 13:19:19.259682  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:19.259845  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2969) took 1281 ms. Trace:
W20260709 13:19:19.338478  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 78855us
I20260709 13:19:19.260157 13064 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:19.261065 13208 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:19.262367 13213 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.677s	user 0.002s	sys 0.009s
I20260709 13:19:19.346930  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.263428  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.254338  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.267303  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:19.348626  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 81427.7us
I20260709 13:19:19.270690  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1829) took 1241 ms. Trace:
I20260709 13:19:19.322706 13180 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.286517  7237 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:19.360949  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:19.290833  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.291776  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:19.292687  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:19.298560  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.301697  6593 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:19.257700 13194 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.858s	user 0.014s	sys 0.008s
I20260709 13:19:19.359010 13196 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:19.375517  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:19.375919 13196 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:19.376411 13196 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:19.376804  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.377478  7237 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:19.340734 13090 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:19.296746  7230 rpcz_store.cc:269] 0709 13:19:18.030179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.036832 (+  6653us) service_pool.cc:224] Handling call
0709 13:19:19.296004 (+1259172us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54006a3bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:19.296132 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2516227968}
I20260709 13:19:19.218606 12975 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.307901  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:19.225014 13231 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.323482  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.327445  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:19.326524  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.389689  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:19.258585  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.331040  6520 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:19.328956  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:19.338317  6522 rpcz_store.cc:276] 0709 13:19:17.978567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.979399 (+   832us) service_pool.cc:224] Handling call
0709 13:19:17.979720 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.990047 (+ 10327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.990074 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.994891 (+  4817us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.223694 (+228803us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:18.223753 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.223851 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.036102 (+812251us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:19.259531 (+223429us) raft_consensus.cc:1685] finished
0709 13:19:19.259556 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.259785 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40855,"raft.queue_time_us":6630,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":45295,"spinlock_wait_cycles":73088768,"thread_start_us":228432,"threads_started":1,"wal-append.queue_time_us":579785}
I20260709 13:19:19.341444  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:19.347100  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.262205  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.391853  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.392362  7233 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:19:19.350728  7228 rpcz_store.cc:276] 0709 13:19:18.028777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.035221 (+  6444us) service_pool.cc:224] Handling call
0709 13:19:18.035491 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.145601 (+1110110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.270442 (+124841us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400259754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:19.270608 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380878080}
W20260709 13:19:19.275121  6666 rpcz_store.cc:269] 0709 13:19:17.753019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.768750 (+ 15731us) service_pool.cc:224] Handling call
0709 13:19:19.255174 (+1486424us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000e65d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:19.255304 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2471306112}
I20260709 13:19:19.395126  7230 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:19.321116 13078 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:19.318790 13096 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:19:19.363858  7234 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:19.364580  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.402458  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.295480 12889 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:19:19.367867  7236 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:19.344398 13005 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.378289  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:19.379371  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 87303.4us
I20260709 13:19:19.221225 13076 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:19.385617  6593 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:19.414413  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.415086  6594 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:19.391718  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.362164  7232 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
W20260709 13:19:19.398092  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51066.8us
W20260709 13:19:19.399143  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 91314.5us
I20260709 13:19:19.417135  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:19.417294  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:19.399868  6522 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:19.303079 13121 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.408078  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1670) took 1026 ms. Trace:
I20260709 13:19:19.424569  7231 rpcz_store.cc:276] 0709 13:19:18.381334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.479070 (+ 97736us) service_pool.cc:224] Handling call
0709 13:19:18.479347 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.479553 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:19.147687 (+668134us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.407676 (+259989us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.407741 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:19.407799 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.407880 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.408017 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20271,"prepare.queue_time_us":676309,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":21954,"spinlock_wait_cycles":41216,"thread_start_us":866774,"threads_started":2}
W20260709 13:19:19.408847  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 79991.6us
I20260709 13:19:19.366354 13180 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:19.415954  6517 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:19.420169  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.420042  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.394858  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.396173 13226 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:19.421113 13238 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:19.422123  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.421121  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.427266  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.407342 13195 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:19.421074  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.401563 13078 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:19.433041  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:19.453493  6588 raft_consensus.cc:606] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.435128 13180 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.435436  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.435753  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.457083  6522 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:19.414307 13060 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:19.437050  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:19.437525 13230 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:19:19.468933  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.422556 12607 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:19.438946  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.439296 13114 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:19.404152 12975 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.442947 13165 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:19.408356 13152 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:19.430945  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2970) took 1450 ms. Trace:
I20260709 13:19:19.401635 12915 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:19.414252 13076 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:19.415803 13005 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:19.402010 13231 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:19.471104  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:19.745450 12937 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.001s
I20260709 13:19:19.747223  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:19.670900 13239 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:19.747612  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 320461us
I20260709 13:19:19.748624 13239 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:19.684478 13227 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.361s	user 0.002s	sys 0.000s
I20260709 13:19:19.452353 13195 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.751109 13195 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:19.457651 12568 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:19.710554  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.751157 13114 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.755631  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:19.756842  7229 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:19.471906 13233 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:19:19.757207 13236 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:19.757691 13236 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.757864 12975 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:19.758044 13236 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:19.751678  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 331572us
I20260709 13:19:19.538012  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:19.751547  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.752053  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:19.507398 13099 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:19.746450  6518 rpcz_store.cc:276] 0709 13:19:17.980669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.980834 (+   165us) service_pool.cc:224] Handling call
0709 13:19:17.981153 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.989998 (+  8845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.990028 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.998849 (+  8821us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.382912 (+384063us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:18.382969 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.383081 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.587379 (+204298us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b4800062958. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f 00007f04be098f37 00007f04be0ad55c 00007f04be0ad508 00007f04be0ad488 00007f04be0ad443 00007f04be0ac6ec 00007f04bcfd6a96
0709 13:19:19.261759 (+674380us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:19.430668 (+168909us) raft_consensus.cc:1685] finished
0709 13:19:19.430698 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.430874 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8448,"raft.queue_time_us":8981,"raft.run_cpu_time_us":6920,"raft.run_wall_time_us":596454,"spinlock_wait_cycles":142127744,"thread_start_us":383668,"threads_started":1,"wal-append.queue_time_us":535549}
I20260709 13:19:19.751252 13005 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.467355 12889 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.768481  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:19.453789 13238 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:19.521739 12989 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:19.760934  6671 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:19.750141  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:19.745735 13107 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:19.739369 13249 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:19.451511 13226 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:19.745880 13247 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
I20260709 13:19:19.756688  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:19.825378  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 68758us
I20260709 13:19:19.763332 13239 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:19.457458 13078 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:19.766783 13076 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:19.785260  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 368069us
W20260709 13:19:19.785918  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 386383us
W20260709 13:19:19.799925  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51276.4us
I20260709 13:19:19.803864  6667 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: [16.14-17.15]
W20260709 13:19:19.814507  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64432.6us
I20260709 13:19:19.456465 13127 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:19.843928  6519 consensus_meta.cc:287] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.000s
I20260709 13:19:19.844301  6519 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:19.845155  6515 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:19:19.770794 13060 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:19.768781 12889 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:19.765134 13152 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:19.840662  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:19.745967 12937 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.764605 13231 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:19.782773 13238 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.861403 13260 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:19.861915 13242 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
W20260709 13:19:19.862550 13244 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
W20260709 13:19:19.863241 13232 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 13:19:19.864627  7200 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.585s (SENT) [suppressed 25 similar messages]
I20260709 13:19:19.866739 13192 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:19.867750 12607 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:19.868268 13252 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:19:19.869227 13246 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:19:19.869639 13105 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:19.833987 13226 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.878146  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:19.832540 13099 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:19.838550 12975 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:19:19.844702 13162 consensus_meta.cc:287] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.031s	user 0.011s	sys 0.001s
I20260709 13:19:19.896706  7234 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 13:19:19.884639 12979 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.001s
I20260709 13:19:19.901615  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:19.881249 13152 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:19.904302  6592 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:19.935180 12975 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.910545  7235 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: [15.13-15.13]
W20260709 13:19:19.913151  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 153855us
I20260709 13:19:19.914767  6668 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:19.851468 12915 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:19:19.892757 12889 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.920171  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.923795  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.883339 13103 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:19:19.940112 12979 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:19.959606 12979 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:19.941780 12975 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:19.947683  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.963456  7235 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:19.938594  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 2995) took 1172 ms. Trace:
W20260709 13:19:19.947692  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 69690.3us
I20260709 13:19:19.967638  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:19.969229  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:19:19.953076  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.940009 13050 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.000s
W20260709 13:19:19.970587  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.971799  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:19.964778  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:19.907874 13192 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:19.950722  7228 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:19.983691  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:19.986028  7228 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:19.922443 13105 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:19.965404  6663 rpcz_store.cc:276] 0709 13:19:18.765935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.775207 (+  9272us) service_pool.cc:224] Handling call
0709 13:19:18.775545 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.891649 (+116104us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:19.217400 (+325751us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.468202 (+250802us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.468264 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:19.468321 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.938389 (+470068us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54000cc954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:19.938431 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.938540 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":179645,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":2898,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":19177,"spinlock_wait_cycles":266513280,"thread_start_us":535561,"threads_started":2,"wal-append.queue_time_us":665117}
W20260709 13:19:19.936357 13176 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.005s
W20260709 13:19:19.999501  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 119731us
W20260709 13:19:20.000476  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.000723  6516 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:19.948387 13269 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.947700 13257 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:19.976635 12568 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:19.976327 12607 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:19.949097 13265 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:19.987144 12979 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:20.019254  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 154647us
I20260709 13:19:19.955742 13162 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.022497  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:20.029065 12937 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.018512 12607 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:20.022687  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 50962.4us
I20260709 13:19:20.030598 12607 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:20.038379 13103 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:20.038458 13217 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:20.039335 13103 consensus_queue.cc:1243] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:20.039850 13103 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:20.040205 13103 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:19.983534 13076 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.009835 13257 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:19.989271 13114 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:19.989555 13195 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.050768  6595 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20. Ops: [19.10-19.10]
W20260709 13:19:20.043562 13196 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.000s
I20260709 13:19:20.007040 13176 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.021557 12568 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:20.047976 13257 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:20.045768 13170 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:20.068500  6591 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:20.069638  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59200.3us
I20260709 13:19:20.071202  6666 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:20.052182 13275 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:20.050643 13265 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:20.070852  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.011242 13269 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.032622 12979 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.076330 13270 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:20.075183 13275 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:20.077520 13265 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:20.068262 13196 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.035944 13192 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.085865 13263 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.041076 13103 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.081830 13170 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:20.057137 13105 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.092705 13051 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.091943 13270 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:20.124137 13091 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.012s	user 0.008s	sys 0.004s
I20260709 13:19:20.124751 13152 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:20.125615 13090 consensus_meta.cc:287] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.724s	user 0.007s	sys 0.005s
I20260709 13:19:20.233790 13114 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:20.095997 13278 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:20.234088 13176 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.234565 13278 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.236073 12979 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.237167 13278 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.238029 13195 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.238250  6665 raft_consensus.cc:2410] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:20.099073 13265 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:20.115200 13263 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.242704 13236 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.243230 13103 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:20.244927 13236 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:20.244998 13263 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:20.121773 13180 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.640s	user 0.004s	sys 0.004s
I20260709 13:19:20.245820 13170 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.246070 13180 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.246995 13091 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.248418 13180 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.129194 12607 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.249680 13091 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.250515  6669 raft_consensus.cc:2368] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19.
I20260709 13:19:20.250972 12607 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.250666 13270 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.129995 13078 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.131426 13162 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.252003 13270 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:20.253451 13078 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.255110  6516 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:20.256242  6671 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:20.257628 12975 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.141444 13226 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.155045 13275 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.265301 13275 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:20.161463 12568 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.162389 13277 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:20.162521 12998 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:20.078179 12889 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:20.112694 13259 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:19:20.194370  6662 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:20.232602 13076 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.232913 13269 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.199062 13105 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:20.233521 13152 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:20.321756  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 248282us
W20260709 13:19:20.293753  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 260939us
I20260709 13:19:20.314003 13226 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.331293  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.293212 13078 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:20.385360  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.342242  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:20.389780  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.414383  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 72264.9us
I20260709 13:19:20.265321 13257 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:20.409060 13245 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.750s	user 0.008s	sys 0.003s
W20260709 13:19:20.319118 13005 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.000s
I20260709 13:19:20.415580  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:20.317044 13236 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:20.399083 13152 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.426595  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:20.427722  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.417153 13103 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:20.436064  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:20.436877  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:20.439337  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:20.440685  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.437539  7229 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:20.358528 13091 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 13:19:20.425439 13005 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.445479  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:20.435119 13238 consensus_meta.cc:287] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.003s
W20260709 13:19:20.443660  6487 leader_election.cc:336] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.668s (SENT)
I20260709 13:19:20.427050 13226 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.293332 13277 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:20.451556  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.456307  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:20.454288 13256 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:20.474068  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.507040  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:20.598497  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.598961  7233 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:20.599082  6514 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:20.600014  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:20.600746  6522 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:19:20.600833  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 157192us
I20260709 13:19:20.431578 13263 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:20.509811  6519 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:20.437777 13280 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:20.438118 13103 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:20.510708  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65859.9us
I20260709 13:19:20.511031  7236 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:20.608462 13103 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:20.609246  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.519387 12889 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.610685 13241 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.508162 13238 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.611370  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:20.611204 13241 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.612223  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:20.537654  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2589) took 1434 ms. Trace:
I20260709 13:19:20.612802 13238 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:20.533651 13283 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:19:20.612942  6671 rpcz_store.cc:276] 0709 13:19:19.103458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.830833 (+727375us) service_pool.cc:224] Handling call
0709 13:19:19.831278 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.256698 (+425420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.537347 (+280649us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54000cb554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:20.537578 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976796032}
I20260709 13:19:20.557904 13196 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:20.442138 13152 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.615411  6518 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:20.581501 12975 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:20.589438 13235 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.214s	user 0.013s	sys 0.019s
I20260709 13:19:20.610940  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:20.416615 13078 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.433297 13271 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:20.431912 13278 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:20.594552  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:20.628908  6514 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:20.605306 13265 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.606041  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:20.606763  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1249) took 1490 ms. Trace:
I20260709 13:19:20.633816  6669 rpcz_store.cc:276] 0709 13:19:19.116670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.167531 (+ 50861us) service_pool.cc:224] Handling call
0709 13:19:20.606556 (+1439025us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000cba54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:20.606708 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2472094080}
I20260709 13:19:20.630527 13069 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:20.607921 13171 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:20.612293  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.613124 13241 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.604192 13263 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.658548  6668 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:20.435590 13236 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.658394  6518 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
W20260709 13:19:20.662396 13273 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.725s	user 0.000s	sys 0.010s
I20260709 13:19:20.663069 12889 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:20.663605 13182 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:20.667035  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 61074.4us
I20260709 13:19:20.631263 13271 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:20.672134  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.672712  6519 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:20.671159 13271 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:20.659332 13278 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.675359  6513 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:20.675870 13278 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.677420 13152 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.677767 13257 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:20.678828 13243 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.300s	user 0.007s	sys 0.004s
I20260709 13:19:20.678651  7229 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:20.671713 13078 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:20.685801  6519 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:20.689958  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.692498  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3006) took 1700 ms. Trace:
I20260709 13:19:20.692945  6662 rpcz_store.cc:276] 0709 13:19:18.991971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.759274 (+767303us) service_pool.cc:224] Handling call
0709 13:19:19.759554 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.312931 (+553377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.692269 (+379338us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54000e5bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:20.692445 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999360000}
I20260709 13:19:20.699611  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2915) took 1446 ms. Trace:
I20260709 13:19:20.694435  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1243) took 1700 ms. Trace:
I20260709 13:19:20.702085  6666 rpcz_store.cc:276] 0709 13:19:18.994430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.468658 (+474228us) service_pool.cc:224] Handling call
0709 13:19:20.694261 (+1225603us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b540005a854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:20.694383 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382967168}
I20260709 13:19:20.704463  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:20.694415  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:20.709131  6667 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:20.700109  6590 rpcz_store.cc:276] 0709 13:19:19.252978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.365495 (+112517us) service_pool.cc:224] Handling call
0709 13:19:19.365796 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.365989 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:19.366221 (+   232us) log.cc:844] Serialized 14 byte log entry
0709 13:19:20.699024 (+1332803us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b18001d7be0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:19:20.699203 (+   179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:20.699264 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:20.699322 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.699408 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.699542 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13731,"prepare.queue_time_us":12875,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"spinlock_wait_cycles":19149568}
I20260709 13:19:20.711038  6522 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
W20260709 13:19:20.712322  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 1.37083e+06us
I20260709 13:19:20.714797  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:20.714947  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1247) took 1663 ms. Trace:
W20260709 13:19:20.715608  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.715652  6665 rpcz_store.cc:276] 0709 13:19:19.051434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.470859 (+419425us) service_pool.cc:224] Handling call
0709 13:19:20.709078 (+1238219us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54001900d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:20.714886 (+  5808us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131372416}
I20260709 13:19:20.712968  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:20.719522  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.720045  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.721135  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:20.722002  7232 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:20.722155  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:20.723078  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:20.723683  7228 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:20.724118  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:20.725422  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:20.630144 13226 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.728547  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:20.732906  7236 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:19:20.733462  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 1.40598e+06us
I20260709 13:19:20.708720 13286 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.732933  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:20.744763  7231 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:20.724651  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:20.736467  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.736830  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:20.734274  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3022) took 1661 ms. Trace:
I20260709 13:19:20.738929  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.739728  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 2985) took 1543 ms. Trace:
I20260709 13:19:20.740557  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2907) took 1945 ms. Trace:
W20260709 13:19:20.764827  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 43763.5us
I20260709 13:19:20.768963  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:20.769059  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44499.2us
I20260709 13:19:20.766408  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.770725 13208 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:20.771775  6665 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:20.723112 13041 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.773730  6515 rpcz_store.cc:276] 0709 13:19:19.196191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.249926 (+ 53735us) service_pool.cc:224] Handling call
0709 13:19:19.844882 (+594956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:19.845690 (+   808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.739545 (+893855us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54003cc150. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:20.739674 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367464704}
I20260709 13:19:20.776487  7233 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:20.745158  6562 connection.cc:256] RPC call timeout handler was delayed by -1.3403781360000266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:19:20.787642  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:20.737366 12975 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:20.791550  6664 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:19:20.791955  6664 pending_rounds.cc:85] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 13
I20260709 13:19:20.792224  6664 pending_rounds.cc:107] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:19:20.792518  6664 raft_consensus.cc:2889] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:20.788523  6662 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:20.787479  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.789767  6517 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:20.773291  6595 rpcz_store.cc:276] 0709 13:19:18.795307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.795698 (+   391us) service_pool.cc:224] Handling call
0709 13:19:18.796030 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:20.051425 (+1255395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.740327 (+688902us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002f62d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:20.740502 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2884725504}
I20260709 13:19:20.787047 13223 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:20.732124 13261 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:20.773561 13291 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:20.833796  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 120895us
I20260709 13:19:20.860178  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:20.783670 13068 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:20.788292 13248 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:20.790570 12950 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:20.871137  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.871698  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.871208  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.797719 12975 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:20.849632  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.799944  6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:19:20.878219 13076 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:20.862649 13219 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:20.893767  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:20.791678 12568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.019s	sys 0.019s
I20260709 13:19:20.831605 13096 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:20.790475 13208 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 13:19:20.879536 13304 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:20.880333  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 87334.4us
I20260709 13:19:20.851833 13191 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:20.795149 13005 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.775413 13270 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:20.775063 13286 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.928772  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.930011  6662 raft_consensus.cc:2410] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:20.801822  6522 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:20.863384 13241 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:20.769387  6589 rpcz_store.cc:276] 0709 13:19:19.072485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.075095 (+  2610us) service_pool.cc:224] Handling call
0709 13:19:19.098766 (+ 23671us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:19.101768 (+  3002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.101834 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:19.118201 (+ 16367us) log.cc:844] Serialized 52 byte log entry
0709 13:19:19.315826 (+197625us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b180042ba40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:19:19.409050 (+ 93224us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:19.421985 (+ 12935us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.867550 (+445565us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:20.733960 (+866410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.734033 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:20.734071 (+    38us) raft_consensus.cc:1685] finished
0709 13:19:20.734084 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.734221 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1333136,"prepare.queue_time_us":1199470,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":3547,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":24048,"spinlock_wait_cycles":16430208,"thread_start_us":93039,"threads_started":1,"wal-append.queue_time_us":492554}
I20260709 13:19:20.866086 12889 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.915779 12568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.019s	sys 0.019s
I20260709 13:19:20.866402  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.890535  6515 raft_consensus.cc:606] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:20.879238  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:20.883786  7228 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:20.989923  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.888505  7231 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:20.789793 13041 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:20.886778 12998 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 13:19:20.921420  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.896035 12975 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:21.000634  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:20.893090  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:20.888052 12950 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:21.003283  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.003604 12950 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.004997  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.926426 13304 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.006491  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.007642 13304 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:20.934657 12889 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.008088  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.009663 12889 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.935585 13191 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.916404 13068 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:21.010565 13068 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.011687 13191 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.013518 13068 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.916851 13096 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.014158 13041 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:21.014443 13096 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:21.016348  6521 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:21.017590  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.018132  6521 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:20.995317 13241 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:21.019729 13241 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.020700  6416 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:20.995781 12692 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.023114 12692 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.025190 12692 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:20.995667 13219 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:21.026058 13219 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:21.028242 13276 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.903s	user 0.005s	sys 0.008s
I20260709 13:19:21.028723 13219 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.995908 13076 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:21.029557 13076 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.031519 13076 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:21.032786  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 153639us
I20260709 13:19:21.033542  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:21.036414 12704 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:21.037133 12704 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:21.038093  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 145199us
I20260709 13:19:20.996129 12977 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:19:21.038913  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:21.039191 13163 raft_consensus.cc:993] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:19:21.039762 13165 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:20.761255 13278 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:20.947079 13277 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.869025 13291 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.956921  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:20.905224 13223 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:20.957633 13265 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.959154 13236 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:20.959637 13270 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:20.961298 13226 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.875078 13248 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:20.963979 13286 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:20.928217 13105 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.001s
I20260709 13:19:20.975167 13103 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:20.976082 13208 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:20.922389 13261 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:20.922657 13290 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.980911 13192 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:20.985057  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:20.991132  6516 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:20.992501  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:20.992002  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:20.993857  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:21.005903  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.081772  6522 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 12.
I20260709 13:19:21.039814 13238 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.958413 13263 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:20.938131  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.043891  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.047586  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:21.040329 13165 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:21.050160  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 321654us
I20260709 13:19:21.050379  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:21.085076  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:21.085330  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:19:21.050611  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 266445us
I20260709 13:19:21.052464  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:21.047861  6416 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 11, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 11 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.065969 13180 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:21.070452  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 73626.4us
I20260709 13:19:21.042493 13278 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.051584 13076 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:21.084026  6662 raft_consensus.cc:2410] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:19:21.088970  6562 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:19:21.106784  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54383us
I20260709 13:19:21.111836  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:21.048204 12977 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:19:21.118733  6666 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:21.048521 13041 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:19:21.117172  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:21.117421  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 132496us
I20260709 13:19:21.118757  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.114262  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:21.063301 13257 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:19:21.049948 13078 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:19:21.047402 13280 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:19:21.129179  6670 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:21.078167 12692 raft_consensus.cc:1081] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Signalling peer 53fee9894731461a821a131be24dbbca to start an election
I20260709 13:19:21.075572 13208 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:21.121385  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:21.075371 13226 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:19:21.083154 13238 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.118505 13005 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:21.115685 13163 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:21.113853 13275 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:21.113546 13278 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.203007  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.119904 13060 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:21.113425 13121 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:21.208597  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 91362us
I20260709 13:19:21.131080 13219 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:21.218405  6664 raft_consensus.cc:2368] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19.
I20260709 13:19:21.113291 13076 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:21.118688 13304 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.086s
W20260709 13:19:21.228232  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 106909us
I20260709 13:19:21.221509  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.075876 13223 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.225932  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:21.221421  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 109735us
I20260709 13:19:21.122679 13195 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 13:19:21.122843 13162 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:19:21.292687  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.221760 13298 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:21.232929  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.290680  6520 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:21.230059  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 141779us
I20260709 13:19:21.230252  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.325156 13103 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.224330 13121 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:21.228886 13041 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:21.325987 13121 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:21.217713 13270 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.326956  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.328107  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.329092  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.203209 13236 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:19:21.225404 13005 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:21.229601 12950 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.214978 13163 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:21.117015 13261 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.352604 13261 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.122136 12998 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.098s
W20260709 13:19:21.331427  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 217228us
I20260709 13:19:21.356881  6516 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:21.224406 13219 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.115540 13290 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.332278  6670 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:19:21.277940 13182 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
I20260709 13:19:21.220083 13170 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:21.228848 13076 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:21.227272 12704 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:21.343998  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.400264  6664 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:21.279201 13275 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:21.343497 13103 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:21.348580 13005 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.349617 12950 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.350397 13163 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:21.331736 13298 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:21.395844  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71813us
I20260709 13:19:21.394984  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.411808  6670 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:21.401867  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:21.412719  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 122038us
I20260709 13:19:21.413875  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:21.415406  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.414234  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.406558  6595 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:19:21.420356  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:21.420884  6595 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.408020  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.410341  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:21.388967 13219 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:21.379688 13051 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:21.424317  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.426061  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.428679  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.442999  6592 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:21.411892 13305 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:19:21.403589 12704 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:21.406268 13269 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.403810 13275 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:21.409459 13103 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.323704 13208 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.450416  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:21.450400  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.451507  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.417093 12977 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:21.458544 12977 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:21.459472 12977 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.460803 12977 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:21.461860 13231 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:21.462266 13231 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:21.455470  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:21.409390 13261 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:21.463107 13231 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.463300 13261 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.464145  6595 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:21.464393 13231 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.449731  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.465991  6594 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:21.417565 13291 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:21.425028 12692 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.416843 13298 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.498603  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:21.487717  6591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1402) took 2399 ms (client timeout 2113 ms). Trace:
W20260709 13:19:21.500339  6591 rpcz_store.cc:269] 0709 13:19:19.075844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.280748 (+204904us) service_pool.cc:224] Handling call
0709 13:19:21.474235 (+2193487us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b5400210f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:21.475123 (+   888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1809099648}
I20260709 13:19:21.429354 13313 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.429627 13176 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:21.443506 13315 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.502023  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:21.476579 13262 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.517s	user 0.000s	sys 0.012s
I20260709 13:19:21.457201 13103 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.494813 13041 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.506834  6664 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:21.506768  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.426659 13304 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:19:21.531554  6597 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:21.533114  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.498485 13162 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:21.522495 12889 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.541749  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:21.542373  6597 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.446254 13051 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.457288 13270 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:19:21.455790 13269 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.446939 12975 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.506806 12692 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.551035 13051 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.555357 13182 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:21.555822  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2590) took 2452 ms. Trace:
I20260709 13:19:21.612179  6667 rpcz_store.cc:276] 0709 13:19:19.103770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.846902 (+743132us) service_pool.cc:224] Handling call
0709 13:19:20.694502 (+847600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.709534 (+ 15032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:21.555628 (+846094us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b540005a850. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:21.555765 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1868112640}
I20260709 13:19:21.556804 13256 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:21.557384  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.557968  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.558753 12889 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.559608  6522 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:21.613919  6521 raft_consensus.cc:2410] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:21.558789  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.560591 13170 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:21.563297  6588 raft_consensus.cc:2410] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:19:21.565049 13261 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:21.566150 13306 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:19:21.568783 13307 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 13:19:21.573560  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.574898  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.579636 13226 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:21.580627 13300 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.508s	user 0.000s	sys 0.002s
I20260709 13:19:21.665122  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.665973  6521 raft_consensus.cc:2368] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
I20260709 13:19:21.511731 13313 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.536181 13041 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.511251 13115 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.586915 12704 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.517345 13176 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.588402  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.523201 13315 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.589769  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.590911  6669 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:21.625031 13170 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:21.597623 13271 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.924s	user 0.009s	sys 0.000s
I20260709 13:19:21.510030 13248 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:21.589919  6597 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:21.514287 13208 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:21.601975  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 103425us
I20260709 13:19:21.504354 13316 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:21.606003  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.613729  6514 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
W20260709 13:19:21.626924  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.634817  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.613660 13324 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:21.581817  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.535858 13312 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:21.616197 13256 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:21.599411 13162 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.692967 13050 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.698055  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.698763  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3019) took 1068 ms. Trace:
I20260709 13:19:21.700728 13176 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:21.701764  6591 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:21.702477 13280 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:21.704102 13041 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:21.707742  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1689) took 1012 ms. Trace:
I20260709 13:19:21.708635 13315 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.616544 13182 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.748222  7234 rpcz_store.cc:276] 0709 13:19:20.695759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.695900 (+   141us) service_pool.cc:224] Handling call
0709 13:19:20.696387 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:20.696663 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:20.696700 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:20.697317 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 13:19:20.910349 (+213032us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:20.913451 (+  3102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.913611 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:21.579946 (+666335us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:21.707441 (+127495us) raft_consensus.cc:1685] finished
0709 13:19:21.707466 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:21.707682 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8115968,"thread_start_us":181797,"threads_started":1,"wal-append.queue_time_us":361383}
I20260709 13:19:21.698841 13115 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:21.711989 13308 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 13:19:21.533072 13275 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.641433 13321 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:21.724006  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.724172  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.761752  6597 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:21.724877  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.725844  7198 leader_election.cc:400] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19.
I20260709 13:19:21.763111  6515 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
W20260709 13:19:21.763432  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44436.8us
I20260709 13:19:21.729115  6519 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
W20260709 13:19:21.733196  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.765239  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:21.738286  6594 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:21.738193  6669 rpcz_store.cc:276] 0709 13:19:20.630757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.634782 (+  4025us) service_pool.cc:224] Handling call
0709 13:19:21.590493 (+955711us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:19:21.590712 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:21.698505 (+107793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:21.698712 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31072000}
W20260709 13:19:21.730552 13241 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.016s	sys 0.002s
I20260709 13:19:21.719514 13316 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.719858 13223 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:21.749928  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 428244us
I20260709 13:19:21.773308  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.745204  6563 leader_election.cc:400] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
I20260709 13:19:21.774605  6563 leader_election.cc:403] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:21.691872 13226 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.760947  6589 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:21.774382  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:21.735072  6635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.000s	sys 0.008s
W20260709 13:19:21.776863  6635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.000s	sys 0.008s
I20260709 13:19:21.747062 13280 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:21.777362  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.777496  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:21.771642  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:21.726118 13271 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.776449  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.749449 12937 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:19:21.783457  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:21.778762  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:21.779824  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:21.772631 13241 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.017s	sys 0.002s
I20260709 13:19:21.788342  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.788672  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.793090  6594 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:21.785480  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.781872  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.786283  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.794700  6664 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:21.765674 13277 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.772718 13005 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:21.746078 13050 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.804647  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:21.796543  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:21.798029  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.839478  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:21.767292 13315 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:21.769445 13290 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.795706  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.842952  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.844182  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:21.844789  7237 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:21.816618  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.838666  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:21.772533 13060 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:19:21.781579 13256 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.846443  7231 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:21.803090  6668 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:21.784003 13324 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:21.847631  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:21.848496  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:21.835548  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.001223  7228 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:21.826432  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:21.851244 13328 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:22.015429  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:21.858891  7237 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:21.860291 13313 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.021019  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:21.860916 12692 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.862965 13115 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:21.864928 13226 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:21.866807 13091 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:21.868270  7200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.021s	sys 0.032s
I20260709 13:19:21.959241  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:22.020344 13231 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.001s
I20260709 13:19:21.858114 13103 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:21.828414  7235 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:21.967250  6667 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:22.053387  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:22.053988  6667 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:22.054162  7234 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:21.785918 13223 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:21.772818 13316 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:21.962769  6662 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:21.784226 13041 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
I20260709 13:19:21.953437  6589 raft_consensus.cc:2376] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 18. Re-sending same reply.
I20260709 13:19:21.995138  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.016657 12975 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:22.060300  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.013437  7228 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:22.015172  7232 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:21.959390 13334 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.099s	user 0.002s	sys 0.001s
W20260709 13:19:21.956861 13163 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.002s
I20260709 13:19:21.790714 13217 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.172s
I20260709 13:19:22.052484  6665 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d"
dest_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45444
I20260709 13:19:22.027841  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.075273  6671 raft_consensus.cc:2368] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13.
I20260709 13:19:22.035009  7231 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:22.036052  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:22.035995  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.767535 13170 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.823966 13277 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:21.795807 13248 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:22.043913  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 52037.4us
I20260709 13:19:22.047305  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:22.047958  7200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.022s	sys 0.032s
I20260709 13:19:22.085336  6517 raft_consensus.cc:2410] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:22.007385  6666 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:21.957605 13280 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.063499  7230 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:22.067152  6662 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:22.033151 13324 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:22.071651  6667 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:21.949986 12977 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.078482  6521 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:22.056639 13223 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.057768  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.101398  7234 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.057356 12889 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.086537  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.086835  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.087230  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:22.087283  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:21.950261 13315 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.090794  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.093415  7228 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:22.099095  6665 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:22.114661  6665 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.102049  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:22.115734  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:22.116302  6666 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:22.117024  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:22.117506  6666 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:22.104326  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.110692  6594 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:19:22.113193  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.118191  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:22.124531  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.125686  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.129177  7200 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.132870  7230 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:22.135520  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48349.9us
I20260709 13:19:22.136337  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:22.137454  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:22.140349 13315 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.141693 13325 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:22.143949 13115 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:22.144375 13115 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:22.145226 13115 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.131925 13326 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:22.080052 13163 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.138249 13324 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.091570 13248 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.111313 12977 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.105036 13182 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.146559 13115 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:22.185040  6668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1272) took 1447 ms (client timeout 1885 ms). Trace:
I20260709 13:19:22.173756 13041 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:22.187179  6664 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:22.131832 13121 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.805s	user 0.008s	sys 0.004s
I20260709 13:19:22.167917 13050 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:22.147275 13257 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:22.188616 13121 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.189852 13257 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:22.191011  6595 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:22.158824 12975 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:22.147800 13332 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:22.194229 13332 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:22.195495 12704 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.159262 13103 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:22.169118  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:22.199321  6514 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:22.199640  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.200767  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.201349  6591 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:22.202147  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.202307 12977 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.155540 13238 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:22.203192 13298 consensus_meta.cc:287] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.005s
I20260709 13:19:22.203619 13298 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.205626 13298 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:22.159040 13330 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.162803 13162 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:22.209182  7236 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:19:22.209565  7236 pending_rounds.cc:85] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 14
I20260709 13:19:22.209766  7236 pending_rounds.cc:107] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:19:22.209789 13275 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.210197  7236 raft_consensus.cc:2889] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:22.165349 13271 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.210863  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:22.147622 13337 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:19:22.211499  6592 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:22.161695  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.212378  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.213076  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:22.213737  6671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 3017) took 1597 ms (client timeout 2124 ms). Trace:
W20260709 13:19:22.214144  6671 rpcz_store.cc:269] 0709 13:19:20.616053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.616157 (+   104us) service_pool.cc:224] Handling call
0709 13:19:21.573402 (+957245us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:22.213551 (+640149us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54000fee54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:22.213681 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990500608}
I20260709 13:19:22.207861  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.106210 13223 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.214214  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.136497  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.253156  7233 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:22.213106  6592 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.234213  6591 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:22.181447  6662 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:19:22.256567  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 119165us
I20260709 13:19:22.235476  7234 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:22.222220  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.258199  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.259871  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.222407  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:22.185845  6668 rpcz_store.cc:269] 0709 13:19:20.737405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.738702 (+  1297us) service_pool.cc:224] Handling call
0709 13:19:22.184541 (+1445839us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540005afd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:22.184916 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2367618432}
W20260709 13:19:22.261317  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 43784.3us
I20260709 13:19:22.262760  6562 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:22.265589  6590 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.266510  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.267459  6591 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.267856  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.267941  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:22.224597  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.269145  7234 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:22.265810  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.271415  6592 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:22.272719  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:22.180940 13096 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.166s	user 0.003s	sys 0.009s
I20260709 13:19:22.122709 13312 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:22.278321  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.291637  7230 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:22.186344 13163 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.280236  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:22.282238  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.283910  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.294486  6592 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.286012  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.295101  7235 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:22.272603 13208 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:22.298655 13208 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:22.163643 12950 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.299551 13208 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.287915  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.264961 13248 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.301710 13096 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:22.290966  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 154525us
W20260709 13:19:22.273526  6406 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b75fb632ebbc415092b08b38d0835144 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 64b95375974b48d18c596a23c9cac75c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:22.293891  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.284646  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.304126  7234 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:22.305929  6592 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:22.307780  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.303702  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:22.316190  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.316393  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.316455  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.316991  7233 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.317384  6591 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:19:22.307083  6665 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.321581  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.277575 13231 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:22.328660  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:22.329524  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.324365  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.324710  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.326736  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:22.330639  6591 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:22.331861 12950 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:22.332413 12950 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:22.333254 12950 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.326766 13221 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:22.335963 13096 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.336966  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:22.338142 13163 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:22.338982 13050 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:22.338824 13041 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:22.339758 13041 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:22.340087 13041 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:22.340878 13041 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.342012 13041 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:22.343027  6520 raft_consensus.cc:2368] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:22.343290 13121 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.302758 13313 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:22.344009 13313 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.344365 13313 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:22.345104  6590 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 14: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.347462  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:22.349675  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.196276 13332 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.350497  6563 leader_election.cc:400] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:22.350822  6563 leader_election.cc:403] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:19:22.356204  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.357128  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.358294  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.363963  7233 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:22.365049  6520 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:22.358881  7230 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:22.318974 13180 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:22.342017 13278 consensus_meta.cc:287] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.114s	user 0.003s	sys 0.008s
I20260709 13:19:22.321177 13051 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:22.387640  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.372073  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.192682 13257 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.394019  7231 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:22.381606  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.398945  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:22.400130  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:22.401276  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:22.356470 13076 consensus_meta.cc:287] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.947s	user 0.011s	sys 0.000s
I20260709 13:19:22.407778  6591 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.410125  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.402995  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:22.404711  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:22.416736  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:22.365891 13121 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.356750 13231 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:19:22.384474 13336 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:19:22.435447  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 170394us
I20260709 13:19:22.388238 13170 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.402254 13051 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.409145 13182 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.431826 13330 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:22.426564  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.409090 12889 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:22.445237  7233 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.454062 13050 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:22.455019 13269 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:22.458735 13219 consensus_meta.cc:287] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 1.011s	user 0.013s	sys 0.000s
I20260709 13:19:22.507373  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.407327 13278 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.459748  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.465068  6592 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:22.465384  6419 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 334) took 17945 ms (client timeout 15000 ms). Trace:
I20260709 13:19:22.466740 13277 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.425000 13266 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:22.467061 13280 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.468189  6416 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:22.471130  6487 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:22.424942 13076 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.473973  6513 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:22.439268  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:22.491796  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.539600  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:22.429080  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.490833  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.546708  6518 raft_consensus.cc:2368] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20.
I20260709 13:19:22.546905  7234 raft_consensus.cc:2410] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:22.435396 13231 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:22.552480  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.553129  6590 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:22.521481  7233 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:22.444056 13051 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.534116  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.535807 12568 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:22.472954  6594 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:19:22.548382  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 119404us
I20260709 13:19:22.560006  6596 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:22.561343  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:22.560456  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.484225 13182 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:22.484246  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 346974us
I20260709 13:19:22.377674 13165 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:22.534554  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 63503.4us
I20260709 13:19:22.570452  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.570806  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:22.533917 13266 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.551394  6416 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.544111 13277 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:22.582791  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:22.583878  6515 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.512260 13219 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.590862  6416 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 13 to 16, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 16 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:22.555490 13231 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.555225 13330 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:22.580529  6590 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:22.587963  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:22.593111  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:22.593523  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.600051  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.599877  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:22.604324 13221 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:22.605520 13221 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:22.605778  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 166726us
I20260709 13:19:22.612902  7228 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:22.615078  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1417) took 1194 ms. Trace:
I20260709 13:19:22.615511  6590 rpcz_store.cc:276] 0709 13:19:21.420203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.564168 (+1143965us) service_pool.cc:224] Handling call
0709 13:19:22.615005 (+ 50837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:22.558940 12568 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:22.563093 13331 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.622272  6416 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 13, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 13 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:22.624902  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51264.5us
I20260709 13:19:22.628053  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.628454  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:22.629802  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.631539 13271 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:22.632643 13271 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:22.633249 13271 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:19:22.510922  6419 rpcz_store.cc:269] 0709 13:19:04.520057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.520282 (+   225us) service_pool.cc:224] Handling call
0709 13:19:06.174631 (+1654349us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8feb090 00007f04b9089f3a 00007f04bee401cc 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc 00007f04bd3fe392 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:22.465085 (+16290454us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:10.987053 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:11.039961 (+ 52908us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:11.039999 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:19:11.040016 (+    17us) tablet.cc:662] Decoding operations
0709 13:19:11.041958 (+  1942us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:11.042016 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:11.042041 (+    25us) tablet.cc:685] Acquiring locks for 29 operations
0709 13:19:12.079005 (+1036964us) tablet.cc:701] Row locks acquired
0709 13:19:12.079027 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:19:12.079214 (+   187us) write_op.cc:270] Start()
0709 13:19:12.079371 (+   157us) write_op.cc:276] Timestamp: P: 1783603152079155 usec, L: 0
0709 13:19:12.079400 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:19:12.250897 (+171497us) log.cc:844] Serialized 11272 byte log entry
0709 13:19:12.759831 (+508934us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b
0709 13:19:13.506987 (+747156us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:13.953586 (+446599us) op_driver.cc:464] REPLICATION: finished
0709 13:19:14.216792 (+263206us) write_op.cc:301] APPLY: starting
0709 13:19:14.216940 (+   148us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:15.005898 (+788958us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:15.005949 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:16.095265 (+1089316us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:16.095768 (+   503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 13:19:16.095829 (+    61us) write_op.cc:312] APPLY: finished
0709 13:19:16.096452 (+   623us) log.cc:844] Serialized 252 byte log entry
0709 13:19:16.326839 (+230387us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.477201 (+150362us) write_op.cc:454] Released schema lock
0709 13:19:16.499529 (+ 22328us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:16.538785 (+ 39256us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.538815 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":327635,"reported_tablets":46,"spinlock_wait_cycles":550962048,"child_traces":[["op",{"apply.queue_time_us":198170,"apply.run_cpu_time_us":104162,"apply.run_wall_time_us":2370949,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125289,"num_ops":29,"prepare.queue_time_us":275704,"prepare.run_cpu_time_us":36591,"prepare.run_wall_time_us":2227588,"raft.queue_time_us":146285,"raft.run_cpu_time_us":59039,"raft.run_wall_time_us":152458,"replication_time_us":1873987,"spinlock_wait_cycles":72063616,"thread_start_us":1364793,"threads_started":5,"wal-append.queue_time_us":1163555,"wal-append.run_cpu_time_us":56480,"wal-append.run_wall_time_us":2940870}]]}
I20260709 13:19:22.633569 13271 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:22.635843 13304 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:22.635426  6416 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 9, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 9 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:22.636399 13304 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:22.636897  6416 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 12, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 12 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:22.639139 13341 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:19:22.644166  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.584872 13266 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:22.592476 13096 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:22.646584  7229 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:22.646733  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.650425  6593 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.576135 13312 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:22.650689  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:22.636924 13304 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:22.652558 13241 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.652972 13091 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:22.652997  6416 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 13, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 13 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:22.654835  6515 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:22.616205 13345 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.583519 13277 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:22.591853 12975 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 13:19:22.667547 13165 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:22.673833  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46110.8us
I20260709 13:19:22.615103 13269 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:22.677975  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:22.568568 13316 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:22.660274 12568 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:22.681489  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.623939 12889 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:22.678794 13091 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:22.672502  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.682579 13241 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.691056  6416 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:22.693445 13313 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:22.634428 13271 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.695012 13076 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.697523 13304 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.700811 13051 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:22.666778 13096 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:22.706171  6665 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.708885  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.712709  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:22.681774 13208 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:22.721814 13344 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:22.721781  6590 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:22.683874 13115 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:22.688895 13277 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.732493  6416 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 9 to 14, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 14 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:22.735634  6597 raft_consensus.cc:2368] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18.
I20260709 13:19:22.651790 13331 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:22.719930 13060 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.726941 13091 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.749610  7200 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.748850 13096 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.752048  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.729411 13005 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:22.753269  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:22.788172 13316 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:22.763366  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.751499 13298 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:22.769779  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:22.753724  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:22.795564  7230 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.796819  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:22.797866  6515 raft_consensus.cc:2368] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20.
I20260709 13:19:22.759469 13271 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.766259  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:22.772238  6519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2740) took 1780 ms (client timeout 2210 ms). Trace:
I20260709 13:19:22.804980  6590 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
W20260709 13:19:22.779784  6488 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.784s (SENT)
I20260709 13:19:22.805550  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2921) took 1313 ms. Trace:
I20260709 13:19:22.805846  6590 rpcz_store.cc:276] 0709 13:19:21.492312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.766030 (+1273718us) service_pool.cc:224] Handling call
0709 13:19:22.805492 (+ 39462us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:22.806272  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:22.806807  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.725107 13248 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.781950  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:22.804908  6519 rpcz_store.cc:269] 0709 13:19:20.992079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.992223 (+   144us) service_pool.cc:224] Handling call
0709 13:19:22.772058 (+1779835us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b5400044054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:22.772170 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255462784}
I20260709 13:19:22.668939 13312 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:22.783671 13208 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:22.759644  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:22.791755  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.799880  6416 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 10 to 12, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 12 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.771718  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2930) took 1219 ms. Trace:
I20260709 13:19:22.810437  6589 rpcz_store.cc:276] 0709 13:19:21.552074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.796319 (+244245us) service_pool.cc:224] Handling call
0709 13:19:22.771518 (+975199us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54003bbfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:22.771649 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255533952}
I20260709 13:19:22.810878  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:22.811648  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.758239 13238 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.765352 13170 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:22.809386  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:22.758914 13115 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:22.793181 13236 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:22.833285  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:22.834560  6590 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.816591 13238 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:22.837889 13248 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.816824 13170 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:22.839140 13170 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.784061 13060 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:22.842962  7235 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:22.809787 13291 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:22.818738 13286 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:22.812671 13312 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:22.795195 13330 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:19:22.828807 12975 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:22.935829  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:22.859172 13236 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:22.812988 13269 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:22.829279 13115 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.864521 13219 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.862430 13290 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:22.947239 13304 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:22.947738 13223 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.005s
I20260709 13:19:22.902135 12975 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.900061 13330 consensus_queue.cc:1243] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:22.910774 13182 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:22.947574 13313 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:22.947592 13352 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:19:22.898308 13291 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:22.944886 13269 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.951722 13290 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:22.955694 13236 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.080940 13350 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:19:23.095938 13344 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:22.960083 13223 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:23.278247 13361 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:23.282090  6588 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 13: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:22.814210  6416 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 9 to 13, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 13 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:23.283596 13051 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:23.192294  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.198179  6522 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:23.289252 13060 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:22.854825  6563 leader_election.cc:400] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20.
I20260709 13:19:23.295894  6563 leader_election.cc:403] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:19:23.297065  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 442459us
W20260709 13:19:23.297782  6518 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2744) took 2304 ms (client timeout 1879 ms). Trace:
I20260709 13:19:22.859792  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:23.298532  6518 rpcz_store.cc:269] 0709 13:19:20.993104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.004685 (+ 11581us) service_pool.cc:224] Handling call
0709 13:19:23.297138 (+2292453us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b540019afd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.297437 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3543394176}
I20260709 13:19:23.300043  6590 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:22.856161  6595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2931) took 1303 ms (client timeout 1604 ms). Trace:
I20260709 13:19:23.301209 13221 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.301924  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:23.301779  6595 rpcz_store.cc:269] 0709 13:19:21.552363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.588143 (+ 35780us) service_pool.cc:224] Handling call
0709 13:19:22.855909 (+1267766us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54002fc1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:22.856077 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1143570176}
I20260709 13:19:23.302899 13223 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.304239  6595 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:23.305173  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1416) took 1885 ms. Trace:
I20260709 13:19:23.305603  6595 rpcz_store.cc:276] 0709 13:19:21.419981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.303241 (+1883260us) service_pool.cc:224] Handling call
0709 13:19:23.303870 (+   629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.304806 (+   936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.305088 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.306972 13238 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.308233 13223 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:23.207515  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.207942  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.213258  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 540666us
I20260709 13:19:23.215348  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:23.311362 13280 consensus_meta.cc:287] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.000s
W20260709 13:19:23.312882  6592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3041) took 1876 ms (client timeout 2208 ms). Trace:
W20260709 13:19:23.313503  6592 rpcz_store.cc:269] 0709 13:19:21.436305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.316182 (+879877us) service_pool.cc:224] Handling call
0709 13:19:23.312281 (+996099us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003a43d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.312594 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322870784}
W20260709 13:19:23.316929 13266 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.640s	user 0.013s	sys 0.005s
W20260709 13:19:23.302502 13275 consensus_meta.cc:287] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.092s	user 0.006s	sys 0.011s
I20260709 13:19:23.317619 13266 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:23.217700  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.319429 13275 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.218945  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:23.322475  6668 raft_consensus.cc:2376] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 18. Re-sending same reply.
I20260709 13:19:23.143966 13290 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:23.219758  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 430318us
I20260709 13:19:23.221800  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.223289  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.327040  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.328327  6671 raft_consensus.cc:2410] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:19:23.329139  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1628) took 1049 ms. Trace:
I20260709 13:19:23.329362  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.329609  7228 rpcz_store.cc:276] 0709 13:19:22.280058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.280808 (+   750us) service_pool.cc:224] Handling call
0709 13:19:22.281067 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.613732 (+332665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.328624 (+714892us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b540051a254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:23.328931 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762444672}
W20260709 13:19:23.229393  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 418426us
I20260709 13:19:23.231266  7237 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:23.081633 13241 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.072497 13269 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.138851 13331 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:23.138479 13339 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.342s	user 0.002s	sys 0.000s
I20260709 13:19:23.339249 13182 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.147809 13344 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.340162 13182 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:23.149040 13356 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.298s	user 0.001s	sys 0.001s
W20260709 13:19:23.144848 13358 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 13:19:23.264722 12950 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:22.956677 12975 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:23.277669 13291 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:23.309262 13223 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.324281 13290 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.398698 13266 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.320608  6664 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:23.400667  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:23.401611  6664 raft_consensus.cc:2368] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13.
I20260709 13:19:23.321695  6670 raft_consensus.cc:2410] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:19:23.218443  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.403659  6667 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:23.405128  6667 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:23.323935  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.335949  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1854) took 1378 ms. Trace:
I20260709 13:19:23.338029  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1632) took 1054 ms. Trace:
I20260709 13:19:23.374720  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.402551  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:23.279659 13361 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.406579  7229 rpcz_store.cc:276] 0709 13:19:21.957585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.958931 (+  1346us) service_pool.cc:224] Handling call
0709 13:19:23.335384 (+1376453us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b540078e1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.335665 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370039680}
W20260709 13:19:23.408002  7230 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.588s	user 0.000s	sys 0.021s
I20260709 13:19:23.408486  7230 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:23.406803  7237 rpcz_store.cc:276] 0709 13:19:22.283665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.284856 (+  1191us) service_pool.cc:224] Handling call
0709 13:19:22.317952 (+ 33096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.336627 (+1018675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.337611 (+   984us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b540057d854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:23.337914 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098999168}
I20260709 13:19:23.406858  6668 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:23.385186  6590 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.389590 12950 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.407182  6671 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:19:23.406173 13121 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.969s	user 0.011s	sys 0.003s
I20260709 13:19:23.406522  6416 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 9 to 12, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 12 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:23.331178 13275 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.410256  6594 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:19:23.393213  6592 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:23.382436  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2917) took 1892 ms. Trace:
I20260709 13:19:23.448551  6662 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:23.410597 13331 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.419561 13345 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.700s	user 0.007s	sys 0.008s
I20260709 13:19:23.443287 13223 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.128217 13236 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.452564  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3046) took 2013 ms. Trace:
I20260709 13:19:23.448182  6595 rpcz_store.cc:276] 0709 13:19:21.490555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.381177 (+1890622us) service_pool.cc:224] Handling call
0709 13:19:23.381723 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.382055 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.382082 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.382136 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:23.382177 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.382246 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.382369 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.410781 13373 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:23.454789  6592 rpcz_store.cc:276] 0709 13:19:21.438152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.384458 (+1946306us) service_pool.cc:224] Handling call
0709 13:19:23.392898 (+  8440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.451716 (+ 58818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.451942 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.410015 13182 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.452291  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3036) took 2016 ms. Trace:
I20260709 13:19:23.444449  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:23.453583 13170 consensus_meta.cc:287] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.014s
I20260709 13:19:23.472446 13170 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.473767  6515 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:23.474174  6515 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:23.474906 13270 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.475602 13270 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.457399  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:23.457261  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:23.479760 13115 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.004s
I20260709 13:19:23.480247 13263 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:23.480338 13115 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:23.481262 13256 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:23.467746 13236 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.484325 13170 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:23.490878  6588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3047) took 2052 ms (client timeout 2035 ms). Trace:
W20260709 13:19:23.491397  6588 rpcz_store.cc:269] 0709 13:19:21.438480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.207595 (+769115us) service_pool.cc:224] Handling call
0709 13:19:23.490541 (+1282946us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002fcbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.490774 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2357844224}
I20260709 13:19:23.492671  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:23.493307  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1418) took 2072 ms (client timeout 1585 ms). Trace:
W20260709 13:19:23.493820  6593 rpcz_store.cc:269] 0709 13:19:21.420483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.423164 (+  2681us) service_pool.cc:224] Handling call
0709 13:19:23.492853 (+2069689us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54000933d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.493077 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2741026048}
W20260709 13:19:23.494320  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 74579.6us
I20260709 13:19:23.494650  6595 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:23.495292  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:23.495486  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2918) took 2004 ms. Trace:
I20260709 13:19:23.495954  6595 rpcz_store.cc:276] 0709 13:19:21.491103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.493897 (+2002794us) service_pool.cc:224] Handling call
0709 13:19:23.494364 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.495204 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.495403 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.497644 13257 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:23.498358 13257 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:23.498840 13257 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:23.459443  6594 rpcz_store.cc:276] 0709 13:19:21.434083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.344436 (+1910353us) service_pool.cc:224] Handling call
0709 13:19:23.409703 (+ 65267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.447646 (+ 37943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.450068 (+  2422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.463131 13275 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:23.486482 13271 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.676s	user 0.016s	sys 0.000s
I20260709 13:19:23.467010 13377 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.467736  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:23.482424 13221 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20.
I20260709 13:19:23.470011  6664 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:23.471446  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:23.501976  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:23.466794  7200 leader_election.cc:336] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.585s (SENT)
W20260709 13:19:23.464303 13096 consensus_meta.cc:287] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.007s
I20260709 13:19:23.513698  6522 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:23.469797 12950 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.497655 13344 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:23.521705  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:23.456557 13121 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:23.459861 13223 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.507743 13345 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:23.529618  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:23.531435  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2919) took 2040 ms. Trace:
I20260709 13:19:23.531383  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:23.532719  6665 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:23.531935  6595 rpcz_store.cc:276] 0709 13:19:21.491547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.497022 (+2005475us) service_pool.cc:224] Handling call
0709 13:19:23.501494 (+  4472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.528578 (+ 27084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.528610 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.531071 (+  2461us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:23.531117 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.531187 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.531371 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.529698  6416 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:23.526319 13115 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:23.538451  6518 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:23.457719 13331 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.530205 13266 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:23.540225  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:23.542016  6517 raft_consensus.cc:2410] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:19:23.545296  6671 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:23.537236 13344 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:23.535727 13361 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.549515 13344 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:23.549829  6562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.212s	user 0.009s	sys 0.010s
I20260709 13:19:23.550035 13344 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.541996 13266 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:23.549108  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:23.540876  6515 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.536051 12975 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.526973 13377 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:23.548128  6596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1408) took 2148 ms (client timeout 1605 ms). Trace:
I20260709 13:19:23.554709  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:19:23.554769  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3042) took 2118 ms (client timeout 1793 ms). Trace:
I20260709 13:19:23.572453 12889 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:23.572517  6597 rpcz_store.cc:269] 0709 13:19:21.436623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.773056 (+336433us) service_pool.cc:224] Handling call
0709 13:19:23.554234 (+1781178us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b5400080554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.554525 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2210934016}
I20260709 13:19:23.573993  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:23.573110 12889 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:23.530515 13096 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.575589 12889 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.576906 13060 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.577617 13096 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.577575 13060 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.556299  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1296) took 1709 ms. Trace:
I20260709 13:19:23.579437  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:23.580000 13060 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.580626 13362 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:23.575141  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2920) took 2083 ms. Trace:
I20260709 13:19:23.559589 13182 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:23.560393  6519 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:23.560173  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 93546.6us
I20260709 13:19:23.581678  6592 rpcz_store.cc:276] 0709 13:19:21.491964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.512182 (+2020218us) service_pool.cc:224] Handling call
0709 13:19:23.537415 (+ 25233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.574705 (+ 37290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.574741 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.574830 (+    89us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:23.574865 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.574940 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.575080 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.560701  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:23.584539  6588 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:23.585187  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.585906  7237 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:23.561026  6515 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.587612  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:23.587790  7232 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:23.588343  7230 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.547854  6669 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
W20260709 13:19:23.588905 13332 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.154s	user 0.013s	sys 0.026s
W20260709 13:19:23.589303 13332 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.013s	sys 0.026s
W20260709 13:19:23.591536 13277 consensus_meta.cc:287] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.743s	user 0.017s	sys 0.002s
I20260709 13:19:23.592291  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.554462  6416 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 1 to 12, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 12 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:23.592199 13277 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.593827  6513 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
W20260709 13:19:23.578557  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.595158  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.595556  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.595697 13277 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:23.538956 13231 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.870s	user 0.007s	sys 0.011s
I20260709 13:19:23.596561 13231 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:23.557137  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 60041.6us
I20260709 13:19:23.559780  6516 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.561969  6591 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:23.570206  6518 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:23.602346  7230 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
W20260709 13:19:23.568327  6596 rpcz_store.cc:269] 0709 13:19:21.399392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.406805 (+  7413us) service_pool.cc:224] Handling call
0709 13:19:23.547752 (+2140947us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003a1954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.547929 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2616159616}
I20260709 13:19:23.642266  6597 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:23.579677  6670 rpcz_store.cc:276] 0709 13:19:21.846852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.848192 (+  1340us) service_pool.cc:224] Handling call
0709 13:19:23.556039 (+1707847us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540017d4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.556225 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3036172672}
I20260709 13:19:23.635929  6519 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:23.567438 13278 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.646304  6515 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
W20260709 13:19:23.642678  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46404.7us
I20260709 13:19:23.526777 13373 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:23.650116  6562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.312s	user 0.009s	sys 0.010s
I20260709 13:19:23.653038  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.654335  7229 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:23.654982  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:23.655699  7237 raft_consensus.cc:2376] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 13. Re-sending same reply.
I20260709 13:19:23.646793  6515 pending_rounds.cc:85] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 13
I20260709 13:19:23.656543  6515 pending_rounds.cc:107] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:19:23.656863  6515 raft_consensus.cc:2889] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.658918 13060 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:23.659931 13060 consensus_queue.cc:1243] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:19:23.660611 13060 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:23.661029 13060 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:23.555054 13266 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:23.610733 13231 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.645790  6595 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:23.668480 13344 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:23.712293 13344 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:23.712613  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:23.712832  7232 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:23.671228 13345 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:23.675165 13257 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:23.676028 13331 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.713901 13370 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:23.714890 13345 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.720623 13373 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.721299 13345 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.662143 13060 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.716714 13257 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:23.676888 13208 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.860s	user 0.007s	sys 0.011s
I20260709 13:19:23.723389 13060 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:23.570338 13354 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.777s	user 0.022s	sys 0.005s
I20260709 13:19:23.727214  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1331) took 1366 ms. Trace:
I20260709 13:19:23.676564  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.691748  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:23.696693  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 1.35965e+06us
I20260709 13:19:23.698720  6588 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:19:23.554574 13165 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.837s	user 0.007s	sys 0.008s
W20260709 13:19:23.680027 13091 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.899s	user 0.010s	sys 0.011s
I20260709 13:19:23.564754 12975 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:23.713793 13344 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:23.723605 13208 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:23.678483 13326 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.556716 13312 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.644s	user 0.007s	sys 0.007s
I20260709 13:19:23.582094 13182 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:23.734403  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:23.735033  6588 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:23.568089 13377 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.728211  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:23.717404  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:23.737098  6596 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:23.735080  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:23.546087 13223 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.741285  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:23.752770  7234 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:23.753543  6416 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 9, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 9 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:23.721246 13096 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:23.717885 13270 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.759245  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:23.761404  6516 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:19:23.760154  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.763559 13367 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.763839  6595 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:23.764166 13367 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.764653  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2927) took 2213 ms. Trace:
I20260709 13:19:23.765175  6595 rpcz_store.cc:276] 0709 13:19:21.551199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.734587 (+2183388us) service_pool.cc:224] Handling call
0709 13:19:23.763581 (+ 28994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.764326 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.764577 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.768513  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2938) took 1734 ms. Trace:
I20260709 13:19:23.769084  6593 rpcz_store.cc:276] 0709 13:19:22.034017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.767024 (+1733007us) service_pool.cc:224] Handling call
0709 13:19:23.767589 (+   565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.767991 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.768031 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.768108 (+    77us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:23.768172 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.768279 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.768441 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.770977  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:23.773293 13182 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.773149  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:23.774825 12568 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.727638  6521 rpcz_store.cc:276] 0709 13:19:22.360875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.361217 (+   342us) service_pool.cc:224] Handling call
0709 13:19:22.361501 (+   284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:22.369689 (+  8188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:22.369729 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:22.370340 (+   611us) log.cc:844] Serialized 52 byte log entry
0709 13:19:22.604779 (+234439us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:22.604861 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.605010 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:23.569293 (+964283us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:23.726691 (+157398us) raft_consensus.cc:1685] finished
0709 13:19:23.726720 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.727019 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7785,"raft.queue_time_us":8709,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":10854656,"thread_start_us":160808,"threads_started":1,"wal-append.queue_time_us":309828}
I20260709 13:19:23.778939  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:23.780545  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:23.780383 13231 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:23.746136 13344 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:23.782541  6596 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:23.783182 13278 raft_consensus.cc:2764] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:19:23.783382  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2935) took 1996 ms. Trace:
I20260709 13:19:23.783912  6596 rpcz_store.cc:276] 0709 13:19:21.786866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.743123 (+1956257us) service_pool.cc:224] Handling call
0709 13:19:23.782349 (+ 39226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.783093 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.783324 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.787446  6597 raft_consensus.cc:2393] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:23.787317  6590 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:23.788967 13096 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.791261  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2941) took 1394 ms. Trace:
I20260709 13:19:23.791496 13096 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.791661  6591 rpcz_store.cc:276] 0709 13:19:22.396794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.935402 (+538608us) service_pool.cc:224] Handling call
0709 13:19:23.791087 (+855685us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54002fdad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.791213 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1362946944}
I20260709 13:19:23.739065 13091 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:23.790361  6513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 1338) took 1343 ms (client timeout 1641 ms). Trace:
W20260709 13:19:23.794528  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 77264.5us
I20260709 13:19:23.803349  6667 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:23.799625  6513 rpcz_store.cc:269] 0709 13:19:22.447089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.539275 (+ 92186us) service_pool.cc:224] Handling call
0709 13:19:23.790137 (+1250862us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003bbd54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:23.790263 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422219136}
I20260709 13:19:23.804709  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:23.806725  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:23.812388  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 89840us
I20260709 13:19:23.813064  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:23.819093  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:23.820776  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.821275  7230 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:23.740273 13312 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.819154  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.824664  7228 raft_consensus.cc:2368] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22.
I20260709 13:19:23.824759  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.826663  6593 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:19:23.829484  6595 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:19:23.830013  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2937) took 1868 ms. Trace:
I20260709 13:19:23.831674  6595 rpcz_store.cc:276] 0709 13:19:21.961515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.766283 (+1804768us) service_pool.cc:224] Handling call
0709 13:19:23.797989 (+ 31706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.829815 (+ 31826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.829972 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.824673  6664 pending_rounds.cc:85] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 11
I20260709 13:19:23.813244 13257 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:23.833068  6522 raft_consensus.cc:2368] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20.
I20260709 13:19:23.852289  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.832752  6664 pending_rounds.cc:107] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:19:23.853231  6664 raft_consensus.cc:2889] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.853084  6671 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:23.832505  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3055) took 1745 ms. Trace:
W20260709 13:19:23.855194  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.854688  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56812.1us
I20260709 13:19:23.816179 12975 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:23.833176  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.827031 12692 raft_consensus.cc:2764] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:19:23.859608  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:23.852427 13256 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.820473 13270 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:23.863586  6588 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:23.855238  6593 rpcz_store.cc:276] 0709 13:19:22.087529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.770415 (+1682886us) service_pool.cc:224] Handling call
0709 13:19:23.798120 (+ 27705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.832216 (+ 34096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.832454 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.860841  6597 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:23.761497 13326 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.853526 13257 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.879536 13257 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:23.884909  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.885576  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:23.887480 13060 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:23.786561 13344 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.888089 13060 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:23.888590 13060 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.888677  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:23.890132  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.891395  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.892431  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.892962  6666 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:23.867447  6595 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:23.868108  6670 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
W20260709 13:19:23.868582 13163 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.464s	user 0.021s	sys 0.024s
I20260709 13:19:23.869580 12889 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.870846 13278 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:23.870697 13256 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.871546 13389 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:23.872018  6662 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.872915  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:23.853839 13263 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:23.873515  6663 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:23.878183 12975 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:23.878795  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 96361.3us
I20260709 13:19:23.896200  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3060) took 1807 ms. Trace:
I20260709 13:19:23.913612  6597 rpcz_store.cc:276] 0709 13:19:22.089102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.788123 (+1699021us) service_pool.cc:224] Handling call
0709 13:19:23.828989 (+ 40866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.895939 (+ 66950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.896141 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.893802  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:23.897998 13163 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.559s	user 0.021s	sys 0.024s
I20260709 13:19:23.915011  6666 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:23.918352  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:23.919185  6668 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:23.858155 13377 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:23.912894  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 72355.5us
I20260709 13:19:23.926442  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:23.900898  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3063) took 1534 ms. Trace:
I20260709 13:19:23.927325  6595 rpcz_store.cc:276] 0709 13:19:22.366262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.832590 (+1466328us) service_pool.cc:224] Handling call
0709 13:19:23.867319 (+ 34729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.900645 (+ 33326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.900835 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.863842 13091 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:23.894661 13115 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.900116 13278 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.956197 13005 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.956568 13278 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.957193  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:23.956908 13005 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.957844  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:23.959414  6594 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:23.960106 12975 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:23.959376 13005 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:23.931980  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:23.927505 13370 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.929863  6518 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:23.917606 13163 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:23.935000 13389 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:23.935398 13312 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:23.935912  6486 leader_election.cc:400] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13.
I20260709 13:19:23.973337  6486 leader_election.cc:403] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:19:23.935994  6593 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:19:23.944393  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 209398us
I20260709 13:19:23.939659  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2944) took 1169 ms. Trace:
I20260709 13:19:23.915524  7235 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:23.960736 12975 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.934170  6588 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:23.977453  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3064) took 1610 ms. Trace:
I20260709 13:19:23.977790  6588 rpcz_store.cc:276] 0709 13:19:22.366532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.864414 (+1497882us) service_pool.cc:224] Handling call
0709 13:19:23.933904 (+ 69490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.977206 (+ 43302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.977397 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.967016  6662 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:23.975611  6597 rpcz_store.cc:276] 0709 13:19:22.770561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.921595 (+1151034us) service_pool.cc:224] Handling call
0709 13:19:23.939039 (+ 17444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.939289 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.939326 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.939387 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:23.939422 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.939478 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.939584 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.941716 12889 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:23.983408  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1433) took 1558 ms. Trace:
I20260709 13:19:23.983970  6593 rpcz_store.cc:276] 0709 13:19:22.424753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.900601 (+1475848us) service_pool.cc:224] Handling call
0709 13:19:23.935649 (+ 35048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.974569 (+ 38920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.983316 (+  8747us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.987959  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:23.990450  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58575.3us
I20260709 13:19:23.994144  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:23.992419  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:23.986043 13263 raft_consensus.cc:2764] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:19:24.012672  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 91355.5us
I20260709 13:19:24.013778  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.013989  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.014765 13316 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.015420 13316 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:23.990980 13377 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:24.017679 13263 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:24.018038 13377 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.018788 13277 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:24.019317 13277 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.020475  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:24.020243 13277 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.021927  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:23.994210 13370 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:24.023496  6663 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 3020) took 3387 ms (client timeout 2156 ms). Trace:
I20260709 13:19:24.028936  7230 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 13.
I20260709 13:19:23.976735 13386 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:23.961093 13257 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:24.023912  6663 rpcz_store.cc:269] 0709 13:19:20.635608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.654438 (+ 18830us) service_pool.cc:224] Handling call
0709 13:19:21.574739 (+920301us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b58000b7c98. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:24.023229 (+2448490us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b54001ed7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:24.023384 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4927021824}
I20260709 13:19:23.971859 13060 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:23.936542 13115 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:24.035894  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64840.2us
W20260709 13:19:24.036535 13269 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.004s
I20260709 13:19:24.030349 13278 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:24.059813  6669 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:24.059902 13370 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.030431 13182 raft_consensus.cc:2764] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:19:24.039671  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.059635 13269 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.064553  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:23.995934 13290 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 12. Current term is 13.
I20260709 13:19:24.065084 13269 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.066638 13320 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:24.067448 13320 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:24.067948 12975 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.068058 13320 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:24.051062  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 93338.4us
I20260709 13:19:24.051193 13321 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:19:24.071487  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:24.073230  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.028419  7228 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:24.018311 13365 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.074s
W20260709 13:19:24.076064  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 69771.1us
I20260709 13:19:24.058178  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:24.048921 13223 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.067215 13005 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:23.992158 13256 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.047098 13076 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.990967 13389 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.079901 13238 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:24.081846  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:24.053489 13367 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.017867 13316 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.112567  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3062) took 2020 ms. Trace:
I20260709 13:19:24.114179  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:24.109671  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.107082 13256 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:24.117604  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:24.122807  6595 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:24.123917 13256 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.126757  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:24.083420 12975 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:19:24.095368 13076 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.134681  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.126232 13256 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:24.141101  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.142490  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.080004 13163 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.113205  6591 rpcz_store.cc:276] 0709 13:19:22.089977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.792561 (+1702584us) service_pool.cc:224] Handling call
0709 13:19:23.792913 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.793195 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:23.943239 (+150044us) log.cc:844] Serialized 14 byte log entry
0709 13:19:24.109814 (+166575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:24.109870 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:24.109915 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.110001 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.110154 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":456,"prepare.queue_time_us":218426,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":7781,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":13778,"thread_start_us":257437,"threads_started":2,"wal-append.queue_time_us":135745}
I20260709 13:19:24.134145 13060 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.135466 13326 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.137903 13316 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:19:24.167665  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 143979us
I20260709 13:19:24.159199 13386 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:19:24.177362  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 105791us
I20260709 13:19:24.168992 13223 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:24.214536  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:24.211546  6486 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.154304 13389 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.173009 13121 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:24.232604  6668 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:19:24.240051  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 66593.5us
I20260709 13:19:24.144773 13320 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.203523  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:24.244758  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 119373us
I20260709 13:19:24.246135  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3061) took 2156 ms. Trace:
I20260709 13:19:24.190265 13257 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.226454 13270 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:24.230800 13223 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:24.239427 13115 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.243790 13060 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:24.262472  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 59058.1us
I20260709 13:19:24.247404 13377 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:24.264295  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 195964us
W20260709 13:19:24.268419  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53987.2us
I20260709 13:19:24.274564  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:24.270834  6486 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.246600  6590 rpcz_store.cc:276] 0709 13:19:22.089365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.788145 (+1698780us) service_pool.cc:224] Handling call
0709 13:19:23.823751 (+ 35606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.830027 (+  6276us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:23.970809 (+140782us) log.cc:844] Serialized 14 byte log entry
0709 13:19:24.244414 (+273605us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:24.245745 (+  1331us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:24.245871 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.245950 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.246064 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":969,"prepare.queue_time_us":181956,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":28,"raft.queue_time_us":58894,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":318477,"threads_started":3}
I20260709 13:19:24.288789  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:24.267367 13231 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:24.273067 13060 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.294365  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3058) took 2205 ms. Trace:
I20260709 13:19:24.295692  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:24.273811 13377 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.308626  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:24.267403 13278 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:19:24.320729  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:24.322784  6597 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:24.324860  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:24.326048  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:24.327916  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.332187  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:24.294766  6596 rpcz_store.cc:276] 0709 13:19:22.088538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.797340 (+1708802us) service_pool.cc:224] Handling call
0709 13:19:23.840684 (+ 43344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.854532 (+ 13848us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:24.144073 (+289541us) log.cc:844] Serialized 14 byte log entry
0709 13:19:24.294062 (+149989us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:24.294117 (+    55us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:24.294158 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.294218 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.294315 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27925,"prepare.queue_time_us":157661,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":33,"raft.queue_time_us":21960,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"thread_start_us":336235,"threads_started":3,"wal-append.queue_time_us":112037}
I20260709 13:19:24.332795  6521 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:24.290390 13403 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:24.296314 13060 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.301342 13377 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:24.331835  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:24.309425 13269 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:24.290136 13321 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:19:24.339773  6588 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:24.337109  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:24.346256  6522 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:24.341476  6518 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:24.309000 13270 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:24.299126 13316 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:24.344519  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:24.321365 13231 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:24.350920  6594 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:19:24.390954  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53970.1us
I20260709 13:19:24.392025  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:24.351992  6513 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:24.400009  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55562.9us
I20260709 13:19:24.337314 13403 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:24.341550 13326 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:24.407719  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 131268us
I20260709 13:19:24.409578 12915 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:24.410036 12915 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.410676 12915 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.410746  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:24.410955 12568 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:24.411841 12568 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.412111  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:24.413149 13076 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.413431  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:24.414278  6594 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:19:24.414152 13231 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.414705 13223 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:24.415371 13406 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:19:24.418110  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:24.422932  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:24.424113  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:24.424769  6668 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:24.424928 13060 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.425681  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:24.426029 13321 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:24.427258  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:24.427444 13321 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.429744 13321 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:24.426216 13266 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.611s	user 0.006s	sys 0.006s
I20260709 13:19:24.352026 13270 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:24.431200  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:24.410095  6670 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:24.427840  6519 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:24.409951  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:24.441403  6662 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:24.412744 12568 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.431195  7235 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:24.443210  6595 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:24.398327 13269 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.447163  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.448256  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:24.451050  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:24.452109  6637 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:24.452595 13389 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:24.453174 13389 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.453907  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.406773 13403 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.457182  6597 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:24.443352 13270 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:24.456297  6519 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:24.461367  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.408931 13238 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:24.430476 13266 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.475988 13256 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.477464 13266 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:24.479480 13398 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:19:24.474306 13393 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:19:24.481609 13405 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:19:24.483203 13396 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:19:24.484016  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.485255  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:24.479390  7228 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:24.487732  7235 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 20. Ops: []
I20260709 13:19:24.491341 13269 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.495282 13263 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:24.495787 13263 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.496167 13263 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.498036  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.500188 13416 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.500795 13416 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.500968  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.504869 13256 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.507187  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3032) took 1299 ms. Trace:
I20260709 13:19:24.507616  6667 rpcz_store.cc:276] 0709 13:19:23.208247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.405955 (+197708us) service_pool.cc:224] Handling call
0709 13:19:23.406274 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.803810 (+397536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.506892 (+703082us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540011dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:24.507109 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891035392}
I20260709 13:19:24.505795  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.454702 13404 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:24.454949 13326 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:24.512094  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.517536  7232 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:24.456453 13208 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:24.510749  7236 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:24.454144 13389 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.519114 13115 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:24.519539  6521 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:24.520076  7228 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:24.519892 13170 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:24.472259 13060 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:24.516113  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53678.5us
I20260709 13:19:24.522595  7234 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:24.537844  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:24.543573  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.543977  6521 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 16: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:24.543946  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:24.455425 13412 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.552215  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:24.505604 13266 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:24.512981 13404 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:24.554082  7230 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:24.559576 13386 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:24.561861  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 108053us
I20260709 13:19:24.562053  6588 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 13:19:24.566814  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:24.553623 13321 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:24.578009  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:24.556346 13163 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.590023  7230 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:24.562418 12568 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:24.595461  6521 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:24.596479  7235 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.597235 13324 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:24.597468  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:24.597916 13324 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:24.598969  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:24.599215 13115 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:24.599061 13324 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.600572 13115 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:24.601121 13115 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:24.601320  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:24.602026  6667 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:24.597321 13404 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.603914 13404 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:24.605686 13386 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.600510  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:24.598480  6669 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:24.607404 13373 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:24.556279 13208 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.608032 13373 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.608043  6668 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:24.595364  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:24.610026  7232 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:24.576900 13321 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.595867 13416 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.623857  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:24.624616  6663 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:24.578723 13121 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:24.621114  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.626860  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:24.626922  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:24.622776  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:24.627661  6668 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:24.626036  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:24.655990  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:24.550611 13316 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.658728  6665 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:24.655460 13182 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:24.659391  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 60704.8us
I20260709 13:19:24.564615 13412 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.662695 13096 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:19:24.664839 13096 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:24.665277 13096 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.665565 13412 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.666019 13271 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:24.602137 13115 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.576947 13266 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:19:24.667433 13266 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:24.624616 13321 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.667831 13266 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.668844 13266 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.670224 13266 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:24.656533 13091 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:24.670951 13091 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.671335 13091 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:24.671525 13005 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.672132 13005 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.675766  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:24.656698  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 100496us
I20260709 13:19:24.666446 13271 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.684681  6591 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:24.684731  7235 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:24.627565  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.692834  6668 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:24.711519 13170 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.657766 13060 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:24.713904  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:24.700079 13271 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:24.617415 13163 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.723949  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.726636  6663 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
W20260709 13:19:24.727474  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 68896.9us
I20260709 13:19:24.726891 13428 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:24.729003  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:24.730859 13208 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:24.722880 13416 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.733481 13416 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.735531 13416 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.736848  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:24.737046 13403 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.737641 13269 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:24.738113 13269 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:24.739391 13269 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.742743 13408 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:24.677446 13256 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:19:24.716077 13060 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.657796 13326 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.705264 13377 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.754523 13331 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:24.773022 13331 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:24.774181 13331 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.755216 13005 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.777562 13005 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:24.753086 13208 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:24.778578 13208 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.778996 13208 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:24.780540 13429 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:24.778777  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 151320us
I20260709 13:19:24.783254  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.722318 13170 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.751407 13060 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:24.778048 13263 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.808388  7237 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
W20260709 13:19:24.786428 13423 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:19:24.811759  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 151550us
I20260709 13:19:24.812902  6637 leader_election.cc:400] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18.
I20260709 13:19:24.815186 13373 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.815673 13170 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.817799 13256 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:24.820616 13096 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.821612 13403 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.822283 13096 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:24.836175 12915 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.838581 12915 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.818403 13256 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.840905 13256 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:24.844883 13271 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.851317 13263 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.853284  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:24.854367  6668 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:24.860029 13321 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.860607 13377 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:24.864959  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2943) took 2094 ms (client timeout 2009 ms). Trace:
W20260709 13:19:24.865391  6589 rpcz_store.cc:269] 0709 13:19:22.770271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.811322 (+ 41051us) service_pool.cc:224] Handling call
0709 13:19:24.864600 (+2053278us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540019cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:24.864768 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3822011648}
I20260709 13:19:24.866745  6592 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:24.867342 13424 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:19:24.870111 13182 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:24.872658  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 89511.4us
I20260709 13:19:24.873275 13407 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:24.874950 13163 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:24.875344  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.889710 13270 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:24.862407 13321 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:24.890758 13286 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:19:24.891064  6663 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:24.892096 13384 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.163s	user 0.001s	sys 0.005s
I20260709 13:19:24.893787  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1341) took 1426 ms. Trace:
I20260709 13:19:24.894194  6514 rpcz_store.cc:276] 0709 13:19:23.467454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.470414 (+  2960us) service_pool.cc:224] Handling call
0709 13:19:23.470940 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.471434 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.471491 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:23.472182 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:19:23.541420 (+ 69238us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:23.541461 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.541534 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.856279 (+1314745us) spinlock_profiling.cc:243] Waited 9.51 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:24.893400 (+ 37121us) raft_consensus.cc:1685] finished
0709 13:19:24.893425 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.893633 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21869952,"thread_start_us":64181,"threads_started":1,"wal-append.queue_time_us":245682}
I20260709 13:19:24.893460 13091 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:24.896494 13270 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:24.908538  7232 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:24.937309  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2774) took 1228 ms. Trace:
W20260709 13:19:24.941744  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 217858us
I20260709 13:19:24.942814  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:24.937875  6515 rpcz_store.cc:276] 0709 13:19:23.708981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.709169 (+   188us) service_pool.cc:224] Handling call
0709 13:19:23.727112 (+ 17943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.794127 (+ 67015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.794160 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.804314 (+ 10154us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:23.804375 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.936970 (+1132595us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b54003ccb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:24.937043 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.937229 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76786432}
W20260709 13:19:24.952653  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.953183  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.956019  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:24.957211  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:24.962575  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.965402  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.932147 13345 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:24.974629  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 94355us
W20260709 13:19:24.957913 13362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.282s	user 0.000s	sys 0.024s
I20260709 13:19:24.979426  6563 leader_election.cc:400] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20.
W20260709 13:19:24.980265  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 166283us
I20260709 13:19:24.959290 13324 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.965164 13377 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:24.965389 13321 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:24.964170 12975 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:24.988469 13403 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:24.977104 13362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.328s	user 0.000s	sys 0.024s
I20260709 13:19:24.995875 12692 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:19:24.999904  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:25.007939  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:24.972975 13312 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:25.007359  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:24.974824 13345 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:25.012650  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:25.014302  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.015064  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:25.015579  6589 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:25.049671  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:25.016870  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:24.954026 13208 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.018347  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:25.018541  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:25.013548 13321 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.973754 13370 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:19:25.000926 13377 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:25.073139  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56342.2us
W20260709 13:19:25.052426  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.052603  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:25.074705  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59761.3us
I20260709 13:19:25.075016  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:25.053226  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:25.052649  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:25.053660  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:25.053782  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:25.019944  6515 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 20: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:25.342793  7235 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:25.342880  7228 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:25.342979  7230 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:25.345148  7236 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:25.350268  7228 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:19:25.049472 13223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.021s	sys 0.024s
W20260709 13:19:25.351246 13223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.021s	sys 0.024s
I20260709 13:19:25.051759  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:25.351697 13060 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.542s	user 0.016s	sys 0.005s
I20260709 13:19:25.063284  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:25.352233  6596 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:25.350478  7230 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:25.352115 13060 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:25.017542 13231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.006s	sys 0.006s
W20260709 13:19:25.354815 13231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.008s	sys 0.011s
I20260709 13:19:25.021039 13076 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:19:25.064136  6514 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:25.064659  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:25.356119  6593 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:19:25.356648  6593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2951) took 1307 ms (client timeout 1738 ms). Trace:
W20260709 13:19:25.357028  6593 rpcz_store.cc:269] 0709 13:19:24.049648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.049155 (+999507us) service_pool.cc:224] Handling call
0709 13:19:25.356585 (+307430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:25.066903  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:25.358023 12568 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.742s	user 0.014s	sys 0.010s
W20260709 13:19:25.358830  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 337656us
I20260709 13:19:25.066666 13321 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:25.080255 13377 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:25.365139 13060 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:25.027347 13324 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.127801 13236 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.372402 13324 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:25.371937  6589 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:19:25.155673  6561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.711s (SENT) [suppressed 53 similar messages]
W20260709 13:19:25.379196  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 313061us
W20260709 13:19:25.156565 13434 log.cc:927] Time spent T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:19:25.064708 13163 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.385794  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:25.272370 13266 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:25.403383 13060 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:25.403118 13266 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.405395  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:25.405719 13266 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:24.996789 13386 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:19:25.337375  6588 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:25.406438 13060 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:25.407673 13377 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.410166 13377 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:25.341126 13096 consensus_meta.cc:287] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.010s
I20260709 13:19:25.010725 12975 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.412122 12975 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.355679  6520 raft_consensus.cc:2368] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18.
I20260709 13:19:25.358943 13290 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:25.414908 13290 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.415328 13290 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:25.021200 13362 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:25.420353 12975 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:25.391413  6561 leader_election.cc:336] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.503s (SENT)
I20260709 13:19:25.397315  6596 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:25.422385 13362 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:25.430261 13413 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.003s	user 0.003s	sys 0.000s
I20260709 13:19:25.427039  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:25.007241 13408 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:19:25.411057 13321 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:25.441275  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 365872us
W20260709 13:19:25.442021  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.585497  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:25.444025 13331 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:25.472900 13163 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:25.473008 13076 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:25.473361  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 81953.1us
I20260709 13:19:25.590814 13163 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:25.479346  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:25.592003  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:25.497941 13415 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.011s	user 0.000s	sys 0.004s
I20260709 13:19:25.594152 13060 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:25.500234  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 2924) took 4006 ms. Trace:
I20260709 13:19:25.595498 12568 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:25.595947 12568 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:25.595821  6592 rpcz_store.cc:276] 0709 13:19:21.493451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.582966 (+2089515us) service_pool.cc:224] Handling call
0709 13:19:24.866487 (+1283521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.867196 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.499940 (+632744us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540019cdd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:25.500145 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2950111488}
I20260709 13:19:25.504176 13115 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.506951 13091 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:25.597162 12568 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:25.598533 12568 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:25.511497  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1879) took 1014 ms. Trace:
I20260709 13:19:25.602890  7237 rpcz_store.cc:276] 0709 13:19:24.496775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.499435 (+  2660us) service_pool.cc:224] Handling call
0709 13:19:24.499828 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:24.808872 (+309044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.511251 (+702379us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b540025c454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:25.511439 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709005696}
I20260709 13:19:25.514184  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:25.514240  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:25.605428  6670 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:25.514972  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:25.611008  6664 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:25.515162  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:25.514708  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:25.516043 13266 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.515879  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:25.612890 13266 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:25.615496 13266 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:25.536849 13412 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:25.537621 13096 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:25.627187 13096 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.630079 13096 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:25.630765 13412 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:25.631906 13412 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.633430 13412 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:25.549414 13417 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.068s	user 0.000s	sys 0.004s
I20260709 13:19:25.591970 13163 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.635730 13163 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:19:25.538537 13271 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:25.643639  6670 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
W20260709 13:19:25.656652  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.657399 12915 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:19:25.658152  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 140755us
W20260709 13:19:25.659293  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 73888.2us
I20260709 13:19:25.661387  6667 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
W20260709 13:19:25.649417  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 583715us
I20260709 13:19:25.401186 13236 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:25.658888  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:25.674731 13091 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:25.675561  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 55878.1us
I20260709 13:19:25.340652 13208 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:25.697834  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 624498us
I20260709 13:19:25.698146  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:25.705243  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46439.6us
I20260709 13:19:25.718989  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:25.723977  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:25.724678  6521 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:25.725090  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:25.725631  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:25.726188  6596 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:25.729127  6520 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:25.733251  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:25.737412  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:25.683719 13091 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:25.743108  6670 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:25.683641 13076 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:25.748091  6593 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:25.745349  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47327.9us
I20260709 13:19:25.716571 13290 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.720432 13403 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.771440  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:25.771833  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:25.775647  6663 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:25.775733  6588 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:25.773317  6592 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:19:25.767995  6487 leader_election.cc:400] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22.
W20260709 13:19:25.777979  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.778100  6487 leader_election.cc:403] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:19:25.752772 13389 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:25.732497 13331 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:25.736557 13277 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.688623 12975 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:25.791404  6667 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:25.792722 12568 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22.
I20260709 13:19:25.794363  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:25.790668  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:25.802739  6670 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:25.803972 12568 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:25.804603  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:25.806447  6596 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:25.807794  6521 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:25.811000 13404 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.813014  6487 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:25.716252 13231 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:19:25.828749  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:25.833002  6663 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:25.817440 13377 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.837810 13377 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.822671 13389 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:25.843185  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:25.845049 12568 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:25.846489  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:25.851358  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 105746us
I20260709 13:19:25.854588 12568 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:25.863047 13362 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:25.869947  6513 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:25.772329 13096 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 13:19:25.842240 13320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.417s	user 0.042s	sys 0.043s
I20260709 13:19:25.877473  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:25.881526  6519 raft_consensus.cc:2410] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:19:25.819881 13324 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.887740  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:25.845168 13266 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:25.840698 13370 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:19:25.902426 13431 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:25.903079 13431 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:25.903992 13324 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.904091 13431 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.888622 13362 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.878115 13377 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:25.907059  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:25.907698  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.907208 13362 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.910228 13362 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:25.887681 13320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.462s	user 0.042s	sys 0.043s
I20260709 13:19:25.916139 13316 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:25.916880 13316 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:25.921619  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:25.819500 13277 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.922384  6663 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:25.811510 13404 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.924418 13404 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:25.853964 13389 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:19:25.898422  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:25.925022 13266 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:25.925278  6513 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:25.926093 13266 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.788368 13345 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.823897 13331 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:25.924464 13096 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:25.790458 13403 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.940701  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:25.855350 13386 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:25.906411 13324 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:25.946919  6593 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:19:25.942342  6521 raft_consensus.cc:2368] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18.
I20260709 13:19:25.906821 13409 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:25.933902 13433 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.939082 13096 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:25.956285 12568 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:25.948073 13076 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.950512 13389 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.945904 13121 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:25.957559  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 146349us
W20260709 13:19:25.991251  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.882347 13091 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:26.070823  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.085985  7237 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:26.081625  7230 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:25.951241 13386 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.083962  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:26.082831  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:25.937726 12915 consensus_queue.cc:1175] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64b95375974b48d18c596a23c9cac75c" responder_term: 17 status { last_received { term: 17 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 17 index: 14 } }
I20260709 13:19:26.047720 13115 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.972005 13076 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:26.091715  7228 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:26.048861 13321 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.127588  7229 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:19:26.135871  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:26.101135  7234 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:26.102485  7232 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:26.103538  7236 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:26.094156  7200 leader_election.cc:400] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18.
I20260709 13:19:26.139214  7200 leader_election.cc:403] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:26.057826 13365 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:26.127295  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.142521  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.148224  6666 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:26.143996  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 66250.8us
I20260709 13:19:26.148859  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.149262  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.149817  6662 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:26.086210 13231 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.161511  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:25.969704 13409 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:25.971720 13433 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.159472  6664 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:26.150054  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:25.984148 13389 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.090560 13060 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.105448 13208 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.174010  6589 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:26.177189  6589 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:26.178308  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.178530  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.179244  6595 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
W20260709 13:19:26.179489  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 43701us
I20260709 13:19:26.181911  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.182870  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:26.183115  6671 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:26.183990  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:26.187960  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.188589  6592 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:26.189738 13377 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.189858 13457 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:26.068886 13121 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.149523 13404 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.190995  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.161758 13091 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:26.223026  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:26.165585 13429 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:26.192948 13456 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:26.267419  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.268576  6592 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:26.268492  6589 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.269560  6589 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:26.270985  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.271871 13427 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:26.272320 13427 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.272686 13427 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.172633 13444 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.160509 13437 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:26.135316 13115 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.259424  6666 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:26.261145  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:26.191905 13442 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:19:26.283951  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.297253  6594 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:26.165115 13312 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.098s
W20260709 13:19:26.300264  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.302476  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.258680  6662 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:26.308084  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.172942 13231 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:26.329118  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.192445 13458 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:26.331806  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 108893us
I20260709 13:19:26.144284 13331 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.182359 13418 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.127s
I20260709 13:19:26.205588 13060 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.219915 13208 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.354490  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:26.355688  6665 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:26.204433 13096 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.276289 13404 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.413316  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:26.331254 12915 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:26.331611 13458 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.417074  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:26.417974  6515 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:26.418964  6594 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:26.420814 12568 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.297245 13444 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.421415 12568 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.381373 13407 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:19:26.414471 13448 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.264s	user 0.000s	sys 0.002s
I20260709 13:19:26.429611  6589 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:26.383350 13316 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.356647 13459 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:19:26.420217 13429 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:26.442245  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:26.420256 13458 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.427728 13096 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.445127 12568 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.425001 13277 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.437809  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:26.451982  7231 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.427495 13407 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:26.451771  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 80883us
I20260709 13:19:26.453243 13266 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.453991  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:26.450202 12568 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.459008  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.444972 13316 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:26.460601 12568 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.462844  6563 leader_election.cc:400] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18.
I20260709 13:19:26.464442 13471 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.464990 13471 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.465864 13389 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.449071 13464 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.467198 13464 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.468164  6514 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:26.468499 13115 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.469234  6519 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:26.469306 13464 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.469712 13386 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.470677 13409 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.470830  6514 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:19:26.472000 13453 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:19:26.473990 13443 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:26.472940  6514 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:26.476833  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:26.477468  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.477918  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.477969  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:26.483902  7231 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:26.484130  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:26.459249 13418 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:26.458216 13441 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:26.493161 13163 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.857s	user 0.011s	sys 0.002s
I20260709 13:19:26.460258 13266 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.496487 13290 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.497915 13455 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:26.502347  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.446002 13454 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.313s
I20260709 13:19:26.503043  6594 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:26.498399 13455 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.504626  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.514228  6593 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:26.514464  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.498898 13266 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.485229 13389 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.519166  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.518144  6590 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:26.520663 13223 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:26.521019  6592 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:26.521104 13223 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:26.522284 13223 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.529611 13463 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.512815 13076 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.530234 13463 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:26.531096 13439 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.375s	user 0.002s	sys 0.000s
I20260709 13:19:26.483507 13408 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:19:26.551846  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.552552  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.518714 13472 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.516229 13455 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:26.565359  6518 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:26.567289  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:26.571417  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:26.549575 13389 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.573745 13412 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:26.575486 13389 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:26.577093 13467 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:26.557574 13465 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.503333 13441 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.583439  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:26.595153  6514 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:26.597908  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.602743 13440 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.322s	user 0.000s	sys 0.002s
I20260709 13:19:26.604313  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.607575  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.608392  6591 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:26.610198  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.611025  6662 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:26.611059  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.611761  6666 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:26.606942  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.561297 13091 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:19:26.592998 13060 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:26.623476  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 147141us
I20260709 13:19:26.624177  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:26.624568  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.577097 13480 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:19:26.677093  6666 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:26.675251  6663 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:26.623435  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.674469  6517 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:26.678975  6595 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:26.679462  6595 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:26.620102 13464 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.614034 13256 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.620597 13444 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.569922 13472 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:26.701377  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 121139us
W20260709 13:19:26.701923  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 95060.6us
W20260709 13:19:26.708390  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 150203us
I20260709 13:19:26.682830  6668 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:26.707263  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:26.716157  6514 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:26.709122  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.619151 13463 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:26.719643  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.720129  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:26.720402  6518 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:26.720767  6519 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:26.720561  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:26.721526  6522 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:26.703200 13266 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:26.722827 13266 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.723323  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.678874 13428 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.595258 13465 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.677253 13474 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:26.723471 13452 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.459s	user 0.000s	sys 0.003s
I20260709 13:19:26.708199 13277 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.736706  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.733400  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.739161  6670 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:26.720846 13060 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:26.723325 13266 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.707597 13256 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.730515 13435 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:26.743256  6595 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:26.743384 13431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.012s	sys 0.014s
I20260709 13:19:26.706605 13464 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.732142 13473 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:26.761727  6595 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.727363 13407 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.768267 13407 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:26.758106 13060 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:26.755345 13435 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.771067 13435 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:26.771797 13472 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.740978 13474 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:26.760438 13431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.012s	sys 0.014s
I20260709 13:19:26.757452 13463 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:26.782217 13464 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.786089  6594 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:19:26.786542  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:26.787374  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:26.789717  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.789718 13377 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.001s
I20260709 13:19:26.790901 13377 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:26.790982  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.791841  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.792289  6662 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:26.805667  6662 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:26.792531  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.805968  6596 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 20: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:26.806530  6588 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:26.793459  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.794674  7230 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:26.734959 13428 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.798022  6591 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:26.750120 13277 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.760072 13473 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:26.819965  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:26.822261  6593 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:26.828212  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.822897  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:26.825919  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:26.831348  6591 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:26.835307  6595 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:26.755326 13481 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:19:26.792954 13463 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:26.836782  6592 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:26.837507  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:26.830312  6662 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.843596  7236 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:26.844123  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:26.850682  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:26.851508  6588 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:26.853931  6662 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:26.858204  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 139144us
I20260709 13:19:26.858974  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:26.860718  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:26.863121  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 121811us
I20260709 13:19:26.863234  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.865424  7236 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:26.866880  6591 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:26.867153  6595 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:19:26.830003 13473 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:26.857723 13456 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.016s	sys 0.026s
I20260709 13:19:26.869975  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:26.870507  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.843643 13463 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:26.872480  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:26.879978  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.879978  7236 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:26.880908  6664 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:26.875452 13256 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:26.886302  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:26.875244 13456 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.016s	sys 0.027s
I20260709 13:19:26.885085  6520 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:26.892602  7236 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:19:26.893034  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.885130  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:26.889894  6671 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:26.896354  6662 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:26.896688  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:26.897190  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.897733  6662 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:26.888652  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.899134  6664 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:26.889398  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.869228 13441 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.901163  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:26.903020  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 169729us
I20260709 13:19:26.901752 13486 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:26.903406  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.910529 13223 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.911535  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:26.915216 13483 log.cc:927] Time spent T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 13:19:26.916229 13489 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:19:26.916122  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.917271  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.917752 13096 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:26.918469 13389 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:26.919222 13096 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:26.919665 13096 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.918094  6671 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 13: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:26.917200  6665 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:26.920392  7236 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:26.920441  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.921916  6588 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:26.920050 13096 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:26.925661  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.925987  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.886651 13488 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:26.884367 13496 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:26.930017  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.930585  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:26.930769  6588 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 11.
I20260709 13:19:26.931514  6590 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:26.934430  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:26.935113  6664 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:26.932004 13389 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:19:26.935612 13493 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:19:26.937346  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:26.937747 13477 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:19:26.933557 13496 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.938769 13496 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:26.773800 13482 consensus_queue.cc:1175] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "53fee9894731461a821a131be24dbbca" responder_term: 18 status { last_received { term: 14 index: 13 } last_committed_idx: 13 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:26.899289 13456 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:26.942492 13456 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.943118  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.938542 13465 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.944904  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.948623 13433 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.949258 13433 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:26.949687 13433 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:26.951061  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:26.953003  6671 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:26.950589 13496 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.923465 13454 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:19:26.955027 13478 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:19:26.927654 13486 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:26.956573 13486 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.957157 13433 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.934368 13475 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:26.959026 13475 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.959388 13475 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:26.902025 13377 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:26.964247  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.967514  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.968169  6590 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
W20260709 13:19:26.962608 13437 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.659s	user 0.005s	sys 0.006s
I20260709 13:19:26.969802 13437 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18.
I20260709 13:19:26.874883 13060 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:26.969475 13389 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:19:26.975369 13428 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.976670 13277 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.976797 13475 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.979182 13475 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.980082  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:26.980885 13435 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:26.982156 13496 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.956916 13403 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.963231 13474 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:26.987916  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:26.988451 13076 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:26.988721  7232 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:26.989298 13464 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:26.990222  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:26.968892  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:26.934763  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:26.993000  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 59204us
I20260709 13:19:26.994679  7230 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:26.994679  6664 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:26.996486  7237 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:26.990741 13464 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:26.998392 13493 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:26.998740 13464 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.999219 13464 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:26.999895 13493 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:27.001278 13493 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:27.003340  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.006917 13465 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:27.007406 13403 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:26.930279 13290 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:27.008709  6669 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:27.009614  6662 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:27.009799  6517 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:26.998766  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:27.012449  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:27.015594  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:27.019479  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:26.997342 13458 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.002s
W20260709 13:19:27.013660 13427 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.691s	user 0.007s	sys 0.003s
I20260709 13:19:27.024634  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:26.986317 13463 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.026815  7231 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:27.029650  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 60811.8us
I20260709 13:19:27.041287  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.033545  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:27.034032 13447 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.000s	sys 0.012s
I20260709 13:19:27.029356  7237 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:27.014358 13290 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:27.043865  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:26.956928 13455 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.044108  7232 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:27.062295  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:27.093879  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.023504 13321 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.013s	sys 0.009s
I20260709 13:19:27.048090  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:27.044356 13474 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:27.039076 13389 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.104746  7235 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:27.048090 13447 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.000s	sys 0.013s
I20260709 13:19:27.106052  7231 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:19:27.049180  7234 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:19:27.046587  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:27.017918 13465 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:27.108579  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:27.108601  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.142529 13465 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:27.143013  6662 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:19:27.109582  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65821.7us
I20260709 13:19:27.109859  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.143103 13465 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:27.143813  6517 raft_consensus.cc:2368] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21.
I20260709 13:19:27.144390  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:27.144639  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.028048 13403 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:27.109953  6670 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:27.028558 13428 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:27.103776 13290 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:27.113058 13325 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.106997 12975 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.103554 13478 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.116767 13389 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.117482 13473 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:27.118889 13449 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.061s	user 0.001s	sys 0.003s
I20260709 13:19:27.030058 13266 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.173741 12568 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.174402 13266 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.044564 13475 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.120488  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:27.175242 13475 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.176986  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.111614 13208 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:27.177577 13475 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.103273 13435 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.178380  6517 raft_consensus.cc:2368] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20.
I20260709 13:19:27.129380 13491 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:27.178982 13491 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.163223 12975 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.179562 13491 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.180485  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:27.181077  6590 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:27.181380 12975 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.182240  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:27.182825  6590 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:27.178948 13475 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 13:19:27.138597  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.046164 13496 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:27.139585  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.143096  7232 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:27.147508  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.196830  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.197402  6515 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:27.149845  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.199061  6666 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:27.150444  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.149895  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.200349  6669 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:27.109297 13321 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.013s	sys 0.014s
I20260709 13:19:27.151461 13377 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:19:27.155540 13428 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.156088 13487 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:27.158864 13370 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:27.038992 13427 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.106783 13437 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.144444 13418 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.164829  7231 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:27.039820 13458 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.167675 13404 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:27.219229  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.168780 13389 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.171379  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.161006 13455 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.171860 13403 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.125046 13479 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:27.136900 13407 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:27.185359  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.177604  6668 raft_consensus.cc:2376] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19. Re-sending same reply.
W20260709 13:19:27.232484  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47220.5us
I20260709 13:19:27.200268  6671 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:27.201356  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.208992  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:27.212509  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:27.234086  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:27.152784  6589 raft_consensus.cc:2368] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18.
I20260709 13:19:27.195633 13496 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:27.215929  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71616.4us
I20260709 13:19:27.217931  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.234938 13496 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:27.207729 13271 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.222855  6590 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:27.224821  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:27.223076  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.226094  7228 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:27.239321  6669 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:27.185505 13464 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.228310  7236 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:27.229027  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.229841 13427 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:27.163312 13325 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.164372 13478 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.232642  7234 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:27.234364  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:27.244972  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:27.173071 13290 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.247774  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.235258  6666 raft_consensus.cc:2376] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19. Re-sending same reply.
I20260709 13:19:27.246523  6563 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.204272 13076 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:19:27.234463  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:27.249626  6594 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:27.218722 13418 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:27.243104  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:27.247044  6670 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:27.219233  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.250862  6515 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:19:27.259047  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.262953  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.263624  6671 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:27.231170 13479 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:27.265455 13473 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.265898 13377 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.265744 13418 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.227218 13454 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:19:27.266963 13256 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:27.295553  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:27.269052 13491 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.270124  6521 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:27.244822 13316 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:27.300169  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.237155 13389 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:27.270864  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.302680  6668 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:27.302750  6670 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:27.271173 13464 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.237967 13435 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:27.273303  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:27.266475 13325 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:27.281455  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56710.2us
I20260709 13:19:27.315215  7233 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:27.315846 13464 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:27.316437  6637 leader_election.cc:400] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21.
I20260709 13:19:27.293036  6519 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:27.316876  6637 leader_election.cc:403] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:27.317283  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.317772  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:27.297238  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:27.289697  6515 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:27.318498 13256 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:27.227283 13386 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:19:27.223230 13437 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.261664 13266 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.262315 13409 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:27.262559 13407 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:19:27.302480 13389 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:27.310871  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.363926  6668 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:27.244479 13271 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.365243  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:27.316357  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:27.318117  6670 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:27.298843  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.298856  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:27.367107  6666 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:27.319190  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:27.324193  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.368197  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.347205  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:27.271672 13408 raft_consensus.cc:2764] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:19:27.506816  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 159723us
I20260709 13:19:27.507719  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:27.509325 13441 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:19:27.313076 13508 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:19:27.369007 13266 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.512270  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.513792 13266 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.363639 13409 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:27.514474 13409 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:27.514917  6518 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:27.365145 13389 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.272993 13500 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:27.515944 13500 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:27.516948 13500 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.366706 13479 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.519479 13479 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:27.293583 13377 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.524684 13377 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.295985 13096 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.525960  6513 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 20: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:27.527019 13096 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.348162 13463 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:27.487167  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.527935 13091 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:27.307448 13316 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:27.535141 13316 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:27.538488  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.323673 13431 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:27.533838 13409 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.365123 13454 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:27.504917 13437 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:27.505400 13464 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.509982 13441 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:27.539561  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.538110  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.543812  6666 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:27.539619  6516 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:27.502893 13496 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.551247  6669 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:27.540365  6671 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:27.575721  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.576845  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.578394  6671 raft_consensus.cc:2376] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23. Re-sending same reply.
W20260709 13:19:27.582145  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.588161  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.589808  6669 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
W20260709 13:19:27.569444 13504 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.276s	user 0.000s	sys 0.008s
I20260709 13:19:27.542537 13377 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.599249  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.600194  6666 raft_consensus.cc:2376] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23. Re-sending same reply.
I20260709 13:19:27.604635  7237 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:27.609257  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.610697  6668 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:27.569294 13454 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:27.575460 13365 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.594547 13418 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.617027  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.606449 13091 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:27.599006 13491 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.606132 13408 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.581156 13464 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:27.624742  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 86932.9us
I20260709 13:19:27.623510  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.623382  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.631058  6671 raft_consensus.cc:2368] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
I20260709 13:19:27.622711  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.609792 13386 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.615520 13457 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:27.658517  6669 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.620051 13459 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:27.620282 13428 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.609333 13377 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.624521 13091 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:27.670046  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.670090  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.674398  6516 raft_consensus.cc:2410] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:19:27.679562 12568 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.680197 12568 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.683012  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.683140  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.686939  6669 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:27.664227 13290 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.693763  6522 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:27.695802  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.691522  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:27.656615 13408 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:27.686709 13474 consensus_meta.cc:287] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.001s
I20260709 13:19:27.702075 13474 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:27.702256  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.703104  6662 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:27.700337  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.672031 13271 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:27.705725 13271 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:27.708168 13465 consensus_meta.cc:287] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.006s
I20260709 13:19:27.708596 13091 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.710012 13091 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.710064  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:27.711104 13507 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:19:27.711465  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:27.699394 13510 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:19:27.714681  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.714821  6516 raft_consensus.cc:2368] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:27.664745 13386 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.717213  6671 raft_consensus.cc:2368] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:19:27.669081 13389 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.696823 13496 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.672087 13516 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:19:27.671936 13459 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.719684  6662 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:27.722795  6522 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:27.718876 12568 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.722273 13408 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.693476 13431 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:19:27.669783 13457 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.662979 13454 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.731480  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.732118  6522 raft_consensus.cc:2410] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 15 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 13:19:27.725458  6662 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:27.726038  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.720438 13256 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21.
I20260709 13:19:27.724668 13409 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.728951 13231 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.715301 13458 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.715212 13518 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:27.735590  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.739651  6522 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:27.708608 13465 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.740546 13454 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.741784 13518 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:27.742480 13457 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.742882 13518 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.744184 13459 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.746126  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2800) took 1034 ms. Trace:
W20260709 13:19:27.746733  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.746871 13464 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.747383  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.749043 13389 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.752504 13516 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:27.752945 13516 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:27.754676  6514 rpcz_store.cc:276] 0709 13:19:26.711998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.723631 (+ 11633us) service_pool.cc:224] Handling call
0709 13:19:26.730285 (+  6654us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.736287 (+  6002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.736316 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.736897 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.854475 (+117578us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:26.854515 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.745569 (+891054us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b5400044554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:27.745644 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.745709 (+    65us) raft_consensus.cc:1685] finished
0709 13:19:27.745729 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.746034 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10289,"raft.queue_time_us":4479,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"spinlock_wait_cycles":2766208,"thread_start_us":100434,"threads_started":1,"wal-append.queue_time_us":172795}
I20260709 13:19:27.757879  6671 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:19:27.766536  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.777269  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.777369  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.778128  6516 raft_consensus.cc:2376] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20. Re-sending same reply.
I20260709 13:19:27.778343  6522 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:27.779549  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.780526  6522 raft_consensus.cc:2410] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 14 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 13:19:27.752561  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:27.786692  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.788548  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.799937  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.800944  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:27.802984  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 104977us
I20260709 13:19:27.805405  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.805488  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.806365  6522 raft_consensus.cc:2410] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 18 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:19:27.807094  6486 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.808599  6486 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.809602 12568 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:27.810191 12568 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:27.811087 12568 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.812054  6670 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:27.813165  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.813877  6514 raft_consensus.cc:2393] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:27.813717  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.823926  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.825356  6516 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:27.817337 13389 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:27.817492 13479 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:27.818531 13290 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:27.816910 13377 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.753804 13516 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.829042 13389 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.832726 13290 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.857432 13290 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:27.869050 13290 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.835834 13479 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:27.851979 13389 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:27.888530 13465 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.895748  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.897405  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.898100  6664 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.912390 13290 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.922981 13091 raft_consensus.cc:476] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Not starting pre-election -- already a leader
I20260709 13:19:27.924151 12568 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:27.924728 12568 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.926573 12568 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.927016 12568 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:27.927307  6513 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:27.927423 12568 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.928560  6513 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:19:27.930006 13446 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.662s	user 0.004s	sys 0.010s
I20260709 13:19:27.930874  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:27.931257  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1460) took 2881 ms. Trace:
I20260709 13:19:27.931437  7232 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:27.931617  6597 rpcz_store.cc:276] 0709 13:19:25.049484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.054988 (+  5504us) service_pool.cc:224] Handling call
0709 13:19:25.066841 (+ 11853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:25.067183 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:25.067212 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:25.152877 (+ 85665us) log.cc:844] Serialized 26 byte log entry
0709 13:19:25.370376 (+217499us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:25.370425 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.370497 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.929473 (+2558976us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:27.930971 (+  1498us) raft_consensus.cc:1685] finished
0709 13:19:27.930991 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.931181 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":6127,"spinlock_wait_cycles":210971008,"thread_start_us":30103,"threads_started":1,"wal-append.queue_time_us":834543}
I20260709 13:19:27.932672  7232 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:27.935427 13441 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:27.935818 13441 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:27.936606 13441 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:27.937398  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.937564 13441 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:27.929184 12568 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.881289 13386 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:27.943585 13316 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.947312 13479 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:27.948288  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.949148  6665 raft_consensus.cc:2393] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:27.922432  6518 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:27.950052  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.953760  6665 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.955276  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:27.957434  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.959276  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.961018  6664 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:27.971300  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.971560  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:27.974366  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.973877  6664 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:27.976289  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:27.977272  6664 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:27.946411 13316 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:27.976409  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.981138  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:27.982218 12568 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:27.982740 12568 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:27.983222 12568 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:27.985987  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:27.959564 13437 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:27.978530  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:27.993602  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:27.979099 13326 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:27.995036  6664 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:27.984350 13464 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:27.984867 13290 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:27.993077 13437 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.010093 13290 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.019651 13290 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:28.001653 13464 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:28.021567 13464 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:28.022802 13437 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.993769 13460 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:19:28.027618 13460 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:28.028074 13460 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:28.029062 13460 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:27.997128 13326 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.030948 13326 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:28.052232 13481 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:28.052657 13481 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:28.030521 13460 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:28.053471 13481 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.050323  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:28.055217  6517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 3014) took 1026 ms. Trace:
I20260709 13:19:28.055658  6662 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:28.055595  6517 rpcz_store.cc:276] 0709 13:19:27.029137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.144371 (+115234us) service_pool.cc:224] Handling call
0709 13:19:28.055033 (+910662us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b5400111054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.055165 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75804544}
I20260709 13:19:28.062903 13481 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:28.058413  6517 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:19:28.099421  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.101123  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:28.102559  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:28.106837  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.107604  6662 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.112983  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1386) took 1004 ms. Trace:
W20260709 13:19:28.117178  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 170908us
I20260709 13:19:28.113547  6516 rpcz_store.cc:276] 0709 13:19:27.108176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.830465 (+722289us) service_pool.cc:224] Handling call
0709 13:19:27.834292 (+  3827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.834435 (+   143us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:27.956411 (+121976us) log.cc:844] Serialized 14 byte log entry
0709 13:19:28.112544 (+156133us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:28.112613 (+    69us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:28.112673 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.112760 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.112913 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":166861,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"thread_start_us":263923,"threads_started":2}
I20260709 13:19:28.128506  6664 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
W20260709 13:19:28.119313  6561 connection.cc:256] RPC call timeout handler was delayed by -1.0162860039999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 13:19:28.133826  6561 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.833s (SENT)
I20260709 13:19:28.134353  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:28.139297  6664 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:28.140714  6664 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:28.144433  6662 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:28.149503 12568 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.126421 13389 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:28.150851 13530 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:19:28.152033 13389 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:28.152711 13478 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:28.154827  6665 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:28.160125 13437 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.124150 13377 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:19:28.145437 13316 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:28.175203  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.182194  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:28.180879 13454 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.202843 13454 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.204331  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:28.205238 13316 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:28.205435 13454 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.205688 13316 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.206146 13316 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:28.212905  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1384) took 1184 ms. Trace:
I20260709 13:19:28.213060  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:28.213598  6519 rpcz_store.cc:276] 0709 13:19:27.028246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.028484 (+   238us) service_pool.cc:224] Handling call
0709 13:19:27.103781 (+ 75297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.317042 (+213261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.212610 (+895568us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400354b54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:28.212828 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434744704}
I20260709 13:19:28.214123 13409 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:28.216796  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.217444  6597 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:28.219318  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1745) took 1157 ms. Trace:
I20260709 13:19:28.219808  7237 rpcz_store.cc:276] 0709 13:19:27.061703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.061925 (+   222us) service_pool.cc:224] Handling call
0709 13:19:28.219094 (+1157169us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54006cc354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.219249 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149854592}
I20260709 13:19:28.220953 13316 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.222144  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:28.212841 13326 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.223174 13474 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:28.223683 13474 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.224350  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:28.224539 13474 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.225073 13345 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:28.225945 13326 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:28.232842  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:28.233820  6591 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:28.236653 12568 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.239331  6591 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:28.242200  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:28.206197 13454 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:28.243755  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.244108  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.246973 12568 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:28.247509 12568 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.247905 12568 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.237476 13389 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:19:28.185921 13231 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.242964 13464 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.184723 13386 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.266670  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.276432  6594 raft_consensus.cc:2410] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:28.236524 13409 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.245827 13418 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.020s
I20260709 13:19:28.239197 13479 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:28.281510  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 178756us
I20260709 13:19:28.243726 13491 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:28.289979  6486 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:28.291900  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.292687  6591 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:28.292794  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.293236  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.294059  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.294379  6518 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:28.299602  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.299834  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:28.300730  6593 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 13.
I20260709 13:19:28.303637  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:28.312674 13479 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:28.313977 13479 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.291031 13409 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.317016 13464 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.316294 13479 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:28.276036 13231 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.321544 13231 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:28.323094  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.327801  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.328510  6665 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.339797  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.340701 13409 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.342988  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 3082) took 1144 ms. Trace:
I20260709 13:19:28.343484  6670 rpcz_store.cc:276] 0709 13:19:27.198385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.384192 (+185807us) service_pool.cc:224] Handling call
0709 13:19:28.342781 (+958589us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54000c5dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.342919 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386429696}
I20260709 13:19:28.344826  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.345446  6670 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:28.340766  6591 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:28.348065 13471 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:28.348351  6591 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:19:28.348909  6591 pending_rounds.cc:85] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 14
I20260709 13:19:28.349159  6591 pending_rounds.cc:107] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:19:28.349502  6591 raft_consensus.cc:2889] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:28.349721 13497 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.372s	user 0.000s	sys 0.004s
I20260709 13:19:28.350920 13412 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:28.351359 13412 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:28.351648  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.352263  6591 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.352210 13412 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:28.353749 13412 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:28.355531  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.356211  6588 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.356855  6670 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:28.361371  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:28.348313  6665 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:28.306545 13290 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:28.364526  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1886) took 1339 ms. Trace:
I20260709 13:19:28.364902  7228 rpcz_store.cc:276] 0709 13:19:27.025629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.025795 (+   166us) service_pool.cc:224] Handling call
0709 13:19:28.364374 (+1338579us) spinlock_profiling.cc:243] Waited 83.7 ms on lock 0x7b54007a0154. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.364471 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192506752}
I20260709 13:19:28.365154  6588 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:28.310288 13316 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.331404 13491 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:28.290997 13386 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.368216 12568 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:28.372875  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.374784  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1340) took 1067 ms. Trace:
I20260709 13:19:28.376096  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:28.380986  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 138875us
I20260709 13:19:28.389649  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.390547  6665 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:28.386659  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:28.387504  6664 rpcz_store.cc:276] 0709 13:19:27.307079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.147496 (+840417us) service_pool.cc:224] Handling call
0709 13:19:28.147764 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.147939 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:28.158587 (+ 10648us) log.cc:844] Serialized 14 byte log entry
0709 13:19:28.374480 (+215893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:28.374526 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:28.374564 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.374638 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.374733 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8670,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":3835,"thread_start_us":208164,"threads_started":1}
I20260709 13:19:28.368947 13290 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.407073  6667 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:28.402176  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.404574  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:28.410535  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.411094  6595 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.401674 13464 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:28.416018  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.416620 12568 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.373624 13491 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.429394 13491 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:28.407914 13465 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:19:28.432988  7231 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:28.417248 13470 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:28.434268 13470 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.435596 13470 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.401340 13437 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:28.427774 13474 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:28.406741 13389 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:19:28.439924 13389 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:28.440459 13389 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.440806 13386 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.441504 13389 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.442222  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.443357 13290 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.471246  6596 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:28.445147  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.484823  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1339) took 1210 ms. Trace:
I20260709 13:19:28.486136  6595 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:28.487455  6513 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:28.485502  6662 rpcz_store.cc:276] 0709 13:19:27.282045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.145563 (+863518us) service_pool.cc:224] Handling call
0709 13:19:28.145861 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.146085 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.146116 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.146910 (+   794us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.385167 (+238257us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:28.385216 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.385307 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.484593 (+ 99286us) raft_consensus.cc:1685] finished
0709 13:19:28.484613 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.484759 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":237966,"threads_started":1,"wal-append.queue_time_us":335846}
W20260709 13:19:28.489444 13549 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.084s	user 0.001s	sys 0.004s
I20260709 13:19:28.494652  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:28.500299  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:28.505283  6665 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:19:28.488535 13537 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.156s	user 0.002s	sys 0.005s
I20260709 13:19:28.511878  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.517745  6662 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:28.517905  6670 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.519009  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.520841  6662 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.523049  6591 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:28.516949  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.528818  6596 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.529469  6588 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:28.515854 13324 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:28.516366  6666 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:28.533195  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:28.542306  6670 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:28.547329  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:28.547736  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3085) took 1345 ms. Trace:
I20260709 13:19:28.549234  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:28.514542  6664 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:28.551019  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.519352  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:28.554008  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.497831 13370 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.548256  6668 rpcz_store.cc:276] 0709 13:19:27.201902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.121964 (+920062us) service_pool.cc:224] Handling call
0709 13:19:28.137248 (+ 15284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.141555 (+  4307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.141586 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.143409 (+  1823us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.288599 (+145190us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:28.289099 (+   500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.289173 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.547468 (+258295us) raft_consensus.cc:1685] finished
0709 13:19:28.547495 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.547671 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4378,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":2589,"thread_start_us":108360,"threads_started":1,"wal-append.queue_time_us":231853}
I20260709 13:19:28.557217  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.498974 13389 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.553745 13324 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.559046 13326 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:28.562510 13324 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:28.564539 13326 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.565138 13326 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:28.568041 13455 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:28.568681 13455 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:28.570097  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:28.562958  6520 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2794) took 2085 ms (client timeout 1797 ms). Trace:
W20260709 13:19:28.570952  6520 rpcz_store.cc:269] 0709 13:19:26.477627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.477749 (+   122us) service_pool.cc:224] Handling call
0709 13:19:28.562725 (+2084976us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b540007dad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.562872 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934670592}
I20260709 13:19:28.569782 13455 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.573135 13455 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:28.580863  6521 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48592 (request call id 3024) took 1413 ms (client timeout 1505 ms). Trace:
W20260709 13:19:28.581329  6521 rpcz_store.cc:269] 0709 13:19:27.167801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.208301 (+ 40500us) service_pool.cc:224] Handling call
0709 13:19:28.580652 (+1372351us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b540007dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.580798 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115392896}
I20260709 13:19:28.591526  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.592351  6588 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:28.593998  6664 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:19:28.553259  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:28.546303 13437 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:28.603183  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.604128  6588 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:19:28.605376  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52194.7us
I20260709 13:19:28.598820  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.608316  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.608348  6662 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:28.631995  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.632838  6588 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.527701 13441 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:19:28.639045  6596 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:19:28.644202  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 257645us
I20260709 13:19:28.645889  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:28.653540  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:28.634501 13060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.656s	user 0.029s	sys 0.025s
I20260709 13:19:28.583437 13370 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.654167  6596 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:28.691035  7230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1658) took 1871 ms (client timeout 1941 ms). Trace:
I20260709 13:19:28.659029  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.763229  6590 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:28.741443  6588 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
W20260709 13:19:28.702994  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.640795 13170 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:28.762984  7230 rpcz_store.cc:269] 0709 13:19:26.819231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.819649 (+   418us) service_pool.cc:224] Handling call
0709 13:19:28.690830 (+1871181us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400170a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.690964 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326837632}
W20260709 13:19:28.687932  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47528.5us
I20260709 13:19:28.702564  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:19:28.692086  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 2967) took 2190 ms (client timeout 1738 ms). Trace:
W20260709 13:19:28.776093  6589 rpcz_store.cc:269] 0709 13:19:26.501705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.501843 (+   138us) service_pool.cc:224] Handling call
0709 13:19:28.691827 (+2189984us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b5400080f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:28.691999 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455699328}
W20260709 13:19:28.776980  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 74524.5us
I20260709 13:19:28.709000  6591 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.654332  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:28.702127  6561 leader_election.cc:336] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 2.273s (SENT)
W20260709 13:19:28.744156  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 344622us
W20260709 13:19:28.705732 13547 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:28.767942  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.705173 13121 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:28.743517 13412 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:28.660763 13548 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:28.763459 13060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.786s	user 0.031s	sys 0.025s
I20260709 13:19:28.670713 13441 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:28.787467  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.696795  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:28.827581  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.808866  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:28.697062 13331 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:28.743702 13386 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.654407 13324 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.765123 13345 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.790666  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 136275us
I20260709 13:19:28.644182 13320 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.141s
I20260709 13:19:28.832115  6590 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:28.871448  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:28.653721 13115 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.832664  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51542 (request call id 1342) took 1461 ms. Trace:
I20260709 13:19:28.745816 13455 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:28.794799  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:28.877746  6590 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:28.877939  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 83237.3us
I20260709 13:19:28.875561  6669 rpcz_store.cc:276] 0709 13:19:27.370959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.696438 (+1325479us) service_pool.cc:224] Handling call
0709 13:19:28.832603 (+136165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:28.832563  6589 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:28.787750 13170 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:28.804418  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 392484us
I20260709 13:19:28.793437 13471 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:28.805047 13437 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:19:28.845661  6591 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:28.795799 13290 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:28.839700 13479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.038s
I20260709 13:19:28.857962  6596 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:28.677574 13326 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.836045 13441 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:28.797642 13456 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:19:28.867378  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:28.886086  6590 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:19:28.872496  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 170389us
I20260709 13:19:28.834019 13548 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.891710  6664 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:28.873965  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.840976  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:28.856642  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.882380  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.879650 13115 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.805440 13460 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 13:19:28.842105 13331 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:28.884804 13479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.038s
I20260709 13:19:28.805852 13412 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.854143 13386 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:28.898379  6487 connection.cc:256] RPC call timeout handler was delayed by -1.1320550259999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
I20260709 13:19:28.883209 13471 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.902642  6663 raft_consensus.cc:2410] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:28.873035 13345 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:28.881794 13455 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:28.911477  6589 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:28.912151  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:28.916086  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:28.912387  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:28.917970 13556 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:28.918604 13556 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:28.922574  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:28.923405  6514 raft_consensus.cc:1240] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:19:28.898720 13290 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.933383  6517 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:28.935704  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:28.942299  6515 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 20. Ops: []
I20260709 13:19:28.932171  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:28.943240  6513 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:28.932026  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:28.943681  6516 raft_consensus.cc:1240] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:19:28.944096  6519 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:28.910638 13471 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:28.907483 13480 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.949637  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 153271us
I20260709 13:19:28.945163  6521 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:28.901849 13464 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:28.932540 13320 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:28.960489  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:28.966662  6519 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:28.933338 13556 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:28.951894  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:28.965947 13437 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:19:28.978309  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:28.978765 13370 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:28.959815 13480 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:29.094020  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.975733 13316 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.119s
I20260709 13:19:29.102187  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:29.097239  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 145445us
I20260709 13:19:28.948390 13441 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:29.105532  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 223389us
I20260709 13:19:29.100144  6519 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:28.981951  6664 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:28.949843 13470 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:28.977259 13320 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:28.978981  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:28.913498 13170 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.139948  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:29.141356  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:29.107429  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3064) took 3175 ms. Trace:
I20260709 13:19:29.141983  6667 rpcz_store.cc:276] 0709 13:19:25.920819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.921888 (+  1069us) service_pool.cc:224] Handling call
0709 13:19:25.922191 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.407502 (+2485311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.095844 (+688342us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b54002bcd54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:29.096498 (+   654us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5714098432}
I20260709 13:19:28.966918 13548 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.106827 13324 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.149111  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:29.136318 13316 consensus_queue.cc:1243] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:29.154057  6589 raft_consensus.cc:2410] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:29.151304  6561 leader_election.cc:400] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 13 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 13. Current term is 14.
I20260709 13:19:29.161657  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:29.162881  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:29.163889  6593 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: [16.11-19.13]
I20260709 13:19:29.146409 13326 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.099198 13290 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.160372 13223 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:19:29.168104 13475 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:19:29.215760 13460 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:29.215987  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54398us
W20260709 13:19:29.215770 13563 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
W20260709 13:19:29.216351 13565 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:19:29.170511 13409 raft_consensus.cc:3055] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:19:29.217695 13561 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:19:29.218276 13412 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:29.219486 13544 log.cc:927] Time spent T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.585s	user 0.006s	sys 0.007s
I20260709 13:19:29.291334 13412 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:29.222046 13553 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.354s	user 0.000s	sys 0.009s
W20260709 13:19:29.223470 13551 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.356s	user 0.006s	sys 0.003s
W20260709 13:19:29.224825 13557 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.356s	user 0.004s	sys 0.004s
I20260709 13:19:29.237396 13516 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.242908  6667 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:29.243346  6662 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:29.244174  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2619) took 2355 ms. Trace:
I20260709 13:19:29.264151 13345 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:29.249722  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.313771  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.106428 13464 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:29.315253 13567 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:19:29.251547 13470 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.318028 13470 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.319672  6519 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:29.321363 13470 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.251782 13558 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.322486 13558 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.325384 13558 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.253082 13370 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:29.254423 13480 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.256050  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:29.328461  6522 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:29.257897  6520 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:29.163242 13441 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:29.277127  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.332963 13456 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:29.333526 13456 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:29.161403 13455 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.169744 13256 raft_consensus.cc:2764] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:19:29.278079  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:29.279837  6561 leader_election.cc:400] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:19:29.279925 13571 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.280730 13223 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:29.281282 13572 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:29.282056 13575 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:29.282579 13460 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.275089 13409 raft_consensus.cc:740] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:29.284595  6517 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:29.291805 13412 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:29.214385 13331 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:29.167936 13569 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:29.309571  6663 rpcz_store.cc:276] 0709 13:19:26.888962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.889156 (+   194us) service_pool.cc:224] Handling call
0709 13:19:29.243967 (+2354811us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b540012a054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:29.244117 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4601966336}
W20260709 13:19:29.245008  7231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1668) took 1968 ms (client timeout 1597 ms). Trace:
I20260709 13:19:29.335438  6516 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:29.172222 13316 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.140s
W20260709 13:19:29.336058  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57020.1us
W20260709 13:19:29.352133  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 379021us
I20260709 13:19:29.294138 13386 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.338802 12568 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:19:29.274626 13326 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:19:29.344152  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:29.339337 13412 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.336316 13571 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.336480 13223 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.337854 13370 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:19:29.344439  7231 rpcz_store.cc:269] 0709 13:19:27.276804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.277008 (+   204us) service_pool.cc:224] Handling call
0709 13:19:28.361235 (+1084227us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ceee 0000000000375939 00007f04bdfd01e0 00007f04be0990c4 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:29.244842 (+883607us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b54004de254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:29.244946 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168452224}
I20260709 13:19:29.349133  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:29.366091  6667 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:29.337623 13460 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.340605  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1471) took 1123 ms. Trace:
I20260709 13:19:29.366379  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:29.366950  6597 rpcz_store.cc:276] 0709 13:19:28.226436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.226559 (+   123us) service_pool.cc:224] Handling call
0709 13:19:28.226896 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.235629 (+  8733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.235672 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.239881 (+  4209us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.639567 (+399686us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:28.639606 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.639688 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.222817 (+583129us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:29.340312 (+117495us) raft_consensus.cc:1685] finished
0709 13:19:29.340335 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.340546 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8310,"raft.queue_time_us":8916,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7265,"spinlock_wait_cycles":26864384,"thread_start_us":351033,"threads_started":1,"wal-append.queue_time_us":593464}
I20260709 13:19:29.358953  6522 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:19:29.362870  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 446816us
W20260709 13:19:29.372872  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.375157  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:29.378067 12568 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:29.378540 12568 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.379108 12568 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:29.381230  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.365597 13223 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.334599 13456 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:29.387871  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.368489 13316 consensus_queue.cc:1243] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:29.365345 13370 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:29.389016  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 410099us
I20260709 13:19:29.402511  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:19:29.407150  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.363441 13558 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:19:29.415866  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 313748us
I20260709 13:19:29.416900  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:29.418560  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:29.395747  6488 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.738s (SENT)
W20260709 13:19:29.397835  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.308254 13320 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.431478  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:29.338124 13409 consensus_queue.cc:260] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.374284 13386 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:19:29.403981 13412 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:29.443267 13470 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:29.443543  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47850.6us
I20260709 13:19:29.446934  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:29.410475 13316 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:29.458480  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:29.450883 12568 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.460110  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1398) took 1488 ms. Trace:
I20260709 13:19:29.460495  6518 rpcz_store.cc:276] 0709 13:19:27.972200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.027945 (+ 55745us) service_pool.cc:224] Handling call
0709 13:19:28.028272 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.302003 (+273731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.459927 (+1157924us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54001e8a54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:29.460063 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611425024}
W20260709 13:19:29.466348  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 320743us
I20260709 13:19:29.468052  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:29.442153 13331 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:29.481160  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 62695.8us
I20260709 13:19:29.482422  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:29.486112  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:29.486750  6516 raft_consensus.cc:2410] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:19:29.487555 12568 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.489688  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:29.490394  6670 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:29.408262 13576 consensus_queue.cc:1059] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:19:29.457666 13455 consensus_queue.cc:1059] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:19:29.481860 13256 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:29.433977  6597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:29.516721  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:29.517261  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.517686  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.511371  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 358922us
I20260709 13:19:29.523401  7198 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:29.497080 13316 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:29.512724  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:29.539880  6637 leader_election.cc:400] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:29.540247  6637 leader_election.cc:403] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:19:29.541545  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:29.529677  6670 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:29.512079 13589 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:29.543877  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:29.549124  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:29.535636 13316 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:29.545864 13170 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:29.484707 13370 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.538414 13256 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.570919  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:29.578284  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:29.545023 13345 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:29.545818 13331 raft_consensus.cc:2764] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:19:29.546523 13589 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:29.546473 13312 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:29.548322 13548 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:29.571152 13316 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:29.551257 13582 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:29.582011 13584 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:29.587791 13568 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.649153 13568 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:29.651959 13593 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:19:29.600442 13578 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:19:29.574111 13569 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:19:29.597714 13312 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:29.656179 13312 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.588886 13589 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:29.668177  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1492) took 1028 ms. Trace:
I20260709 13:19:29.578828 13271 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.594348  6662 raft_consensus.cc:2410] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:29.585088 13370 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.579162 13577 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:19:29.601257  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:29.551011 13460 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:29.582778 13437 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:29.591090 13170 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:29.596809 13562 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.731s	user 0.008s	sys 0.003s
I20260709 13:19:29.590591 13345 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.607201 13548 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:29.590888 13592 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:29.577976 13256 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:29.631515  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.670310  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:29.718158  6516 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:29.577286  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:29.672917  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:29.718943  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:29.675722  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:29.677652  6597 raft_consensus.cc:606] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:29.720125  6597 raft_consensus.cc:614] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:29.720177  6522 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:29.694157  6593 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:29.719996  6588 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.676396  6590 rpcz_store.cc:276] 0709 13:19:28.639565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.894305 (+254740us) service_pool.cc:224] Handling call
0709 13:19:28.902474 (+  8169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.902741 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:29.302207 (+399466us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.667823 (+365616us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:29.667879 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:29.667926 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.667997 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.668119 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56715,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":1281,"raft.queue_time_us":66637,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":21226,"thread_start_us":744208,"threads_started":3,"wal-append.queue_time_us":277401}
I20260709 13:19:29.722040  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:29.723244  6589 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:29.678805 13586 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.725381 13115 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.725155 13223 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.693768 13271 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.727754 13223 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.730472  6595 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:19:29.730927  6595 pending_rounds.cc:85] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 12
I20260709 13:19:29.731178  6595 pending_rounds.cc:107] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:19:29.731434  6595 raft_consensus.cc:2889] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.732911 13256 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.733125  6595 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:29.739387  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 3000) took 1333 ms. Trace:
I20260709 13:19:29.738543 13571 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.740262  6590 raft_consensus.cc:3055] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:29.740739  6590 raft_consensus.cc:740] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:29.741125  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:29.741892 13463 raft_consensus.cc:2764] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:19:29.741662  6590 consensus_queue.cc:260] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.743043  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:29.727751 13271 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.743839  6593 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:29.742316  6488 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:29.744025 13324 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.743729 13548 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.744560 13324 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.746102  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:29.748248 13437 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:29.717980 13577 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:29.740078  6591 rpcz_store.cc:276] 0709 13:19:28.405787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.607720 (+201933us) service_pool.cc:224] Handling call
0709 13:19:29.739219 (+1131499us) spinlock_profiling.cc:243] Waited 99.9 ms on lock 0x7b54002f67d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:29.739328 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229743872}
I20260709 13:19:29.749179  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:29.753351  6590 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:29.761390  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.748850 13474 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:29.748683 13464 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:29.767104  6588 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:29.746590 13324 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.758167 13463 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:29.757938 13586 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.771296  6590 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:19:29.771677 13568 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:29.751610 13437 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.778810  6590 pending_rounds.cc:85] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 10
I20260709 13:19:29.779350  6590 pending_rounds.cc:107] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:19:29.779678  6590 raft_consensus.cc:2889] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.784327 13464 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:29.785001 13464 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:29.787496 13576 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:29.788209 13576 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.790982 13474 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:29.792511 13474 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.723333 13345 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:29.797634  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.799285 13115 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.799911 13115 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.800360 13456 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:29.803021 13441 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:29.807092 13468 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:29.807679 13468 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:29.808655 13468 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.810766 13468 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:29.822942  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53967us
I20260709 13:19:29.802333 13115 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:29.825829  6597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
W20260709 13:19:29.826591 13554 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.964s	user 0.000s	sys 0.014s
I20260709 13:19:29.826715  6597 raft_consensus.cc:606] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:29.827416  6597 raft_consensus.cc:614] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:29.827927  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1482) took 1541 ms. Trace:
I20260709 13:19:29.828301  6594 rpcz_store.cc:276] 0709 13:19:28.286534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.286676 (+   142us) service_pool.cc:224] Handling call
0709 13:19:28.303093 (+ 16417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.320527 (+ 17434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.320579 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.321663 (+  1084us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.591296 (+269633us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:28.591367 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.707328 (+115961us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b540034fb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:28.707411 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.825964 (+1118553us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:29.827704 (+  1740us) raft_consensus.cc:1685] finished
0709 13:19:29.827734 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.827868 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2251,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":1379,"spinlock_wait_cycles":77004928,"thread_start_us":269070,"threads_started":1,"wal-append.queue_time_us":517048}
W20260709 13:19:29.839480 13471 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.870s	user 0.011s	sys 0.006s
W20260709 13:19:29.839860  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.839983 13471 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.844406 13312 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.847136 13345 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.823243 13326 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.855250 13326 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:29.857229  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.857810 13326 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.859901 13592 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:29.844828 13316 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:29.811076 13576 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.812597  6663 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:29.812716  6668 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:29.842350 13471 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.813593  6670 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:29.837323 13582 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:29.882037 12568 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:29.813303  6666 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:19:29.882946  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58465.3us
I20260709 13:19:29.872799  6589 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:29.894775 12568 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:29.900130  6589 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:29.885375 13470 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:19:29.896863 13312 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:29.897778 13316 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:29.924263  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 346045us
I20260709 13:19:29.929832  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:29.898605 13060 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:19:29.934381  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.935964 13582 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:29.934170 13320 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:19:29.939459 13586 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:29.942129  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3084) took 2740 ms. Trace:
W20260709 13:19:29.975378  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.943152  6671 rpcz_store.cc:276] 0709 13:19:27.201581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.759324 (+557743us) service_pool.cc:224] Handling call
0709 13:19:27.792005 (+ 32681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.792530 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:27.851554 (+ 59024us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.515058 (+1663504us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b18004bf940. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:19:29.796317 (+281259us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.796779 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.796901 (+   122us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:29.796932 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.797009 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.941784 (+144775us) raft_consensus.cc:1685] finished
0709 13:19:29.941901 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.942057 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91660,"prepare.queue_time_us":102090,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":106383,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":2607,"spinlock_wait_cycles":46739712,"thread_start_us":217141,"threads_started":2,"wal-append.queue_time_us":2010685}
I20260709 13:19:29.979466 13582 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:29.938338 13464 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.987380 13589 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.984373 13460 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:29.990059 13460 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:29.990598  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.991039 13460 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.991622 13320 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:29.992295 13320 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.992220 13460 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.995174 13320 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:29.997591  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.985634 13582 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:29.993036 13412 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:29.999107 13412 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:30.002219  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.002346 13412 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:30.013456 13463 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:30.014078 13463 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.014690 13558 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:30.015379 13558 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:30.016417 13463 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:30.023420  6636 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.548s (SENT)
I20260709 13:19:29.990140 13589 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:30.038601  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 461362us
W20260709 13:19:30.040568  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 221392us
I20260709 13:19:29.933118  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:30.064615  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 131588us
I20260709 13:19:30.066533  6637 leader_election.cc:400] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20.
W20260709 13:19:30.066367  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 340663us
I20260709 13:19:30.067699  6662 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:19:30.068727  6637 leader_election.cc:403] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.072726  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:30.073431  6666 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:30.060019 13470 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:19:30.078588 13575 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:19:30.044376 13463 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.098249  6669 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:30.100069  7230 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:30.101461  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:30.101693  7230 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:30.103451  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.105124  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.106439 12568 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:30.117676  7228 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:30.043629 13555 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:19:30.102735  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:30.116695  7234 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:30.072878 13606 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:30.133958  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.119481  6667 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:30.107081 13456 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:19:30.127351  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.127630  7229 raft_consensus.cc:1240] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:19:30.106709 13572 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:30.092427 13575 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.104401 13463 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.088052 13345 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:19:30.133363  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.133984  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.140904  7232 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:30.152132  7230 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:30.152484  6667 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:19:30.152432  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.152930  6667 pending_rounds.cc:85] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 13
I20260709 13:19:30.142712  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.153167  6667 pending_rounds.cc:107] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:19:30.153425  6667 raft_consensus.cc:2889] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.153676  7234 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:19:30.147171  6666 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:30.149297  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.157840  7228 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:30.166010  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.167829  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.168677  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.169137  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.169570  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:30.135849 13480 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.011s	sys 0.037s
I20260709 13:19:30.105701 13569 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:19:30.176827  6522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f"
dest_uuid: "921c423406da4f289884afca51ca198f"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:41994
I20260709 13:19:30.177416  6522 raft_consensus.cc:606] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:30.177855  6522 raft_consensus.cc:614] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:30.138401 13558 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:30.173473  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:30.174578  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.170083  6518 raft_consensus.cc:2376] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18. Re-sending same reply.
I20260709 13:19:30.179877  7231 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:30.184129  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.171928  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:30.144260 13589 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:30.179275  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:30.181660  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:30.189883  6517 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:30.172371  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:30.191183  6519 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:30.142573 13572 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.180999  7232 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:30.178257  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 82106.8us
I20260709 13:19:30.185036  6520 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:30.171729  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 105329us
I20260709 13:19:30.182298  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:30.183039  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:30.179168 13607 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20.
I20260709 13:19:30.195693  6637 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:30.196022  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:30.196667  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.197240  7230 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:30.196059  6637 leader_election.cc:403] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.189567  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:30.197543  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:30.178368 13480 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.012s	sys 0.037s
I20260709 13:19:30.207181  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.219331  7230 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:30.207553  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.202831  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:30.221299  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.220387  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.225435  6665 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.225790  7230 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:30.225840  7232 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
W20260709 13:19:30.228063  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.228474  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:30.227720  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.229095  6590 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:30.229418  7232 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:19:30.230163  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:30.236477  6594 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:30.236675  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.237218  6591 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:30.195222 13589 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.196621 13572 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:30.181751 13464 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:30.260283  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45657.3us
I20260709 13:19:30.263120 13312 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:30.295209  7233 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 15. Current term is 17. Ops: []
W20260709 13:19:30.279767  6562 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.574s (SENT) [suppressed 18 similar messages]
I20260709 13:19:30.264863  7232 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:30.265087  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50814.5us
I20260709 13:19:30.149418 13575 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.299793  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:30.221061 13455 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:30.274488  7230 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:30.311900  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.201707 13601 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.297390  6591 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:30.238160 13386 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:30.331111 13386 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:30.333745  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:30.334368  7232 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:30.335836  7229 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:19:30.336269  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.337625  6665 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:30.213025 13606 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.313004  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.269171 13625 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:30.321210  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.323657  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:30.326592 13609 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:19:30.320967 13331 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:30.265151 13115 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:30.271432 13589 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:30.340178  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.344175 13611 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:19:30.343218  7230 raft_consensus.cc:2410] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:19:30.340500  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:30.348568  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.367213  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.367547  7229 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:30.367787  7230 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:30.368275  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.368522  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.331648 13386 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:30.263269  6590 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:30.356380 12568 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:30.371186 12568 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.269506 13555 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:19:30.361454  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:30.362912  6592 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:19:30.277150 13562 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:30.342713  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.363265  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:30.339722 13455 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:30.372983  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:30.411270  6665 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:19:30.411695  6665 pending_rounds.cc:85] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 11
I20260709 13:19:30.411929  6665 pending_rounds.cc:107] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:19:30.412168  6665 raft_consensus.cc:2889] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.373454  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:30.376623  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:30.350546 13480 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.342294 13312 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:30.412827  6591 raft_consensus.cc:606] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:30.414889  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.421175  6591 raft_consensus.cc:614] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:30.414681  7230 raft_consensus.cc:1240] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:19:30.412799  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.424985  6663 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 13:19:30.415908  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:30.426081 12568 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:30.426800  6668 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:30.370077 13584 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:30.340644 13601 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.432384  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.455224  7230 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:30.455617  6591 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:19:30.412782 13345 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:30.411911 13621 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:30.459950 13409 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.462155  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3002) took 1049 ms. Trace:
I20260709 13:19:30.462663  6596 rpcz_store.cc:276] 0709 13:19:29.412566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.412728 (+   162us) service_pool.cc:224] Handling call
0709 13:19:29.447632 (+ 34904us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:29.694748 (+247116us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:29.755862 (+ 61114us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.178598 (+422736us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.178897 (+   299us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:30.189661 (+ 10764us) log.cc:844] Serialized 52 byte log entry
0709 13:19:30.189854 (+   193us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:30.190151 (+   297us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.190712 (+   561us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.197927 (+  7215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.198036 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.457200 (+259164us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b18003a5b20. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:30.461879 (+  4679us) raft_consensus.cc:1685] finished
0709 13:19:30.461910 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.462082 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29454,"prepare.queue_time_us":51972,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":9620,"raft.queue_time_us":115530,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":27575,"spinlock_wait_cycles":44092672,"thread_start_us":562089,"threads_started":2,"wal-append.queue_time_us":458197}
I20260709 13:19:30.437661 13455 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.441572  6520 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:30.423345 13614 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:30.350735 13625 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.443394  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:30.444242 13619 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:19:30.444583  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.486110  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.487141  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.445302 13463 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:30.422735 13480 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:30.447181 13620 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:19:30.447376 13312 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.448539 13474 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:30.468897  6594 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:19:30.469264  6592 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:30.469719  6589 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:19:30.512879  6589 pending_rounds.cc:85] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 12
I20260709 13:19:30.513211  6589 pending_rounds.cc:107] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:19:30.513504  6589 raft_consensus.cc:2889] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.516044  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:30.472545  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:30.477175  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:30.518100  6520 raft_consensus.cc:2393] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:30.518191  6591 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:30.478873  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:30.518949  6595 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:30.519213  7230 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:30.353544 13606 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.480924  6662 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:30.482085  6513 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:30.482154  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:30.493141  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.527668  7233 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:30.454324 13475 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:19:30.526974  6663 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:30.530854  6595 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:30.507004  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.510701  7236 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:30.533805  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:30.534483  7236 raft_consensus.cc:2376] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18. Re-sending same reply.
W20260709 13:19:30.541299  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.546830  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:30.548841  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:30.549016 13437 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:30.549662 13437 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:30.550637 13437 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.552791  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:30.554090  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.539233  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:30.555423  7232 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:30.556156  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:30.527734 13121 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.561512  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:30.560565  6669 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:30.562820  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:30.568121  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.572055  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.573669  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:30.494829 13558 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:19:30.574849  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:30.575686  6594 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:30.577982  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:30.531754 13582 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:30.579795  6597 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:30.580945  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:30.582230  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:30.585412  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:30.585669  6597 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:30.585206  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:30.586043  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 224614us
I20260709 13:19:30.590693  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:30.593516  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.604945  6561 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.933s (SENT)
I20260709 13:19:30.606299  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:30.495015 13627 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:19:30.613766  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:30.616077  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 283932us
I20260709 13:19:30.559753 13571 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:30.623812  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.527886 13471 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:19:30.489147 13614 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.628515  6597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:30.633857  6597 raft_consensus.cc:606] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:30.570664 13316 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:30.574589 13616 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:30.644451  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.645387  7229 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:30.584808 13587 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:30.558254 13588 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:19:30.614832 13312 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:30.570968 13607 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 13:19:30.574388 13331 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 13:19:30.626111 13606 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:30.625015 13582 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:30.669273  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.680989  7229 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:30.647356 13616 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.583741 13121 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.653506 13571 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:30.750914  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:30.754367  7237 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 14.
I20260709 13:19:30.666558 13575 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:30.777017 12568 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:30.777827 12568 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:30.782630  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 232646us
I20260709 13:19:30.783720  6636 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:30.736860 13616 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:30.761796 13474 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:30.783107  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 178347us
I20260709 13:19:30.761703 13555 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:30.823707  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 233122us
I20260709 13:19:30.785454 13576 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:30.828367  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:30.842412  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:30.841599  6590 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:30.869668 13480 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:30.785465 13572 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.870929 13601 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:30.820292 13555 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.809527 13474 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.843350 13582 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:30.962568  6594 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:30.960227  6590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
W20260709 13:19:30.983916  6561 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.552s (SENT)
I20260709 13:19:30.983286 13475 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:30.993355 13555 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:30.958640 13617 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
W20260709 13:19:30.986110 13386 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.007s
I20260709 13:19:30.924538 13601 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:31.001394  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:30.990471 13642 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:31.012396  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:31.018373  6588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.006s	sys 0.016s
W20260709 13:19:31.018940  6588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.007s	sys 0.016s
I20260709 13:19:31.038932  6590 raft_consensus.cc:606] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:31.039631  6590 raft_consensus.cc:614] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:31.047538  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:31.007019 13555 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:31.055306  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:31.059886  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.054673  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.055887 13475 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:31.015178 13625 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.048506 13464 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:31.048319 13571 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.049564 13587 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:31.009160 13386 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.072438  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.073895  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.013741 13601 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.071424 13649 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.068811 13555 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.082136  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.081885  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.068401 13614 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.099006  6592 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:31.099253  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.100251  6520 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:31.067091 13627 consensus_queue.cc:1059] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:19:31.101652 13627 raft_consensus.cc:3055] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:31.102034 13627 raft_consensus.cc:740] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:31.102885 13627 consensus_queue.cc:260] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.104154 13627 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:31.098074 13623 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:31.107079 13623 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:31.107208  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3124) took 1294 ms. Trace:
I20260709 13:19:31.107569 13623 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.107656  6664 rpcz_store.cc:276] 0709 13:19:29.812852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.813660 (+   808us) service_pool.cc:224] Handling call
0709 13:19:29.813987 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.814392 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:29.814676 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.265786 (+451110us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.343193 (+ 77407us) log.cc:844] Serialized 26 byte log entry
0709 13:19:30.343508 (+   315us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:30.343560 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.346480 (+  2920us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.106909 (+760429us) raft_consensus.cc:1685] finished
0709 13:19:31.106940 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.107126 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50113,"prepare.queue_time_us":2536,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":55725,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":26609,"thread_start_us":398936,"threads_started":1,"wal-append.queue_time_us":593991}
I20260709 13:19:31.109441 13386 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.076136 13587 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:31.110380 13589 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.003s
I20260709 13:19:31.110848 13589 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.112915  7234 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 21. Ops: []
I20260709 13:19:31.115271  7229 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:31.076387 13316 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:31.116747 13316 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:31.079347 13575 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:31.117633 13316 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:31.134934  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 61559.9us
W20260709 13:19:31.123175 13409 consensus_meta.cc:287] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.004s
I20260709 13:19:31.113459 13572 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:31.137665  7229 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:31.125392 13474 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:31.126085 13645 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:19:31.126686 13641 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:19:31.128700 13616 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.112341 13587 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.131892 13601 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.098894 13121 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.087095 13649 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.154944  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.155635  7229 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:31.156738 13649 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:31.157424  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.158480 13589 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:31.160948  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.177428  7231 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:19:31.180632  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.181393  6515 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
W20260709 13:19:31.184103  6561 leader_election.cc:336] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 3.236s (SENT)
I20260709 13:19:31.184711  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:31.186283 13575 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:31.187928 13575 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.158571  7229 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:31.193055  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.138525  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.140091  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:31.140875  6593 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:19:31.197423  6593 pending_rounds.cc:85] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 6
I20260709 13:19:31.197713  6593 pending_rounds.cc:107] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:19:31.197981  6593 raft_consensus.cc:2889] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:31.148778  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:31.205001  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 56335.1us
I20260709 13:19:31.151103  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.073204 13571 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.231559  7235 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:31.218495  7229 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:31.237674  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.210507  6520 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:31.238409  7235 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:31.229902  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:31.237407  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 98986.8us
I20260709 13:19:31.239981 13649 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:31.241393  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.244297  6563 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.245834 13386 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:31.245965  6563 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.246798 13601 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:31.204653 13412 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.238394  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.247999  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.231781 13121 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:31.207626 13587 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:31.237046 13616 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.254586  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:31.236476 13614 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:31.246569 13386 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:31.244778 13649 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:31.260172  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:31.253636 13412 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.264284 13409 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:31.264861 13409 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:31.265344 13409 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.251636 13601 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:31.259837 13607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.004s	sys 0.005s
I20260709 13:19:31.261492 13386 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.261718 13614 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:31.257601 13587 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.281062 13614 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.283999 13587 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:31.263129 13664 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:31.284847 13664 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.285334 13664 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.259004 13623 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.290882  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.267812 13649 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.295825 12568 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.298389 13636 consensus_queue.cc:1059] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:31.299158 13623 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.299355 13586 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:31.299494 13636 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:31.300019 13636 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:31.301179  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 517488us
I20260709 13:19:31.300973 13636 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.302428 13636 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:31.315092  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55022.3us
I20260709 13:19:31.320631  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:31.320657 13635 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.570s	user 0.000s	sys 0.004s
I20260709 13:19:31.327009  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1359) took 1152 ms. Trace:
I20260709 13:19:31.326856 13601 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.304289 13386 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.327508  6667 rpcz_store.cc:276] 0709 13:19:30.174324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.175217 (+   893us) service_pool.cc:224] Handling call
0709 13:19:30.175517 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:30.223900 (+ 48383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.227186 (+  3286us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:30.258370 (+ 31184us) log.cc:844] Serialized 52 byte log entry
0709 13:19:30.495379 (+237009us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:30.528523 (+ 33144us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.528727 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.528797 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.747773 (+218976us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4800220158. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f 00007f04be098f37 00007f04be0ad55c 00007f04be0ad508 00007f04be0ad488 00007f04be0ad443 00007f04be0ac6ec 00007f04bcfd6a96
0709 13:19:31.301239 (+553466us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:31.326699 (+ 25460us) raft_consensus.cc:1685] finished
0709 13:19:31.326729 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.326929 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":570883,"prepare.queue_time_us":33447,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":14795,"raft.queue_time_us":34764,"raft.run_cpu_time_us":3856,"raft.run_wall_time_us":534832,"spinlock_wait_cycles":65569280,"thread_start_us":155998,"threads_started":1,"wal-append.queue_time_us":330343}
I20260709 13:19:31.333666 13646 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.334120 13571 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:31.335912  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.336740  7236 raft_consensus.cc:2393] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:31.335840  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.334239 13646 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.342836 13646 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:31.343474 13651 raft_consensus.cc:3055] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:31.343943 13651 raft_consensus.cc:740] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:31.344919 13651 consensus_queue.cc:260] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.331427 13115 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.345682 13664 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.334983 13605 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.346444 13115 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.348212 13409 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.349344 13115 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.350490 13409 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.347988 13664 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.356560  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.357213  7236 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.357465  6594 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:19:31.361243  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.361884  7230 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.361716  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.366287  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.367040  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.367471  7229 raft_consensus.cc:2376] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 14. Re-sending same reply.
I20260709 13:19:31.369403  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.346345 13651 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:31.369774  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.377319  6594 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:31.388628  7235 raft_consensus.cc:2376] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 21. Re-sending same reply.
I20260709 13:19:31.390259  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.391674  7235 raft_consensus.cc:2376] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 18. Re-sending same reply.
I20260709 13:19:31.380287  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:31.392568  6520 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:31.377858  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:31.384609  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 556709us
I20260709 13:19:31.393005  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.380281  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.386041  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.379869 12568 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:19:31.390398  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:31.399061  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:31.400513  7230 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:31.403565 12568 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:31.401521  7229 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:19:31.404248 12568 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:31.404309  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:31.399677 13331 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.409920  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:31.384101 13474 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:19:31.383548 13627 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:31.425256  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:31.424228  6521 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:31.365304 13616 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:31.406607  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.427922  7235 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:19:31.402091  7236 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:31.428087  7228 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.430224  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:31.431783  6520 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:31.432173  6515 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:31.400228 13658 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.407732 13675 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:19:31.363672 13649 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.405777 13664 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:31.467403  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 73773.3us
I20260709 13:19:31.490386  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:31.474203  6519 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:31.419996 13605 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:31.423255 13670 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.435817 13555 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:31.501011  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:31.425727 13627 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.459831 12568 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:31.468281 13648 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.478s	user 0.000s	sys 0.005s
I20260709 13:19:31.455852 13409 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:31.428331 13616 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:31.512799  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.506137  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:31.507447  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:31.513482  6670 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.513756  6662 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.514986  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.519678  7228 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:31.515303  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.521162  6487 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.508618  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:31.477149 13586 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:31.498174 13115 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:31.512631  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.523452  6667 raft_consensus.cc:2368] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
I20260709 13:19:31.524611  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.525673  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.515439 13616 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.527714  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.522502  6663 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:31.529626  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:31.530036  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 226643us
I20260709 13:19:31.530325  6666 raft_consensus.cc:2368] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
I20260709 13:19:31.523135  6669 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 22: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:31.505968  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.532485  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:31.437107 13386 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.512158 13627 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:31.499670 13664 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.476809 13470 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:19:31.475175 13628 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 13:19:31.543144  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.510955 13605 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:31.552809  6670 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:31.523437 13586 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:31.512977 13676 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.550940  6669 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:31.515635 13555 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:31.552397  6596 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:31.549638  6662 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:31.529201 13475 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:31.573902 13475 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:31.575198  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.576140  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:31.576742  6663 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:31.577790  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.565137 13589 consensus_queue.cc:1059] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:19:31.576148 13605 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.511971 13670 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.582177 13670 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:31.560647  6666 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:31.561228  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:31.583344  6594 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:31.583583  6519 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:31.583808 13657 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:31.563844  6667 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:31.566733  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.585314 13605 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:31.567134  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.564718 13115 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.591279 13115 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.594138 13115 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.570578 13468 raft_consensus.cc:3055] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:31.572827 13555 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.574847 13475 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.469143 13658 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.610510 13658 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:31.557772  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:31.584211  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.528264 13409 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:31.613986  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:19:31.614554  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.615299  6590 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:31.614553 13409 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.552795 13572 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:31.574349 13412 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:31.602928 13614 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.602080 13468 raft_consensus.cc:740] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:31.619650  7236 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:31.547045 13312 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:19:31.621681 13586 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.598899  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.627151  6662 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:31.626991  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:31.576159 13664 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:31.630196  6668 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:31.632440 13464 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:19:31.624913  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73055.9us
W20260709 13:19:31.637223  6487 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575): Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.597s (SENT)
I20260709 13:19:31.631553 13468 consensus_queue.cc:260] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.639256  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:31.640761  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:31.643646  6519 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.646036 13610 raft_consensus.cc:2764] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:31.647737 13649 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.636092 13572 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:31.645200 13586 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.654942 13584 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:31.629189 13614 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.711369  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:31.713502  6521 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.707306  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:31.685508 13468 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:31.731566  6519 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:31.692646 13464 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.692390 13628 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:31.686098 13657 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:19:31.692905 13170 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:19:31.743873  6667 pending_rounds.cc:85] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 9
I20260709 13:19:31.744271  6667 pending_rounds.cc:107] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:19:31.744526  6667 raft_consensus.cc:2889] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:31.692129 13612 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:19:31.749934  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.759261  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:31.759861  6521 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:31.760485  6669 raft_consensus.cc:2376] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17. Re-sending same reply.
I20260709 13:19:31.762737  6594 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:19:31.764175  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.764933  6594 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:31.766007  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.771658 13664 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.735376 13312 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:31.781776  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.776079  6594 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:31.772365 13664 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.783797  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:31.789281  6663 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:31.786502  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.694228 13584 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.791359  6563 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:31.739138 13386 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.897264 13345 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:31.908461 13681 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:19:31.908555 13345 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.792438  6594 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:31.912107  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.912917  6594 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 14: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:31.850881  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 1914) took 1354 ms. Trace:
I20260709 13:19:31.852475 13474 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:31.849820  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:31.917129  6597 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:31.872367  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.800267 13678 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.741268 13628 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:31.797676 13605 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:31.779791 13470 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:19:31.773495 13435 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:31.855549  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.738189 13464 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.721395 13572 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.911188 13674 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.914399  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.914345  7231 rpcz_store.cc:276] 0709 13:19:30.496219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.613128 (+116909us) service_pool.cc:224] Handling call
0709 13:19:31.850714 (+1237586us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b540049d254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:31.850820 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259045632}
I20260709 13:19:31.943681  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.944293  6589 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:31.896450 13670 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:31.918745  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.926780 13678 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:31.931639  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.953167  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.934155  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3018) took 1705 ms. Trace:
I20260709 13:19:31.954200  6589 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:31.954067  6588 rpcz_store.cc:276] 0709 13:19:30.228426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.230727 (+  2301us) service_pool.cc:224] Handling call
0709 13:19:30.269853 (+ 39126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:30.489924 (+220071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.489969 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.490489 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:19:31.019387 (+528898us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:31.019438 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.933759 (+914321us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540019cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:31.933814 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.933874 (+    60us) raft_consensus.cc:1685] finished
0709 13:19:31.933886 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.934096 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2119,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":1279,"spinlock_wait_cycles":505049856,"thread_start_us":528526,"threads_started":1,"wal-append.queue_time_us":596526}
I20260709 13:19:31.916494 13409 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:31.947925  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:31.979291  6594 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
W20260709 13:19:31.900750  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 109483us
I20260709 13:19:31.982717  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.983675  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:31.984157  6519 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:31.985746  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:31.986330  7231 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:31.983335  7229 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:31.989499  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:31.937249 13569 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:31.926451 13474 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:31.974306  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 198535us
I20260709 13:19:31.963148  6597 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:31.994565  7229 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:31.995157  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:32.035373  6588 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:31.945029 13605 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:32.043635  6521 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:31.942054 13646 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:32.048272  7229 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:32.053623  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:32.056526  7231 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:32.058116 13682 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:32.059145  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:32.062196  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:32.063493 13682 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.064927 13682 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:32.066740  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:32.080165  7232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1697) took 1692 ms (client timeout 1597 ms). Trace:
I20260709 13:19:31.981274 13345 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:32.084322  7237 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:19:32.080637  7232 rpcz_store.cc:269] 0709 13:19:30.387457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.412562 (+ 25105us) service_pool.cc:224] Handling call
0709 13:19:32.079958 (+1667396us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54004de254. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:32.080094 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306345216}
I20260709 13:19:32.154192  6597 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:32.032099 13409 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:32.045466 13614 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:31.946174 13435 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.086171  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:32.050899 13677 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:31.953030 13674 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.975412 13670 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:31.994112 13672 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.168s
I20260709 13:19:32.077054 13601 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.200s
I20260709 13:19:32.304138  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:32.154770 13345 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:32.334282  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:32.334594  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:32.046157 13569 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:32.283198  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 216419us
W20260709 13:19:32.283106 13685 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
W20260709 13:19:32.342800  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.343004  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:32.299739 13331 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:32.307135 13456 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:32.313994 13664 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.081022 13646 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:32.338472  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 303690us
I20260709 13:19:32.387465  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:32.341349 13312 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:32.346061 13678 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:32.364635 13686 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.528s	user 0.000s	sys 0.001s
I20260709 13:19:32.395675 13456 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.305732 13677 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.284134 13474 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:32.428309  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 83682.1us
I20260709 13:19:32.429966  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:32.340437 13345 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:32.442731  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 94782.9us
W20260709 13:19:32.444983  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.409248 13628 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:32.448459  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:32.375069 13409 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.450301 13474 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:32.451391 13693 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:19:32.452770 13699 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:19:32.339046 13683 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.103s
W20260709 13:19:32.428507 13651 consensus_meta.cc:287] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.028s	user 0.011s	sys 0.002s
I20260709 13:19:32.354626 13700 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.107s
I20260709 13:19:32.460134  6561 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:32.417073 13694 raft_consensus.cc:2764] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:19:32.447100 13614 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:32.355832 13605 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.466328 13571 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:32.455741 13572 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:32.499274  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.472028 13697 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:32.482017 13689 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.477s	user 0.001s	sys 0.000s
I20260709 13:19:32.481835 13664 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:32.481611 13464 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.479660 13692 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:19:32.502799 13687 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.415s	user 0.001s	sys 0.001s
I20260709 13:19:32.501724 13694 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:32.541158  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 107685us
I20260709 13:19:32.546167  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:32.555634 13345 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.502465 13572 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:19:32.540416 13710 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:32.557513 13710 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.559868 13710 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.540928 13664 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:32.561329 13569 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:32.561267 13664 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.562407 13670 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:32.542297 13685 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:19:32.562883 13694 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.565358 13694 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:32.567703  6664 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:32.570561 13713 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:32.571159 13713 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:32.573424 13468 consensus_meta.cc:287] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.843s	user 0.008s	sys 0.000s
I20260709 13:19:32.573453  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1717) took 1040 ms. Trace:
I20260709 13:19:32.535197 13715 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:32.566461 13616 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.027s	user 0.008s	sys 0.004s
I20260709 13:19:32.576267  7228 rpcz_store.cc:276] 0709 13:19:31.533396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.533638 (+   242us) service_pool.cc:224] Handling call
0709 13:19:31.533943 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.742608 (+208665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.742667 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.744995 (+  2328us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.081037 (+336042us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:32.081117 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.081296 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.573213 (+491917us) raft_consensus.cc:1685] finished
0709 13:19:32.573239 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.573389 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31325,"raft.queue_time_us":51010,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":331386,"threads_started":2,"wal-append.queue_time_us":792093}
W20260709 13:19:32.591797  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 110479us
I20260709 13:19:32.582756 13692 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:32.581168 13713 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:32.631385  6667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.016s	sys 0.043s
W20260709 13:19:32.631907  6667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.017s	sys 0.043s
I20260709 13:19:32.679569  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1513) took 1323 ms. Trace:
W20260709 13:19:32.682782  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 83560.4us
I20260709 13:19:32.685428  7228 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:32.691417  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3022) took 2461 ms. Trace:
I20260709 13:19:32.691937  6593 rpcz_store.cc:276] 0709 13:19:30.229908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.231182 (+  1274us) service_pool.cc:224] Handling call
0709 13:19:30.231442 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.198732 (+967290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.691155 (+1492423us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b54002fcbd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:32.691359 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2090610432}
I20260709 13:19:32.680235  6592 rpcz_store.cc:276] 0709 13:19:31.356046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.378098 (+ 22052us) service_pool.cc:224] Handling call
0709 13:19:31.408495 (+ 30397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.408795 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.408821 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.409224 (+   403us) log.cc:844] Serialized 26 byte log entry
0709 13:19:31.603548 (+194324us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b180022be80. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:19:31.776665 (+173117us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:31.776721 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.776808 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.612947 (+836139us) raft_consensus.cc:1685] finished
0709 13:19:32.612967 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.679489 (+ 66522us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3134,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":71,"spinlock_wait_cycles":3667328,"thread_start_us":172888,"threads_started":1,"wal-append.queue_time_us":521618}
I20260709 13:19:32.698566  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:32.704677 12568 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:32.705114 12568 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:32.705464 12568 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:32.693897 13627 consensus_meta.cc:287] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.140s	user 0.010s	sys 0.010s
I20260709 13:19:32.703819 13646 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:32.707763 13677 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.708482 13627 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.713722 13627 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:32.716811 13688 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.796s	user 0.006s	sys 0.002s
I20260709 13:19:32.594035 13616 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.724663 13616 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.725342  6671 raft_consensus.cc:1240] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:32.727069 13637 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 1.733s	user 0.011s	sys 0.008s
I20260709 13:19:32.728341  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3138) took 2294 ms. Trace:
I20260709 13:19:32.728868  6665 rpcz_store.cc:276] 0709 13:19:30.433888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.434305 (+   417us) service_pool.cc:224] Handling call
0709 13:19:30.434606 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:30.434946 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.434973 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.435449 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:19:30.641673 (+206224us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:30.641715 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.641793 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.592965 (+1951172us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:32.727979 (+135014us) raft_consensus.cc:1685] finished
0709 13:19:32.728011 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.728274 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1045,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":3211008,"thread_start_us":205872,"threads_started":1,"wal-append.queue_time_us":405409}
I20260709 13:19:32.587634 13715 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.734648  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.735842  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:32.736168  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.736521  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1695) took 2395 ms. Trace:
I20260709 13:19:32.736881  7237 rpcz_store.cc:276] 0709 13:19:30.341432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.766939 (+425507us) service_pool.cc:224] Handling call
0709 13:19:32.080109 (+1313170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.084743 (+  4634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.736354 (+651611us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004de250. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:32.736470 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3018935680}
I20260709 13:19:32.737802  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.738739  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1895) took 2529 ms. Trace:
I20260709 13:19:32.739082  7234 rpcz_store.cc:276] 0709 13:19:30.209588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.209737 (+   149us) service_pool.cc:224] Handling call
0709 13:19:30.210054 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.113340 (+903286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.738495 (+1625155us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54005195d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:32.738661 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075877632}
I20260709 13:19:32.739461 12568 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:32.612059 13707 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:19:32.741962  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:32.738543  6663 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:32.743758  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:32.695914 13586 consensus_meta.cc:287] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 1.012s	user 0.012s	sys 0.006s
I20260709 13:19:32.629531 13694 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.089s
I20260709 13:19:32.750473  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:32.691049 13692 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:32.752136  7236 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:32.741432 12568 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:32.746121  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:32.744951  6634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.006s	sys 0.031s
I20260709 13:19:32.758384  6662 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:32.742540  7235 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:32.759420  7234 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:32.702235 13698 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
W20260709 13:19:32.760071  6634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.007s	sys 0.031s
I20260709 13:19:32.761265 13616 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.761938 13616 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.764863 13616 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.703154 13456 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:32.694480 13700 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:19:32.803987 13584 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:32.810559  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:32.811352  7232 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:32.810503  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:32.814340  7234 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:32.815455  7235 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:32.823982 13715 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:32.826859  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2671) took 1083 ms. Trace:
I20260709 13:19:32.702860 13717 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:19:32.827363  6667 rpcz_store.cc:276] 0709 13:19:31.743222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.743327 (+   105us) service_pool.cc:224] Handling call
0709 13:19:31.743670 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.745088 (+  1418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.745127 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.745534 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.632381 (+886847us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:32.702537 (+ 70156us) log.cc:844] Serialized 14 byte log entry
0709 13:19:32.702750 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.702826 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.826556 (+123730us) raft_consensus.cc:1685] finished
0709 13:19:32.826584 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.826778 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":69974,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":35375,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":70,"thread_start_us":832120,"threads_started":1,"wal-append.queue_time_us":988405}
I20260709 13:19:32.842357 13692 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:32.746205 13628 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:19:32.854888  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.855646  6670 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:32.858367  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.859483  6667 raft_consensus.cc:2410] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:32.863097  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.864029  6663 raft_consensus.cc:2410] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:19:32.866125  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:32.870671  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.871874  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.872967  6665 raft_consensus.cc:2410] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:32.881762  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.893093  6664 raft_consensus.cc:2410] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:32.896109  6668 raft_consensus.cc:2368] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
W20260709 13:19:32.816737 13647 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.736s	user 0.006s	sys 0.009s
W20260709 13:19:32.897368  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 510136us
I20260709 13:19:32.897487  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.759563 13676 consensus_queue.cc:1059] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 13:19:32.905462  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.898705  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:32.899477  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.900972  6667 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:32.907857  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:32.901373  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.901678  7232 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:32.913770  6669 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 22: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:32.914070  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.917342  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.918467  6663 raft_consensus.cc:2410] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:32.907353  6664 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:32.920152  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.921479  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:32.922219  6668 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:32.934322  7199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.011s	sys 0.039s
I20260709 13:19:32.935786  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:32.936925  6593 raft_consensus.cc:2410] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:19:32.936112  7199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.012s	sys 0.040s
I20260709 13:19:32.938089  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.939538  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:32.940222  6666 raft_consensus.cc:2376] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19. Re-sending same reply.
I20260709 13:19:32.941648  6667 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:32.942022  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.942709  6663 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
I20260709 13:19:32.980950  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:32.981328  6669 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 13:19:32.984529  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.893105 13614 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:32.989208  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 199502us
I20260709 13:19:32.990867  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:32.848613 13589 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 13:19:32.902582 13331 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:32.904297 13649 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:33.016222  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:32.939639 13558 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:33.019199  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 228957us
I20260709 13:19:33.023690  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:33.017657 13463 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.002s	sys 0.031s
I20260709 13:19:33.012876 13675 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.029585 13675 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:33.019160 13584 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.041260  6668 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:33.042277  6666 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:33.045070  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:33.045114  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:33.049127  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:33.050689  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:33.052881  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:33.061317  7232 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:33.062598  7234 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:33.061342  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:33.065088  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:33.073838  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:33.076663  7232 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:33.078526  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:33.079440  7232 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:19:33.082217  7232 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:33.084601  7232 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:33.085117  7232 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:33.031641 13463 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.003s	sys 0.031s
I20260709 13:19:33.030555 13696 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:33.113009  7234 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:33.033610 13586 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:33.031797 13736 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.054728 13676 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:33.072427 13115 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:33.073752 13677 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:33.074162 13646 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:33.073987 13692 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:33.019398 13614 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.072588 13616 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:33.033546 13649 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.031476 13584 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:33.150930  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 160153us
I20260709 13:19:33.151620  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3152) took 1591 ms. Trace:
I20260709 13:19:33.156615  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1514) took 1800 ms. Trace:
I20260709 13:19:33.159554  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:33.140060 13586 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.159719  6596 rpcz_store.cc:276] 0709 13:19:31.356443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.358434 (+  1991us) service_pool.cc:224] Handling call
0709 13:19:31.358776 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.565302 (+206526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.156306 (+1591004us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54002fe9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:33.156533 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444846848}
I20260709 13:19:33.159299  6671 rpcz_store.cc:276] 0709 13:19:31.559863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.577765 (+ 17902us) service_pool.cc:224] Handling call
0709 13:19:31.578009 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.725800 (+1147791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.151381 (+425581us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000c58d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:33.151562 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638122752}
I20260709 13:19:33.157799 13676 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:33.165146  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:33.168145  6664 raft_consensus.cc:2376] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 22. Re-sending same reply.
I20260709 13:19:33.161136  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:33.166539  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:33.146961 13677 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.183751  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:33.152308 13616 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.194809  6517 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:33.147153 13692 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.187170  6515 raft_consensus.cc:1240] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:33.171099  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:33.185554  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:33.195818  7232 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.163058 13628 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:33.167845 13586 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:33.207852  7232 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:33.160847 13739 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.167160 13675 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:33.219026  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:33.248908  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56549.6us
I20260709 13:19:33.188331 13677 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.142114 13736 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.178570 13707 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.199232 13616 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:33.199693 13692 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:33.218516 13480 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:33.149859 13331 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:33.290675  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 119644us
I20260709 13:19:33.311107 13480 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:33.311674 13614 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.309289  6486 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:33.199358 13576 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:19:33.262480 13676 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:33.279697 13670 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.353943  7232 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:33.199208 13664 consensus_queue.cc:1059] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.108s
I20260709 13:19:33.260238 13115 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.283597 13675 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:33.385037 13628 consensus_queue.cc:1059] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:19:33.386047 13733 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
W20260709 13:19:33.387125 13743 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:19:33.387517  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1377) took 1652 ms. Trace:
I20260709 13:19:33.469249  6662 rpcz_store.cc:276] 0709 13:19:31.735515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.735693 (+   178us) service_pool.cc:224] Handling call
0709 13:19:31.735960 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.759299 (+1023339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.387269 (+627970us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540011d0d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:33.387445 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350645632}
I20260709 13:19:33.390035 13548 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:19:33.391563 13745 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 13:19:33.392264 13735 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:19:33.387516 13674 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.263753 13739 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.396564 13636 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:33.404362 13736 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.423614 13584 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:33.424211 13734 log.cc:927] Time spent T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 13:19:33.305665 13707 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.426198 13649 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:33.428241 13474 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.960s	user 0.011s	sys 0.000s
I20260709 13:19:33.468544 13675 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.494966 13739 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:33.434281  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.360678 13676 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.496771 13675 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.497015 13676 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:33.486196 13636 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:33.499159 13756 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:33.365779 13670 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:33.465781 13682 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.294s	user 0.013s	sys 0.002s
I20260709 13:19:33.471570 13463 raft_consensus.cc:3055] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:33.492736 13707 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.476586 13548 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:33.464757 13115 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.499701 13636 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:33.509253 13749 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:19:33.509680 13435 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.509495 13548 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.511785 13670 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:33.514853 13115 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.506228 13463 raft_consensus.cc:740] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:33.527921 13463 consensus_queue.cc:260] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.529559 13463 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:33.519418 13754 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:19:33.508170 13682 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:33.607149  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 446032us
I20260709 13:19:33.604276 13646 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:19:33.608775 13702 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.296s	user 0.001s	sys 0.007s
I20260709 13:19:33.670902  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1716) took 2193 ms. Trace:
W20260709 13:19:33.655692 13776 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:19:33.625131 13468 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:33.665360 13646 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:33.667943 13747 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.698246 13747 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.701792 13677 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.702943 13646 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:33.705103 13773 log.cc:927] Time spent T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:19:33.705255 13646 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.687920 13729 raft_consensus.cc:3055] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:33.706845 13729 raft_consensus.cc:740] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:33.707675 13729 consensus_queue.cc:260] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.708609 13729 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:33.711289 13692 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.713369 13692 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:33.714435 13682 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:33.716648 13747 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:33.710229 13676 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.719446 13676 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.729384 13677 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:33.385948  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76732.1us
W20260709 13:19:33.865674  6513 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.006s	sys 0.018s
W20260709 13:19:33.866183  6513 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.006s	sys 0.018s
W20260709 13:19:33.792876  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 325601us
W20260709 13:19:33.867069  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:33.802114  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.671444  7230 rpcz_store.cc:276] 0709 13:19:31.477022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.486421 (+  9399us) service_pool.cc:224] Handling call
0709 13:19:31.486884 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.487214 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:31.487634 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 13:19:31.487903 (+   269us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.488436 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.391471 (+1903035us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b18004b3220. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b8c37d70 00007f04b8c2b293 00007f04b8c20cea 00007f04b8c210d1 00007f04b903ca0f 00007f04b8ff8621 00007f04b908192b
0709 13:19:33.391669 (+   198us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:33.391956 (+   287us) log.cc:844] Serialized 14 byte log entry
0709 13:19:33.667717 (+275761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.667822 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:33.670592 (+  2770us) raft_consensus.cc:1685] finished
0709 13:19:33.670649 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.670821 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":279509,"prepare.queue_time_us":899,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":5696,"raft.queue_time_us":279202,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":9926,"spinlock_wait_cycles":10509568}
W20260709 13:19:33.815130  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.816900  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:19:33.819059  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 210765us
I20260709 13:19:33.875133 13468 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.875969  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:33.876484 13468 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.878491 13471 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.879513 13471 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.703617 13616 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:33.882723 13610 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:33.883466 13610 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:33.885051 13572 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:33.885062 13694 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:19:33.888226 13115 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.889294 13115 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:33.890030 13739 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:33.890421  6667 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:19:33.890882 13739 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:33.891932 13115 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:33.893316 13739 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:33.854818 13312 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.894594 13610 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.895188 13312 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.897872 13646 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:33.898102 13312 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:33.882385 13471 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:33.871078 13386 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.911692 13571 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.912767 13672 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:33.912811 13571 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.913635 13672 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:33.914486 13468 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:33.916832 13412 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.917896  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:33.917727 13412 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.918738  6593 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:33.903909  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:33.919821  6669 raft_consensus.cc:2368] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21.
I20260709 13:19:33.890142 13614 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.911890 13713 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.918674  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:33.921787  6596 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:33.900451 13576 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:33.922057 13412 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.920469  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:33.922334 13576 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:33.916262 13672 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:33.916262 13571 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.921351 13713 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.924257 13576 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.925902 13576 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.926138 13713 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.911827 13386 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.929354 13386 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:33.921332 13614 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:33.898900 13646 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:33.899977  6671 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:33.956163  6415 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:33.958982  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:33.959944  6592 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:33.959252  6415 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 10 to 11, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 11 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:33.971976  6667 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:19:33.974130  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:33.975344  6592 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:33.977209  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:33.982385  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:33.983349  6596 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:33.983660  6415 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 9 to 10, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 10 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:33.985458  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:33.986322  6596 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:33.987836  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:33.988586  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:33.989082  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.004s	sys 0.012s
W20260709 13:19:33.989451  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.004s	sys 0.012s
W20260709 13:19:34.000777  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.001895  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.001904 13739 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:34.002990 13739 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.006040 13739 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:34.006855  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:34.007907  6588 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:33.988718  6592 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
W20260709 13:19:34.024108  6488 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.008s	sys 0.029s
W20260709 13:19:34.034447  6488 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.008s	sys 0.030s
W20260709 13:19:34.024163  6562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.007s	sys 0.007s
W20260709 13:19:34.034505  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 43971.5us
W20260709 13:19:34.039418  6562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.008s	sys 0.007s
I20260709 13:19:34.021454 13616 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:34.047659  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.051733  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:34.052973  6671 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:34.060609  6416 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 332) took 41461 ms (client timeout 15000 ms). Trace:
W20260709 13:19:34.064749  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 848029us
I20260709 13:19:34.062953  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:34.078874  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:34.079531 13586 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.798s	user 0.011s	sys 0.004s
I20260709 13:19:34.079787  6670 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:34.080086 13586 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:34.081691  6415 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 8 to 9, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 9 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:34.083506  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:34.084932  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.086181  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.084539  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:34.086760  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47351.1us
I20260709 13:19:34.087857  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:34.088901  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:34.089887  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:34.086807  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:34.086802 13331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.034s	sys 0.050s
W20260709 13:19:34.092152 13331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.034s	sys 0.050s
I20260709 13:19:34.092471 13569 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:34.093147  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.093055  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:34.093364 13569 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.086225  7230 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 13:19:34.099152  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.101517 13646 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:34.103992  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.106395  6514 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:34.107761  6522 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:34.109144  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:34.109319 13386 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:34.110006 13386 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:34.110772 13412 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:34.111508 13412 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:34.112406 13386 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:34.112993 13386 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.113631 13412 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:34.114538  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:34.117303  6662 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:34.086700  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.118387  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:34.061331  6416 rpcz_store.cc:269] 0709 13:18:52.599221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.599456 (+   235us) service_pool.cc:224] Handling call
0709 13:18:52.673438 (+ 73982us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5400002490. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5d0b2 00007f04beed6a89 00007f04bee6eda6 00007f04bee40088 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc
0709 13:19:34.060267 (+41386829us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:24.140373 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:24.161842 (+ 21469us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:24.161902 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:19:24.161938 (+    36us) tablet.cc:662] Decoding operations
0709 13:19:24.252611 (+ 90673us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:24.276188 (+ 23577us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:24.280345 (+  4157us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:19:24.452485 (+172140us) tablet.cc:701] Row locks acquired
0709 13:19:24.452537 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:19:24.452717 (+   180us) write_op.cc:270] Start()
0709 13:19:24.452884 (+   167us) write_op.cc:276] Timestamp: P: 1783603164452650 usec, L: 0
0709 13:19:24.452930 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:19:24.453745 (+   815us) log.cc:844] Serialized 6635 byte log entry
0709 13:19:25.426481 (+972736us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18002dea40. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b8ffe190 00007f04b8ff7ae7 00007f04b8ff6dd9 00007f04b9001930
0709 13:19:26.195479 (+768998us) op_driver.cc:464] REPLICATION: finished
0709 13:19:27.005270 (+809791us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04bcab7ef0
0709 13:19:27.032720 (+ 27450us) write_op.cc:301] APPLY: starting
0709 13:19:27.032869 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:27.393501 (+360632us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:27.393561 (+    60us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:28.021525 (+627964us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:28.021944 (+   419us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:19:28.022009 (+    65us) write_op.cc:312] APPLY: finished
0709 13:19:28.022475 (+   466us) log.cc:844] Serialized 156 byte log entry
0709 13:19:28.173995 (+151520us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:28.186360 (+ 12365us) write_op.cc:454] Released schema lock
0709 13:19:28.194861 (+  8501us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:28.195321 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:28.195352 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1822620,"reported_tablets":45,"spinlock_wait_cycles":27548928,"child_traces":[["op",{"apply.queue_time_us":836858,"apply.run_cpu_time_us":57339,"apply.run_wall_time_us":1162621,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75153,"num_ops":17,"prepare.queue_time_us":239293,"prepare.run_cpu_time_us":26722,"prepare.run_wall_time_us":499924,"raft.queue_time_us":1521833,"raft.run_cpu_time_us":39771,"raft.run_wall_time_us":945795,"replication_time_us":1742326,"spinlock_wait_cycles":88229504,"thread_start_us":1629983,"threads_started":5,"wal-append.queue_time_us":447781,"wal-append.run_cpu_time_us":25459,"wal-append.run_wall_time_us":3429396}]]}
I20260709 13:19:34.097290  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:34.124291  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.126281 13480 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:34.113603 13386 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.126801 13480 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:34.127300 13480 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:34.127712  6593 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:34.137867 13386 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:34.094043 13713 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:34.145553  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.147642  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:34.184494  6662 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:34.201356  6664 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:34.217525 13475 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.218894  6666 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:34.218413 13475 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.221169 13569 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:34.208273  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:34.228426  6521 raft_consensus.cc:2410] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 18 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:19:34.221580 13627 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.233916  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:34.235419  6514 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:34.221101 13475 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:34.235780 13764 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:34.239687  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.235591 13614 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:34.254421  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.257879  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:34.264441  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 174731us
I20260709 13:19:34.275035  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:34.275714 13627 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:34.276854  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 202817us
I20260709 13:19:34.277951 13586 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:34.279330  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.275282  6517 raft_consensus.cc:1240] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 13:19:34.317107  6670 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:34.328051  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.332129  6522 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:34.281579 13610 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
W20260709 13:19:34.344059  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.365051  6518 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
W20260709 13:19:34.366986  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.367429 13463 consensus_meta.cc:287] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.733s	user 0.005s	sys 0.019s
I20260709 13:19:34.371227  6520 raft_consensus.cc:2410] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:34.372110  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:34.372324  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:34.372864  7230 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:34.374379  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:34.375449  7228 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:34.374475  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:34.375676 13460 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:34.375455 13790 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:34.376415 13460 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.374277  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:34.378957  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 247183us
W20260709 13:19:34.354292 13785 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
W20260709 13:19:34.384755  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 53672.5us
I20260709 13:19:34.337595 13555 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.391990  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:34.389925  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46483.9us
W20260709 13:19:34.395066  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 57802.8us
I20260709 13:19:34.378925  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:34.397421  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:34.416532  7230 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:19:34.419322  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:34.423058  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.436968  6591 raft_consensus.cc:2410] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:19:34.404752 13713 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:34.442656  6666 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:34.447528  6513 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:34.451829  7232 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:34.453809  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:34.462217  7228 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:34.466140  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.473448 13682 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:34.473634 13791 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:19:34.474229 13682 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.478103 13682 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:34.423492 13386 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:19:34.394501 13412 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.077s
W20260709 13:19:34.504735  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 43553.7us
I20260709 13:19:34.456674 13460 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:34.506102  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:34.501610 13475 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:19:34.493508 13576 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:19:34.487653 13569 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:19:34.502218  6486 leader_election.cc:400] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21.
I20260709 13:19:34.547475  6486 leader_election.cc:403] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:34.504362  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 193911us
I20260709 13:19:34.465530 13555 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.428551 13463 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 13:19:34.507187  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.537896  6671 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:34.566718  6517 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:34.498561  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:34.590927  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:34.587497  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 81510.8us
I20260709 13:19:34.546566 13571 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:34.596309  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58120.7us
I20260709 13:19:34.592873 13682 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.610050 13682 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.616326  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:34.620680  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:34.498351 13771 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.162s
W20260709 13:19:34.625435  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 123508us
W20260709 13:19:34.622784  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 169063us
I20260709 13:19:34.643211  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:34.643503  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.666982  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.685814  6671 raft_consensus.cc:2410] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:34.625066 13713 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:19:34.662420 13412 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:34.701826 13798 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:19:34.701815  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.703246  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:34.703539  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.691774  6517 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
W20260709 13:19:34.691288 13788 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.102s	user 0.002s	sys 0.000s
W20260709 13:19:34.714877  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.729898  6518 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:19:34.732724  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:34.734773  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.711651 13627 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:34.744851  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 209278us
I20260709 13:19:34.717002 13331 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.717316 13571 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
W20260709 13:19:34.755594  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 162925us
I20260709 13:19:34.707024 13475 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:19:34.783160  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.799561 13768 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:34.802989 13682 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:34.803191 13801 log.cc:927] Time spent T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.112s	user 0.004s	sys 0.000s
I20260709 13:19:34.808295  7228 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:34.745007 13628 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:34.775104 13480 consensus_meta.cc:287] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.008s
I20260709 13:19:34.750854 13696 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:34.812860  6517 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:34.821614  7232 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:34.755301 13775 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.187s
W20260709 13:19:34.838840  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 86893.6us
I20260709 13:19:34.780988 13627 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:34.816620 13768 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:34.859810  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.756618 13771 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.092s
W20260709 13:19:34.901823  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 50428.3us
I20260709 13:19:34.902161  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:34.895959  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 181531us
I20260709 13:19:34.909582  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:34.910723  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:34.839654 13460 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 13:19:34.915818  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.924252 13677 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:34.924784 13677 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.925143 13677 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:34.926904 13586 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:34.927474 13586 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.915174 13555 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:34.928265  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:34.928925  6517 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:34.927881 13586 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:34.845317 13480 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.931437  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:34.934408  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.887322 13627 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:34.879756 13771 consensus_queue.cc:1243] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer 53fee9894731461a821a131be24dbbca log is divergent from this leader: its last log entry 14.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:19:34.949486 13468 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:34.955103 13468 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:34.956436 13468 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.957533 13468 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:34.959506 13627 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.855053 13331 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:34.951267  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:34.952694  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 107451us
W20260709 13:19:34.966424  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 67554.8us
I20260709 13:19:34.930723 13768 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:34.975898 13768 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:34.978798 13412 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:34.952555 13628 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:34.983019 13628 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:34.949254 13809 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:19:34.985252  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:34.985656  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.987020  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.988119  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.988883  6518 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 24: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:34.989552  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:34.989957 13331 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:34.951545 13480 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:34.991112  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.991755 13672 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:34.992252  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.992363 13672 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:34.993353  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.994828  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:34.990350  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:34.994498 13672 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:35.015000  6561 leader_election.cc:400] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
I20260709 13:19:35.015367  6561 leader_election.cc:403] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:19:35.016566  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:35.018070  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.018538 13480 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 15. Current term is 16.
W20260709 13:19:35.020020  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47161.6us
I20260709 13:19:35.021317  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:35.022457  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.023798  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.026018  6521 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
W20260709 13:19:35.027513  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:35.031643  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 363680us
W20260709 13:19:35.034268  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 123667us
I20260709 13:19:35.035997  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.026949 13812 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:35.043027  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.041805  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:35.031201 13386 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:35.043973  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:35.048480  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.037237 13480 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:35.059182  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.041429 13771 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:35.043809 13569 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.060766 13460 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.064447  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.067976  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.069372  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.047784 13572 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:35.072103  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.075342  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.079900  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.084638  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.093185  6590 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:35.100808  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.104027  6591 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:35.099469  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:35.106803  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:35.107487  6590 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
W20260709 13:19:35.111239 13793 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:19:35.112581  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3044) took 1004 ms. Trace:
I20260709 13:19:35.112962  6596 rpcz_store.cc:276] 0709 13:19:34.108280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.109578 (+  1298us) service_pool.cc:224] Handling call
0709 13:19:34.110007 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:34.110510 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.110540 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:34.114976 (+  4436us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.237801 (+122825us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:34.237828 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.237881 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.112346 (+874465us) raft_consensus.cc:1685] finished
0709 13:19:35.112372 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.112524 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3338,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":195,"thread_start_us":122395,"threads_started":1,"wal-append.queue_time_us":881841}
I20260709 13:19:35.116952  6519 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 23. Current term is 24. Ops: []
W20260709 13:19:35.118876  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.122802  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:35.079059 13771 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.081243 13460 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.114140 13677 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.086163 13569 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.133090  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:35.134300  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:35.132436 13331 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:35.132599 13571 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:35.137357 13386 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:35.089699 13572 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.125753 13802 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.154750  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.145651  6670 pending_rounds.cc:85] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 13
I20260709 13:19:35.162863  6519 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 12: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:35.162955  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.157930  6670 pending_rounds.cc:107] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 14.14
I20260709 13:19:35.165009  6670 raft_consensus.cc:2889] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:35.162741  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.158587  6665 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:35.147848 13460 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:35.153966 13571 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:35.170679  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:35.174856  6664 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:19:35.173624  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.176707  6592 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:35.175263  6664 pending_rounds.cc:85] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 11
I20260709 13:19:35.179980  6664 pending_rounds.cc:107] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:19:35.180266  6664 raft_consensus.cc:2889] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:35.186650  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 102278us
I20260709 13:19:35.198311  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.173144 13571 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:35.178243 13610 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.201643  6519 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:35.168522 13802 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.201157 13386 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:35.223330  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.211366 13696 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21.
I20260709 13:19:35.211205 13480 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:35.225962  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.226701  7232 raft_consensus.cc:2368] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21.
I20260709 13:19:35.225064  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:35.227294 13810 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.176s	user 0.000s	sys 0.002s
I20260709 13:19:35.228364 13816 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 13:19:35.229282 13816 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.229947 13816 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.230769  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:35.231945 13677 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.232378 13802 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.233455  6520 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:35.234377 13628 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.236517  7232 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:19:35.239341  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.221064 13386 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:35.300602  6518 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:35.301191  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:35.214335 13586 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.211053 13672 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:19:35.329366  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.331730  7236 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:35.331754  7232 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:19:35.339249 13768 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:35.320907 13386 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.255827 13610 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.317967 13812 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:35.414005  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.414745  7234 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:35.389322 13768 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:35.425112  7237 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 12.
I20260709 13:19:35.414924 13771 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.432396  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.424146 13331 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:35.431522 13822 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.453303 13475 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:35.454222  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:35.434837 13572 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:35.464461 13843 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:35.483558 13820 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:19:35.484094 13610 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:35.485365  7231 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:35.485047 13768 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.486356 13768 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:35.487735 13569 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:35.479148 13812 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.427587 13713 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:19:35.487807  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:35.494845  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 321355us
I20260709 13:19:35.503793  6519 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:35.503671  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:35.505710  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 140472us
I20260709 13:19:35.508795  7234 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:19:35.514600  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.487758 13555 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:35.527719  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.490424 13586 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:35.530295 13821 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:19:35.531610 13331 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.488190 13843 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.488260 13672 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:35.536373  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.537935  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:35.505149 13822 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.545166  7232 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
W20260709 13:19:35.545889  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58157.7us
I20260709 13:19:35.505892 13677 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:35.549099  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.555017  7228 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:35.556043  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:35.557355  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.561894  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.558568  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.567695  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.569178 13412 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:35.570117  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:35.575104  7231 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:35.546118 13816 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.553264 13475 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:35.580435  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.569658 13412 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:35.583058  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 281966us
I20260709 13:19:35.586588  7228 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21. Ops: [19.15-20.16]
I20260709 13:19:35.587762  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:35.599057  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:35.601852  7235 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.598961  7232 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
W20260709 13:19:35.602185  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 98739.2us
I20260709 13:19:35.575451 13677 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:35.598642 13825 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:19:35.575289 13386 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.609930  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:35.617692  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 109017us
I20260709 13:19:35.588322 13480 consensus_queue.cc:1048] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:19:35.609141 13812 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:35.618521  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:35.617609  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.646795  6671 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:35.623288 13846 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:35.660317 13849 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:19:35.649031  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:35.633754 13843 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.651959 13817 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:35.635596 13460 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:19:35.668560  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:35.673007  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 74316.3us
W20260709 13:19:35.678823 13830 log.cc:927] Time spent T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:19:35.678969  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 3046) took 1529 ms. Trace:
I20260709 13:19:35.679479  6516 rpcz_store.cc:276] 0709 13:19:34.154305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.166105 (+ 11800us) service_pool.cc:224] Handling call
0709 13:19:34.343583 (+177478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:34.418704 (+ 75121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.418735 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:34.466259 (+ 47524us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.660707 (+194448us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:34.660746 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.660822 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.230813 (+569991us) raft_consensus.cc:1685] finished
0709 13:19:35.230837 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.670194 (+439357us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400355a50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:35.678743 (+  8549us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75565,"raft.queue_time_us":71702,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":27648,"spinlock_wait_cycles":406964992,"thread_start_us":162544,"threads_started":1,"wal-append.queue_time_us":487082}
I20260709 13:19:35.681809  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:35.634231 13412 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:35.684367 13412 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:35.684856 13822 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:35.685564 13837 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:35.685993 13837 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:35.686287 13829 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:19:35.684186 13815 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:35.669433 13817 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.688766 13817 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:35.691190 13852 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:19:35.698518  7235 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:19:35.652693 13839 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:35.712930  7237 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 12.
I20260709 13:19:35.658017 13812 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:35.716274  6665 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:35.717525  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 111437us
W20260709 13:19:35.719619 13854 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:35.719947  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:35.720405  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.721963  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.693979  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.730065 13812 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.686309 13837 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.728202 13843 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:35.735173 13839 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:35.761476  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.760483  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:35.756067 13571 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.006s
I20260709 13:19:35.746735 13610 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:35.765496  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71644.2us
I20260709 13:19:35.754282 13480 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:35.762428 13846 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:35.766793  7200 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:35.767696 13480 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.774937 13480 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:35.774981 13848 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.775579 13848 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.777830 13848 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.768127 13840 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:19:35.779129 13677 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.765064 13843 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:35.765910 13386 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:35.786077  6637 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.768s (SENT) [suppressed 35 similar messages]
I20260709 13:19:35.786888 13824 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:35.787367 13824 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:35.787735 13824 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:35.796948 13628 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.797314  6514 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:35.798636  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:35.799451 13846 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:35.802824 13846 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:35.804991  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:35.805877 13846 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:35.786185  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.810302 13826 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:35.812537 13868 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 13:19:35.810346 13856 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:19:35.817058 13846 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:35.818691 13386 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:35.819322 13386 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.819600 13837 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.820990 13826 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:35.757243 13768 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:19:35.821756 13386 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.822768  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:35.823602 13468 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:35.823959 13468 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:35.824337 13468 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.829041  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.802152 13480 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:35.832440 13837 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.837386 13480 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:35.839044 13480 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:35.839666 13672 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:35.838189  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.835214 13824 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:35.853477  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 81682us
I20260709 13:19:35.858198 13480 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:35.859141  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:35.859560  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.867053  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.868044  7237 raft_consensus.cc:2410] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:19:35.871199  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:35.867165  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 230040us
I20260709 13:19:35.768270 13571 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:35.891350  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:35.894724  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:35.885447  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:35.945369  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 51405.9us
I20260709 13:19:35.946537  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:35.944641  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.965018  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.981828  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.964372  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:35.985736  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:35.987071  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.987975  6516 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:35.988780  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:35.989210  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:35.989516  6517 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:19:35.991308  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 131805us
I20260709 13:19:35.993721  6591 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:19:35.988401  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:36.000645  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.994215  6591 pending_rounds.cc:85] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 15
I20260709 13:19:36.016927  6591 pending_rounds.cc:107] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:19:36.017294  6591 raft_consensus.cc:2889] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:35.995692  6520 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:35.963425 13856 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:35.928761 13839 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:36.027413  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45674.2us
I20260709 13:19:35.962218 13555 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:35.943638 13824 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:36.053821  6487 leader_election.cc:400] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21.
W20260709 13:19:36.036201  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71939.5us
W20260709 13:19:36.075510  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:36.078655  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50373.6us
I20260709 13:19:36.079644  6636 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:36.081055 13864 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:36.122112  6636 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:36.123484  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.124802  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.133121  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:36.139858 13331 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:36.140586  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:36.145663  6664 raft_consensus.cc:2393] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:36.145541  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.141209  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.141853  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.142431  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.143036  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.143693 13836 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:36.144167 13571 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:36.133913  6515 raft_consensus.cc:2376] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19. Re-sending same reply.
I20260709 13:19:36.146907  6516 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:36.152122  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.163295  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.072292 13822 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:36.155584  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 75888.4us
W20260709 13:19:36.164960  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.030962 13386 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.124s
I20260709 13:19:36.082062 13555 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.146266 13672 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:36.155920 13331 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.169281 13677 raft_consensus.cc:2804] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:19:36.205348  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 151732us
I20260709 13:19:36.173925 13682 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.168136 13475 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:36.192658  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:36.210316  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:36.189549 13331 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:36.192960 13840 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:36.186321 13837 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:36.232461 13834 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:36.208837 13672 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.252182 13468 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.263281 13837 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:36.264700 13875 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:19:36.266096  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:36.293370  6594 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:36.268533 13873 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:19:36.279263 13867 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:19:36.280728 13555 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.291442  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.219103 13775 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:19:36.214066 13682 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.219818 13475 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:36.304530  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 248377us
I20260709 13:19:36.292189 13480 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:36.250756 13840 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:36.409197  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.416229  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:36.416939  7231 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:36.417940 13877 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 13:19:36.419330 13866 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:19:36.420441 13840 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.423605 13835 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:19:36.423812 13882 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:19:36.424160 13835 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:36.424345 13824 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.424618 13835 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:36.425077 13824 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.427791 13837 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:36.430178 13817 consensus_meta.cc:287] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.000s
I20260709 13:19:36.430655 13817 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.431732  6519 raft_consensus.cc:2368] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19.
I20260709 13:19:36.435297 13817 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.436808 13672 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.450116  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:36.452950  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:36.464525  7231 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:36.468789  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:36.471161  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:36.473853  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:36.475104  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:36.475850  7232 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
W20260709 13:19:36.476950  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 266720us
I20260709 13:19:36.477667  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:36.482657  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.486865  7236 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:36.487887  6665 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:19:36.494526  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.002s	sys 0.019s
W20260709 13:19:36.495056  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.002s	sys 0.019s
I20260709 13:19:36.541586  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:36.545678  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 51722.9us
I20260709 13:19:36.485530 13817 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:36.548619  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 66058.1us
W20260709 13:19:36.557626  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 91081us
I20260709 13:19:36.552459  6591 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
W20260709 13:19:36.554400  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 368139us
I20260709 13:19:36.547721 13817 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:36.578641  6668 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:36.585057  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:36.589291  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:36.590520 13836 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:36.578504  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.586380  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:36.598783 13893 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.599459 13893 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.600484  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:36.607836  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 316463us
I20260709 13:19:36.608652  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:19:36.609376  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.609416  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:36.591460  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1415) took 1054 ms. Trace:
I20260709 13:19:36.612536  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:36.554839 13628 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:19:36.614007 13815 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:36.615198 13883 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:36.617300  7236 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:36.618378 13848 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:36.618985 13768 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:36.619318 13880 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.619905 13880 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.607036 13412 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:36.621115 13412 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:36.622251 13880 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.622007 13412 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.623046 13412 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:36.619010 13848 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.623622  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:36.625129 13848 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.626958  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2866) took 1481 ms. Trace:
I20260709 13:19:36.627703  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:36.628229  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:36.629562  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:36.561370 13824 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:36.628835  6521 rpcz_store.cc:276] 0709 13:19:35.145958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.167197 (+ 21239us) service_pool.cc:224] Handling call
0709 13:19:35.724309 (+557112us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:35.724765 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:35.724806 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:35.754117 (+ 29311us) log.cc:844] Serialized 26 byte log entry
0709 13:19:36.034360 (+280243us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:36.034418 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.034510 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.278718 (+244208us) raft_consensus.cc:1685] finished
0709 13:19:36.278742 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.626714 (+347972us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54003bbad0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:36.626904 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":5624,"spinlock_wait_cycles":1265698560,"thread_start_us":254089,"threads_started":1,"wal-append.queue_time_us":403944}
I20260709 13:19:36.569100 13863 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:36.639200  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:36.572196 13817 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:36.659267  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.660548  6593 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:19:36.658669  6590 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:36.672238  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:36.672891  7232 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:36.679529  7237 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:36.680646  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.610563  6667 rpcz_store.cc:276] 0709 13:19:35.536960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.537110 (+   150us) service_pool.cc:224] Handling call
0709 13:19:35.537505 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.621551 (+ 84046us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:35.942081 (+320530us) log.cc:844] Serialized 14 byte log entry
0709 13:19:36.204998 (+262917us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:36.205112 (+   114us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:36.205201 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.591178 (+385977us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b540005afd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:36.591230 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.591371 (+   141us) inbound_call.cc:177] Queueing success response
0709 13:19:36.600312 (+  8941us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b180044d460. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
Metrics: {"mutex_wait_us":25219,"prepare.queue_time_us":560718,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":25377,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":2199,"spinlock_wait_cycles":204843136,"thread_start_us":489382,"threads_started":2,"wal-append.queue_time_us":293781}
I20260709 13:19:36.683314  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:36.689644  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 81079.1us
I20260709 13:19:36.575255 13331 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.688685  6664 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:36.691220  6562 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.692518  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:36.694934  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.700765  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.705317  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.707357  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:36.708714 13846 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:36.689070 13475 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.709759  6663 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:36.710029  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.694090 13883 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:36.763001  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.716444 13880 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:36.773128  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 63185.5us
I20260709 13:19:36.772068  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:36.695405 13835 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:36.825433  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:19:36.829548  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 57625.3us
I20260709 13:19:36.833611  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:36.759259 13883 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:36.853152  6593 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:36.854236  6663 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:36.822991 13848 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:36.864130 13412 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:36.864777 13771 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:19:36.865950 13899 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:19:36.852710 13677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.029s	sys 0.035s
I20260709 13:19:36.873373  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.844513 13894 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:36.853468 13888 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.883159  6593 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:36.851889 13883 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:36.899392  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:36.900789  6515 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:36.902251  7232 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
W20260709 13:19:36.882900 13677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.029s	sys 0.035s
I20260709 13:19:36.907142  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:36.908847 13905 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:19:36.908988 13331 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.912228  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.913535  6637 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.914945  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:36.917529 13901 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:36.918231 13901 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:36.921020 13571 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:36.921615 13571 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.885154 13412 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:36.932829 13677 raft_consensus.cc:697] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 22 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:36.936010 13571 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:36.938228 13677 consensus_queue.cc:237] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:36.939316 13889 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
W20260709 13:19:36.888111 13896 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:19:36.975780  6669 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:36.957207 13771 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.978124 13771 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:36.980201 13771 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:36.943976 13883 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.890825 13888 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:36.882324 13894 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:36.949173  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.949932  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:36.961680  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:36.968250 13835 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:36.969170  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:36.998775 13846 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:37.010664 13888 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.011103 13870 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:37.011740 13870 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:36.995592  6594 consensus_meta.cc:287] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.699s	user 0.008s	sys 0.002s
I20260709 13:19:37.001971  6665 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:37.002099  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3072) took 1020 ms. Trace:
I20260709 13:19:36.956946 13331 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.018997  6594 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 13.
I20260709 13:19:36.981163 13817 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.016194  6588 rpcz_store.cc:276] 0709 13:19:35.982057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.982373 (+   316us) service_pool.cc:224] Handling call
0709 13:19:36.022891 (+ 40518us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:36.029866 (+  6975us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:36.029902 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:36.077364 (+ 47462us) log.cc:844] Serialized 52 byte log entry
0709 13:19:36.297703 (+220339us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:36.297761 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.297829 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.602393 (+304564us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b4800061ed8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bdfaaaa1 00007f04bdfa1b37 00007f04bdfa19ca 00007f04be09926f 00007f04be098f37 00007f04be0ad55c 00007f04be0ad508 00007f04be0ad488 00007f04be0ad443 00007f04be0ac6ec 00007f04bcfd6a96
0709 13:19:36.982797 (+380404us) raft_consensus.cc:1685] finished
0709 13:19:36.982823 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.002021 (+ 19198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":174799,"raft.run_cpu_time_us":10152,"raft.run_wall_time_us":394084,"spinlock_wait_cycles":12794240,"thread_start_us":94880,"threads_started":1,"wal-append.queue_time_us":276790}
I20260709 13:19:37.040469  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:37.041646  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 3074) took 1004 ms. Trace:
I20260709 13:19:37.076104  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.034744 13475 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.053114  6594 rpcz_store.cc:276] 0709 13:19:36.031130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.165676 (+134546us) service_pool.cc:224] Handling call
0709 13:19:37.035438 (+869762us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:37.014595 13894 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:37.087019  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.089008  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.090215  6594 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:37.080300  6521 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:37.049496  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:37.091549  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.093246  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.095044  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.036144 13835 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:37.037317 13870 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.036394 13614 raft_consensus.cc:2764] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:19:37.080235 13815 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:37.126502  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.130120  6594 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 24.
W20260709 13:19:37.132462  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 217832us
I20260709 13:19:37.049357 13331 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.118257 13475 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.038872 13627 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 13:19:37.301132  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:37.301855  6596 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.008s	sys 0.021s
I20260709 13:19:37.301939  6637 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:37.302315  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 258468us
I20260709 13:19:37.307047  6519 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:37.321305  6596 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.008s	sys 0.021s
I20260709 13:19:37.307142 13894 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.309227  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.316826  6588 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 13: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:37.322448  6665 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:37.321764  6519 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:37.314924 13815 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:37.323206 13775 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:37.368922 13677 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:37.349030 13460 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.348665 13610 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:37.489212  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.367254 13571 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.490271  6594 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:37.519488  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:37.543020  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 241136us
W20260709 13:19:37.530650 13919 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.379s	user 0.003s	sys 0.005s
I20260709 13:19:37.559894  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:37.560298  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 212238us
I20260709 13:19:37.561254  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:37.560509  6593 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:37.530844 13835 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.571815 13835 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.572389 13933 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:37.572947 13933 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:37.574258 13835 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:37.530787 13627 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:19:37.574800 13815 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.575487  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.576622  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.577142  6593 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:37.577913  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.579079 13475 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:37.529881 13610 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.580261 13775 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:37.580611 13775 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:37.581621 13610 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.581408 13775 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:37.516880 13460 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.583765 13883 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:37.584769 13928 log.cc:927] Time spent T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:19:37.584982 13460 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:37.586149 13923 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:19:37.586199 13894 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:37.566721 13926 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 13:19:37.567813 13929 log.cc:927] Time spent T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
W20260709 13:19:37.568734 13918 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.245s	user 0.000s	sys 0.000s
W20260709 13:19:37.598289  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.598989  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.569411 13893 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.551838 13677 raft_consensus.cc:697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:37.583273 13927 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:19:37.545758 13463 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.578435 13331 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:37.618609  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:37.595014  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 114754us
I20260709 13:19:37.620141  6562 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:37.595641  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44673.7us
I20260709 13:19:37.621362  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:37.596655 13920 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:19:37.596589  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.596928 13848 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.604488  6591 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:37.565505 13802 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:37.552560 13571 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:37.625430  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 96322.1us
I20260709 13:19:37.640869  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:37.641677  7237 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:37.643546  7235 raft_consensus.cc:1240] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:37.592592 13817 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:37.628636  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:37.648288  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.626507  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:37.648751  7233 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:19:37.650274 13880 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:37.645975  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.647220  6594 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:37.657233  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.658249  6669 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:37.649207  7233 pending_rounds.cc:85] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 11
I20260709 13:19:37.660585  7233 pending_rounds.cc:107] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:19:37.660925  7233 raft_consensus.cc:2889] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:37.662443  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.662549  6637 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:37.665706  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.667246  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:37.633787 13331 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:37.633543 13713 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:37.675063  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1531) took 1098 ms. Trace:
I20260709 13:19:37.676175  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:37.682663  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1524) took 1186 ms. Trace:
I20260709 13:19:37.683104  6596 rpcz_store.cc:276] 0709 13:19:36.496440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.496554 (+   114us) service_pool.cc:224] Handling call
0709 13:19:36.553090 (+ 56536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.626973 (+ 73883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.627004 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.627464 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:19:37.322090 (+694626us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:37.322130 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.322192 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:37.682414 (+360222us) raft_consensus.cc:1685] finished
0709 13:19:37.682434 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.682570 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75516,"raft.queue_time_us":101114,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":11004,"thread_start_us":694279,"threads_started":1,"wal-append.queue_time_us":744102}
I20260709 13:19:37.684969  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.685700  6596 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:37.686458  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.640344 13933 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.686760 13914 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.649197 13848 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:37.690753 13887 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.690971  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.692348  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.656164 13880 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.700520  6596 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
W20260709 13:19:37.693490  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 127373us
I20260709 13:19:37.655789 13883 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.698882  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.629354 13463 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:37.679370  6592 rpcz_store.cc:276] 0709 13:19:36.576310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.627258 (+ 50948us) service_pool.cc:224] Handling call
0709 13:19:36.627638 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.674282 (+ 46644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.674316 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.684935 (+ 10619us) log.cc:844] Serialized 26 byte log entry
0709 13:19:36.909173 (+224238us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:36.909216 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.909285 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:37.674837 (+765552us) raft_consensus.cc:1685] finished
0709 13:19:37.674861 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.675008 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46250,"raft.queue_time_us":46731,"spinlock_wait_cycles":5005056,"thread_start_us":193754,"threads_started":1,"wal-append.queue_time_us":383742}
W20260709 13:19:37.708873  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46385.4us
I20260709 13:19:37.740964  6637 leader_election.cc:400] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19.
I20260709 13:19:37.741570  6637 leader_election.cc:403] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:37.661903 13610 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
W20260709 13:19:37.752518  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 188865us
I20260709 13:19:37.756320  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1525) took 1204 ms. Trace:
I20260709 13:19:37.748328 13914 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:37.756640  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.704437 13880 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:37.740875 13911 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:37.677516 13802 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:37.766769  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 117111us
W20260709 13:19:37.760380 13930 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.070s	user 0.001s	sys 0.001s
I20260709 13:19:37.757592  6589 rpcz_store.cc:276] 0709 13:19:36.496766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.563514 (+ 66748us) service_pool.cc:224] Handling call
0709 13:19:36.582908 (+ 19394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.643603 (+ 60695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.643644 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.668789 (+ 25145us) log.cc:844] Serialized 26 byte log entry
0709 13:19:37.027208 (+358419us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:37.027249 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.027314 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:37.700479 (+673165us) raft_consensus.cc:1685] finished
0709 13:19:37.700500 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.700630 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48485,"raft.queue_time_us":87437,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":14603,"thread_start_us":305777,"threads_started":1,"wal-append.queue_time_us":638634}
W20260709 13:19:37.763954  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.770056 13914 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:37.772100 13848 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.741317 13887 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.691075 13677 consensus_queue.cc:237] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.702368 13835 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:19:37.788082 13463 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.739859 13883 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:37.807858  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 145775us
W20260709 13:19:37.809253  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 68503.4us
I20260709 13:19:37.762501 13331 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19.
I20260709 13:19:37.810590  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.815514  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.824476  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:37.826306  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68282.4us
I20260709 13:19:37.823999  6589 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:37.828398  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:37.830153  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.832032 13713 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:37.832489 13713 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.833490  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.835347 13480 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:37.835764 13480 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:37.836680 13480 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.832882 13713 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:37.837947 13480 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.839231 13571 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:37.834111  6589 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 24.
I20260709 13:19:37.841282  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.827886 13386 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.842358  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.843449  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.845880  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:37.847255  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:37.847972  6515 raft_consensus.cc:2376] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21. Re-sending same reply.
I20260709 13:19:37.852926  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:37.853746  6514 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:37.843827  6589 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:37.854586  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.855031  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.857411  6589 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:37.859243  6522 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:37.855147  6593 raft_consensus.cc:2376] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21. Re-sending same reply.
I20260709 13:19:37.860232  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:37.830241 13947 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.864446  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:37.827983 13822 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:19:37.867010  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:37.866565  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.867687  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.867913  6589 raft_consensus.cc:3060] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:37.871230  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:37.874382  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:37.853003 13463 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:37.859076 13386 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:37.879371  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.879987  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:37.880235  6666 raft_consensus.cc:2410] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:37.879640  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:37.881348  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53174.3us
W20260709 13:19:37.866720  6637 connection.cc:256] RPC call timeout handler was delayed by -1.5202539370000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24540000000000003s for a response
W20260709 13:19:37.883065  6637 connection.cc:256] RPC call timeout handler was delayed by -1.418003882999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 13:19:37.883520  6637 connection.cc:256] RPC call timeout handler was delayed by -1.3694038829999045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
W20260709 13:19:37.884022  6637 connection.cc:256] RPC call timeout handler was delayed by -1.0709609089999503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 13:19:37.888136  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:37.888772  7230 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:37.889045  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.893453  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:37.877512 13947 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:37.898437 13947 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:37.899005  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:37.899528  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:37.900131  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:37.893146 13881 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:37.900838 13881 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:37.900882  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:37.901244  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:37.901453 13881 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:37.902134  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:37.902281  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:37.902842  7237 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:19:37.903566 13835 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:37.903985 13835 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:37.904363 13835 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.907361 13802 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.908054  6486 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:37.908430 13802 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:37.909970 13802 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:37.872480 13331 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.910861 13914 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:37.911307 13914 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:37.911180 13331 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.911767 13914 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.912722  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.913282  6593 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:37.899070 13463 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.916095  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:37.917212 13463 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:37.915180 13571 consensus_queue.cc:1059] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:37.915315  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.918522  6671 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:37.913952 13911 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:37.919178 13386 raft_consensus.cc:3055] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:37.919593 13386 raft_consensus.cc:740] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:37.920408 13386 consensus_queue.cc:260] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.921775 13911 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.924290 13883 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:37.925326 13331 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:37.927187 13924 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.589s	user 0.002s	sys 0.000s
I20260709 13:19:37.927361  7237 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:37.929801 13887 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:37.932232 13900 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.222s	user 0.003s	sys 0.005s
I20260709 13:19:37.876277 13848 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:37.935118  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3070) took 2034 ms. Trace:
I20260709 13:19:37.935626  6595 rpcz_store.cc:276] 0709 13:19:35.901148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.901314 (+   166us) service_pool.cc:224] Handling call
0709 13:19:35.956924 (+ 55610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.197644 (+240720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.197674 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.231594 (+ 33920us) log.cc:844] Serialized 26 byte log entry
0709 13:19:36.478735 (+247141us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:36.478791 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.478880 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:37.931505 (+1452625us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:37.934720 (+  3215us) raft_consensus.cc:1685] finished
0709 13:19:37.934752 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:37.935040 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7126,"raft.queue_time_us":211171,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":28342,"spinlock_wait_cycles":9983744,"thread_start_us":430464,"threads_started":2,"wal-append.queue_time_us":421521}
W20260709 13:19:37.937976  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.941205  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:37.941977  7230 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:37.942124  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.943280  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.945616  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.945948  6589 raft_consensus.cc:2468] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:19:37.946225  6668 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:37.910472 13802 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:37.949313  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.950066  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.953553  6589 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:37.967830  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.969079  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.969677  6595 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:37.949237 13950 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:37.971606  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:37.971848  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:37.971431 13933 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:37.951527 13463 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:37.975520 13933 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:37.984284  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:37.987820  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:37.988457  6595 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:19:37.990348 13947 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:37.990818 13947 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:37.991194 13947 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:37.994448  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.008181  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:19:38.009101  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 128260us
I20260709 13:19:38.011174  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:37.989058  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:38.015810  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:37.987490 13950 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:38.029467  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:38.008314 13463 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:38.032207 13958 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:19:38.033850 13463 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.033977  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.034911  6595 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:19:38.035332  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1548) took 1062 ms. Trace:
I20260709 13:19:38.035619  6595 rpcz_store.cc:276] 0709 13:19:36.972940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.030752 (+1057812us) service_pool.cc:224] Handling call
0709 13:19:38.035277 (+  4525us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.030494 13331 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:38.037333 13331 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:38.037698 13331 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:38.038102 13331 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:38.040730  6592 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:38.041410  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3086) took 1071 ms. Trace:
I20260709 13:19:38.041702  6592 rpcz_store.cc:276] 0709 13:19:36.969537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.040291 (+1070754us) service_pool.cc:224] Handling call
0709 13:19:38.040554 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.041154 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.041331 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.043958 13933 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.028398 13880 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.044430 13837 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:38.044816 13837 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:38.045483 13682 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:38.045691 13837 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.047451  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1546) took 1075 ms. Trace:
I20260709 13:19:38.047282 13713 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.047861  6588 rpcz_store.cc:276] 0709 13:19:36.972394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.036856 (+ 64462us) service_pool.cc:224] Handling call
0709 13:19:38.047264 (+1010408us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400212854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:38.047396 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373712768}
I20260709 13:19:38.049540 13713 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:38.050603 13837 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:38.046176 13880 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.050644 13802 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.052909  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1542) took 1081 ms. Trace:
W20260709 13:19:38.050019  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 126048us
I20260709 13:19:38.053262  6592 rpcz_store.cc:276] 0709 13:19:36.971284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.042577 (+1071293us) service_pool.cc:224] Handling call
0709 13:19:38.042859 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.043054 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.043079 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:38.052619 (+  9540us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:38.052669 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.052735 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.052857 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.054219  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:38.057727  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:38.015168 13887 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:38.055534 13950 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:38.066685 13950 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:38.060385  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:38.061818  6595 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:38.063997 13911 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:38.064697  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:38.064860  6593 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:38.065367  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:38.070041  7229 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:38.070668  7231 raft_consensus.cc:2393] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:38.017302 13848 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:19:37.986758 13881 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:38.070565  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62465.2us
I20260709 13:19:38.071255  6596 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:38.065928 13614 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:38.066308  6591 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:38.067103 13950 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:38.074357  6593 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:19:38.088451  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:38.077378 13914 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.080118  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3085) took 1364 ms. Trace:
I20260709 13:19:38.091027  6595 rpcz_store.cc:276] 0709 13:19:36.716229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.036253 (+1320024us) service_pool.cc:224] Handling call
0709 13:19:38.036539 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.069966 (+ 33427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.079889 (+  9923us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b5400093dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:38.080065 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45841152}
I20260709 13:19:38.081713 13947 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.083979 13881 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.071627 13572 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:38.075476  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1544) took 1103 ms. Trace:
I20260709 13:19:38.093242  6588 rpcz_store.cc:276] 0709 13:19:36.971834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.056386 (+1084552us) service_pool.cc:224] Handling call
0709 13:19:38.072220 (+ 15834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.072389 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.072413 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:38.072457 (+    44us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:38.075186 (+  2729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.075276 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.075408 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.089398  6593 pending_rounds.cc:85] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 11
I20260709 13:19:38.095000  6593 pending_rounds.cc:107] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:19:38.089434  7229 raft_consensus.cc:2468] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:38.075552  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.074203 13932 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:38.096517  6593 raft_consensus.cc:2889] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:38.097608  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1549) took 1124 ms. Trace:
I20260709 13:19:38.097980  6593 rpcz_store.cc:276] 0709 13:19:36.973208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.070558 (+1097350us) service_pool.cc:224] Handling call
0709 13:19:38.070886 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.097142 (+ 26256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.097552 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.102447  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.119280  6595 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:38.120189  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3122) took 1040 ms. Trace:
I20260709 13:19:38.120591  6595 rpcz_store.cc:276] 0709 13:19:37.079533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.094498 (+1014965us) service_pool.cc:224] Handling call
0709 13:19:38.105053 (+ 10555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.119857 (+ 14804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.120117 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.123667  6588 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:38.121058  6562 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:38.091624 13614 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:38.126806  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:38.094580 13572 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:38.101248 13932 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.133141  6593 raft_consensus.cc:1275] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:19:38.133579  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 144618us
I20260709 13:19:38.135320  6588 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:38.101435 13911 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:38.134964  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.135949  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.119535 13848 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:38.220830  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.128165 13887 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:38.132481 13572 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:38.134663 13932 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.129074 13960 raft_consensus.cc:2804] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:19:38.240976  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.245587 13914 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.246299 13914 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:38.246665 13914 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:38.247057 13914 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:38.253402 13614 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:38.257738  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.232396 13964 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:38.239995 13962 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:38.243125 13960 raft_consensus.cc:697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:38.232820 13848 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.244419 13887 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.272070 13964 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:38.270943 13965 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:38.231792 13911 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.295022 13911 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.295482 13965 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:38.296025 13965 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:38.298885 13960 consensus_queue.cc:237] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.284154 13947 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.301941  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1536) took 1444 ms. Trace:
I20260709 13:19:38.313977  6596 rpcz_store.cc:276] 0709 13:19:36.875035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.039250 (+1164215us) service_pool.cc:224] Handling call
0709 13:19:38.071075 (+ 31825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.087121 (+ 16046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.301709 (+214588us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b5400210a50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:38.301876 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72505216}
I20260709 13:19:38.302856  6519 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.316583  6519 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.304430 13887 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.305979  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:38.278636 13835 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.322546 13835 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:38.328018 13848 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.331255  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:38.332099 13837 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:38.332758  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:38.333658 13943 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:38.333877  6488 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:38.334079 13943 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.334546 13943 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:38.334780 13771 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:38.335316 13771 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:38.335405 13838 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:38.336580 13838 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:38.336290 13771 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.336926 13838 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:38.342895 13838 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.344149 13838 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:38.334149 13837 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:19:38.351297 13768 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:38.351765 13768 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:38.352438  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3073) took 2346 ms. Trace:
I20260709 13:19:38.352867  6597 rpcz_store.cc:276] 0709 13:19:36.002597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.129293 (+126696us) service_pool.cc:224] Handling call
0709 13:19:36.183993 (+ 54700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.215701 (+ 31708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.215733 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.251046 (+ 35313us) log.cc:844] Serialized 26 byte log entry
0709 13:19:36.553685 (+302639us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:36.553729 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.553802 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:38.348194 (+1794392us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b24000f8588. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5e1a9c4 00007f04b5e18acb 00007f04b5fd3dc9 00007f04b5fd3a55 00007f04b5fd2fb0 00007f04b8c49488 00007f04b8c49be8 00007f04b8c21398 00007f04b8c1c76a 00007f04b8c1bbb6
0709 13:19:38.348820 (+   626us) raft_consensus.cc:1685] finished
0709 13:19:38.348852 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.349051 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11594,"raft.queue_time_us":58001,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3826,"spinlock_wait_cycles":33417600,"thread_start_us":159724,"threads_started":1,"wal-append.queue_time_us":401040}
I20260709 13:19:38.355702 13468 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:38.359211  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:38.360112  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1543) took 1388 ms. Trace:
I20260709 13:19:38.360491  6591 rpcz_store.cc:276] 0709 13:19:36.971550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.044421 (+1072871us) service_pool.cc:224] Handling call
0709 13:19:38.044661 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.074912 (+ 30251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.359882 (+284970us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b54002fc6d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:38.360053 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49382784}
I20260709 13:19:38.367022 13846 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:38.367564 13846 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.367569 13572 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.371870 13846 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.372756  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:38.373276 13572 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:38.374529  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 239634us
I20260709 13:19:38.375094  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2705) took 1495 ms. Trace:
I20260709 13:19:38.376595  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2874) took 1478 ms. Trace:
I20260709 13:19:38.375535  6663 rpcz_store.cc:276] 0709 13:19:36.879775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.879954 (+   179us) service_pool.cc:224] Handling call
0709 13:19:36.894893 (+ 14939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.957068 (+ 62175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.957105 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.978413 (+ 21308us) log.cc:844] Serialized 26 byte log entry
0709 13:19:38.055892 (+1077479us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b18003283c0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1 00007f04b903ca0f
0709 13:19:38.262054 (+206162us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:38.272843 (+ 10789us) log.cc:844] Serialized 14 byte log entry
0709 13:19:38.273087 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.273184 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:38.374817 (+101633us) raft_consensus.cc:1685] finished
0709 13:19:38.374915 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.375040 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5548,"prepare.queue_time_us":10605,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":1959,"raft.queue_time_us":28687,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":175,"spinlock_wait_cycles":10601728,"thread_start_us":205977,"threads_started":1,"wal-append.queue_time_us":1394477}
I20260709 13:19:38.376698 13331 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.377010  6521 rpcz_store.cc:276] 0709 13:19:36.898581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.898729 (+   148us) service_pool.cc:224] Handling call
0709 13:19:36.904048 (+  5319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.090780 (+186732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.376363 (+1285583us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400044054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:38.376527 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404909440}
I20260709 13:19:38.379331 13331 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:38.380517 13950 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.380913  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.375263  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:38.382154  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 195590us
W20260709 13:19:38.397085  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 55373.9us
I20260709 13:19:38.398491  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.399118  6519 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:38.399951  6519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1464) took 1787 ms. Trace:
W20260709 13:19:38.401084  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 265233us
W20260709 13:19:38.401741  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.402319  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.403155  6589 raft_consensus.cc:2410] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
W20260709 13:19:38.381774 13921 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 1.054s	user 0.004s	sys 0.000s
I20260709 13:19:38.405997  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.406242  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.407456  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.408416  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.409592  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.409875  6594 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:38.410607  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.409193  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.412230  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.412329  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.412539  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.411983  6521 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:38.412976  6595 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:38.415086  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.415778  6588 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:38.416050  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.417174  6520 raft_consensus.cc:2410] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:19:38.388073 13846 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:38.400362  6519 rpcz_store.cc:276] 0709 13:19:36.612538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.612706 (+   168us) service_pool.cc:224] Handling call
0709 13:19:36.613678 (+   972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.306929 (+693251us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b540029fca0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8ff8dc0 00007f04b90831c0 00007f04b9084231 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c
0709 13:19:38.302475 (+995546us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b540029fca0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b8fff06f 00007f04b9077f60 00007f04b90797de 00007f04b9083b49 00007f04b9084416 00007f04b9080d8f 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62
0709 13:19:38.399700 (+ 97225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.399898 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109067904}
I20260709 13:19:38.405683 13331 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:38.407615 13628 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 13:19:38.425554  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.425874  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:38.429008  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:38.411800 13962 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:38.430934  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.432209  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:38.444936  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.416250 13627 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:38.440466  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:38.414697 13944 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:38.447579  7229 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:38.400638 13914 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.440276  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.467350  6664 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:38.404618 13932 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.468561  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.453356  6666 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:38.429719 13846 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.424185 13911 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:38.405458 13848 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:19:38.414786 13837 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:19:38.462854 13972 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:38.463058 13835 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:38.463378 13950 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.424304 13802 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:19:38.352861 13768 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.464570  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.465126 13944 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.465597  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.472946  6671 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:38.470995  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3078) took 1849 ms. Trace:
I20260709 13:19:38.479588  6594 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:38.480161  6596 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:38.480443  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.473781  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:38.423872 13947 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:19:38.466024 13962 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:38.477407  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:38.458045 13943 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.481698  7229 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:38.482148  6664 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:38.411339 13965 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.499085 13965 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.499929 13475 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:38.500963 13475 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:38.501199  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:38.502410  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.503049  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.504093  6517 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:38.496472 13962 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.507547 13962 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:38.511449 13944 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.514225  6671 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:38.479867  6590 rpcz_store.cc:276] 0709 13:19:36.621592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.688689 (+ 67097us) service_pool.cc:224] Handling call
0709 13:19:36.689017 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.689396 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:36.689428 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.693229 (+  3801us) log.cc:844] Serialized 26 byte log entry
0709 13:19:37.042276 (+349047us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:37.042319 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.042389 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:38.381265 (+1338876us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:38.466860 (+ 85595us) raft_consensus.cc:1685] finished
0709 13:19:38.466885 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.470928 (+  4043us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7119,"raft.queue_time_us":521,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5407,"spinlock_wait_cycles":28851072,"thread_start_us":265938,"threads_started":1,"wal-append.queue_time_us":396775}
W20260709 13:19:38.517690 13922 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.812s	user 0.000s	sys 0.011s
I20260709 13:19:38.523813  6670 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:38.526183  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:38.475878 13972 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:38.527415 13972 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:38.533756  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.534534  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.534533  6596 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:38.501305 13475 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:38.486120  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:38.540668  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.544214  6662 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:38.481715 13677 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:38.583776  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.494650 13940 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:38.583259 13971 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:38.614238  6513 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:38.618871 13971 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.619248 13972 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:38.625909  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 195031us
I20260709 13:19:38.615532  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.602732  6590 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:38.551050 13943 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:38.633652  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.635188  6665 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:19:38.640223  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.640974  6665 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:38.644202  6668 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:19:38.628430 13972 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:38.591296 13627 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.641134  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.645555  6588 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.549016 13768 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:38.653606  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:38.642510  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:38.667430  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.687811  6590 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:38.632318 13971 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.698607  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.628984 13940 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:38.700045  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.700346  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.701118  6668 raft_consensus.cc:2410] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:19:38.635221 13965 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:19:38.631305 13835 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:19:38.635557 13962 consensus_queue.cc:1048] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:19:38.699638 13880 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.722488 13880 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.724535 13880 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.725735  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:38.726354  7235 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:38.706423  6670 raft_consensus.cc:2393] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:19:38.706758  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 287800us
I20260709 13:19:38.710368 13914 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:38.704038 13940 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:38.710530 13965 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:19:38.727402  7200 connection.cc:256] RPC call timeout handler was delayed by -1.1706589890000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:19:38.728559  7200 connection.cc:256] RPC call timeout handler was delayed by -1.1706589890000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 13:19:38.595487 13677 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.732379  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.651418 13627 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.743330 13463 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.743849 13463 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.744223  6667 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:38.745049 13822 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:38.745869 13822 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:38.746335 13822 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:38.745932 13463 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.747216 13822 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.748615 13822 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:38.749470  6597 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:38.729149 13331 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:38.734234 13848 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:38.757280 13848 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.758904  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.759528 13848 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.736145 13976 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:38.760648 13976 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.731009 13460 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:38.763111 13460 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.765261 13460 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:38.767639 13627 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:38.731331 13965 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:38.770931  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:38.775565  6519 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:38.779194  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.782824  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.779055  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.796525  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.797636  6664 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:38.786516 13460 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:38.789819 13888 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:38.801472  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.798820 13965 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.802968  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.802595 13331 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:38.803640  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:38.804319  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.805825  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.788437 13976 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:38.818285  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.813491  6667 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:38.814692  6670 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:38.811666  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.816402  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:38.821089  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.788336 13846 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:38.819869  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.812690  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.823251  6592 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:38.823777  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:38.824003  6664 raft_consensus.cc:2410] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:19:38.827052  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.828472  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.826457 13965 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:38.829259  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1545) took 1857 ms. Trace:
I20260709 13:19:38.831218  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.835657  6592 rpcz_store.cc:276] 0709 13:19:36.972113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.821608 (+1849495us) service_pool.cc:224] Handling call
0709 13:19:38.821933 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.825364 (+  3431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.829205 (+  3841us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.837299  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:38.835269 13976 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:38.839253  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.838286  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.842710  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.846472  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.845556  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.826378 13880 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:19:38.857839  6590 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:19:38.858877  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.859807  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.860694  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.848793  6594 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:38.846361  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:38.836181 13846 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.851595 13475 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:38.866823  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.835912 13571 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:38.837924 13614 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.849747  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:38.866494  6662 raft_consensus.cc:1240] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:38.853153  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:38.872119  6666 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:38.867959  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.870656  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:38.874455  6670 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:38.875684  6595 raft_consensus.cc:2393] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:38.836969 13848 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:19:38.856027 13572 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:38.854974 13908 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:38.893321  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.874162 13982 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:19:38.895548  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.890477  6588 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
W20260709 13:19:38.914752  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.921648  6594 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:19:38.914497  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:38.925058  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 78683.3us
I20260709 13:19:38.882332 13949 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:38.894173 13960 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:38.936460  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.940822  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.894744 13835 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:38.907619 13971 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.906520 13837 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.910148 13848 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:38.913266 13947 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:38.953614  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:38.954358  6671 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:38.956331  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 118124us
I20260709 13:19:38.930809 13627 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:38.903348 13962 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:19:38.960219  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:38.960816  6667 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.961699  6561 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:38.965003  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.967360  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.970161  6666 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:38.975835  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.977059  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.977193  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:38.978186  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.978627  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.979306  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:38.980414  6517 raft_consensus.cc:1275] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:38.980754  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:38.981848  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:38.980440  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:38.978309  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:38.983927  6514 raft_consensus.cc:2368] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 24. Current term is 25.
I20260709 13:19:38.985707 13993 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:38.985785  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.986546 13993 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:38.953251 13939 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:19:38.952067 13848 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:38.948375 13992 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:19:38.959283 13775 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:39.001808  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:39.001982  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:38.958738 13837 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:39.011497  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 170212us
I20260709 13:19:39.012101  6637 leader_election.cc:400] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 24. Current term is 25.
I20260709 13:19:39.002854 13992 consensus_queue.cc:1243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:39.012491  6637 leader_election.cc:403] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:19:38.971489 13949 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:19:39.016192  6667 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:38.998914 13972 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:39.025624  6515 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
W20260709 13:19:39.027083  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 81387.4us
I20260709 13:19:39.011201 13971 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:39.040068  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 73533.9us
I20260709 13:19:39.040892  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.043354  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.005890 13982 consensus_queue.cc:1048] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:19:39.031006  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.022779 13880 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:39.048625  6671 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:19:39.045192  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.052939  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:39.055369  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:39.055766 13988 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:19:39.056243 13677 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.056525  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:39.062084 13836 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:39.062521 13960 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:39.062645 13836 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.050763 13947 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 24. Current term is 25.
I20260709 13:19:39.058553  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.066028  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:39.067313  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.049984 13998 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:39.064894 13836 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.068361 13386 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:39.071059 13993 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:39.057067 13888 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:39.071728 13949 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:39.072691  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 1966) took 1463 ms. Trace:
I20260709 13:19:39.051067 13996 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:19:39.059823 13990 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:39.068725 13960 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.068900 13991 raft_consensus.cc:2764] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:19:39.113255  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:39.086427  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.083874 13995 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:39.089476 13887 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:39.076159 13976 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:39.081801  7234 rpcz_store.cc:276] 0709 13:19:37.609053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.609164 (+   111us) service_pool.cc:224] Handling call
0709 13:19:37.629053 (+ 19889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.636449 (+  7396us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:39.007041 (+1370592us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b18002fb360. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf
0709 13:19:39.007166 (+   125us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.053240 (+ 46074us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.072320 (+ 19080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:39.072380 (+    60us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:39.072425 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.072493 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.072611 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26570,"prepare.queue_time_us":219400,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":792,"raft.queue_time_us":12870,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":15810,"spinlock_wait_cycles":6152064,"thread_start_us":218381,"threads_started":3,"wal-append.queue_time_us":24491}
I20260709 13:19:39.105119 13940 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.127351 13887 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:39.133021 13940 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.110982 13837 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.133668 13887 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.127671 13991 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.134829 13887 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:39.083143 13949 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:39.135388 13949 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:39.111827 13992 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:39.136023 13992 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:39.121701  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.102404 13846 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:39.138393  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 81910.8us
I20260709 13:19:39.139607  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:39.138500  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.143611  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.145160  7200 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:39.146260  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:39.147132  6588 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:39.151034  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.141233 13991 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:39.152025 13965 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.148927 13870 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:39.153295  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.154060 13965 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.155278  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.155994 13965 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:39.157116  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.149722 13996 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.157859 13965 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:39.149914 13982 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:39.158946  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.158983  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:39.159998 13982 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:39.160156  6637 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.054247 13848 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.160919 13982 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:39.161412 13982 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:39.162263 13982 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:39.174721  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 95514.4us
I20260709 13:19:39.178695 13880 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:39.148463 13992 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.175702  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:39.203023 13992 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:39.184937  6668 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:39.191138  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.203223  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:39.204294 13978 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.513s	user 0.003s	sys 0.000s
I20260709 13:19:39.205260 13990 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:39.205688 13990 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.206017 13990 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:39.191488  6589 raft_consensus.cc:2410] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:39.174744 13572 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:39.189798 13976 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
W20260709 13:19:39.207782  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.212734 14003 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:39.213125 13989 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:19:39.174300 13822 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:39.215363  6669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1449) took 1231 ms. Trace:
I20260709 13:19:39.220472  6669 rpcz_store.cc:276] 0709 13:19:37.984233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.984408 (+   175us) service_pool.cc:224] Handling call
0709 13:19:37.985880 (+  1472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:38.137476 (+151596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.137511 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:38.260930 (+123419us) log.cc:844] Serialized 52 byte log entry
0709 13:19:38.480814 (+219884us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:38.480861 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.480926 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:39.203720 (+722794us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:39.210665 (+  6945us) raft_consensus.cc:1685] finished
0709 13:19:39.210685 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.215303 (+  4618us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1582,"raft.queue_time_us":21261,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":7495,"spinlock_wait_cycles":20384128,"thread_start_us":314903,"threads_started":2,"wal-append.queue_time_us":318752}
I20260709 13:19:39.220139 13976 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.220521 13822 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:39.239327  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:39.259162  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:39.260114  6519 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:39.266147  6519 raft_consensus.cc:3055] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:39.266707  6519 raft_consensus.cc:740] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:39.267715  6519 consensus_queue.cc:260] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.269302  6519 raft_consensus.cc:3060] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:39.357879  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 143679us
W20260709 13:19:39.393016  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:39.479455  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 247115us
I20260709 13:19:39.255653 13822 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:39.498733 13911 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:39.504750  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:39.215083 13460 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 13:19:39.243947 13940 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.117s
I20260709 13:19:39.236609 13696 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.121s
I20260709 13:19:39.220381 13838 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:19:39.231961 13944 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.117s
W20260709 13:19:39.516664  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.515384  6563 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:39.519695  6519 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:19:39.515733  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 252368us
I20260709 13:19:39.499840 14011 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:39.524735  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.504945 13815 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:39.564107  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:39.525799 13911 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:39.567085 13848 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:39.568764 13460 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:39.569166 13838 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:39.526135 13995 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:39.559782 13837 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:39.573191  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 57936.3us
I20260709 13:19:39.502200 13870 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:39.630291 13614 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:39.554608 13949 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.630841 13880 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:39.627260 13960 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.570101 13944 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:39.659313  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 134715us
I20260709 13:19:39.677286  6596 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:39.677106  7200 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.651945 13838 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:39.659978  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 85529.4us
I20260709 13:19:39.585057 13991 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:39.576256 14011 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.665859  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:39.586277 13460 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:39.673044  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:39.651860 14003 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:39.650828 13998 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.575176  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:39.665544 13614 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:39.678295 13888 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:39.676550 13944 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:39.689950  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:39.663841 13837 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.695036 13949 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:39.696241 13837 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.691172 13888 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.697433 13888 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:39.689459 13991 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:39.699796 13991 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:39.691753 13870 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:39.688681 14003 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:39.694391 14002 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:19:39.704687 13944 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.705811 13911 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:39.707109 13911 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.707625 13911 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.709550 13838 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:39.706427 13944 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:39.686517 13880 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:39.715922 14011 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:39.678925 13960 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.718984 13960 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.722807 13846 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:39.723775 13460 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.725106 13460 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:39.669824 13976 consensus_queue.cc:1059] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
W20260709 13:19:39.728782  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48059.8us
I20260709 13:19:39.729220 14013 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:39.729110 13614 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.730800 14015 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:39.710881 13838 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:39.731969 14015 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:39.739567 13976 raft_consensus.cc:3055] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:39.740036 13976 raft_consensus.cc:740] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:39.740965 13976 consensus_queue.cc:260] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.742429 13976 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:39.743700 13990 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.010s
I20260709 13:19:39.743294 14015 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.744122 13990 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.744791 14015 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:39.745954 13992 consensus_queue.cc:1059] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:39.746670 13992 consensus_queue.cc:1243] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:19:39.747977  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.750460 13932 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:39.752393 13965 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:39.752810 13965 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:39.753187 13965 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:39.755155  6591 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:39.758231  7229 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:39.687546 13998 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.776435  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.731444 13996 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:19:39.787339  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:39.787407  7229 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:39.790339  7232 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:39.762506 13990 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.806483  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.768811 13815 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:39.808414  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:39.821735 13998 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.822933  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:39.823627  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 546736us
W20260709 13:19:39.831969  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 240818us
I20260709 13:19:39.833114  6517 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:39.798522 13911 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.788242 13822 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:39.839129  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.840267 13822 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:39.841331  7230 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:39.841566 13815 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:39.840399  7231 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:39.848446 13888 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:39.851588  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 121687us
I20260709 13:19:39.852723  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.853623  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.853506  7228 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:39.858904  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:39.859139  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 169279us
I20260709 13:19:39.860584  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:39.860844  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 285739us
I20260709 13:19:39.866526  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:39.867157  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:39.867466  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:39.867790  6514 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:39.868218  6513 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:39.868431  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:39.867673 14019 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.869160 14019 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:39.867267  6517 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:39.871712 14019 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:39.872531 14019 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.873003  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.873008 14019 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:39.876022 14019 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:39.877517  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.878691  6514 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:39.882658  7231 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:39.889524  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.890009  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:39.893728 13998 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:39.895635 13908 raft_consensus.cc:3055] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:39.895792 13960 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.896088 13908 raft_consensus.cc:740] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:39.896404 13960 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.897013 13908 consensus_queue.cc:260] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.890699  7228 raft_consensus.cc:2376] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 19. Re-sending same reply.
I20260709 13:19:39.851508 14017 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:39.898252  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.901060  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:39.903759  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2723) took 1098 ms. Trace:
I20260709 13:19:39.906282  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:39.907589  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:39.908167  6596 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:39.910923  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.904181  6668 rpcz_store.cc:276] 0709 13:19:38.802041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.814359 (+ 12318us) service_pool.cc:224] Handling call
0709 13:19:39.900026 (+1085667us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54001cb1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:39.900120 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656068480}
I20260709 13:19:39.912060  6563 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:39.914196  7230 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:39.915894  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.916510  7230 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:39.902503 13960 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:39.920181  7234 raft_consensus.cc:1240] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: [19.15-21.16]
I20260709 13:19:39.922745  6561 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:39.901772 13990 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:39.931789  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:39.932467  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.933061  7234 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:39.939268  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:39.956987  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.929754 13848 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:39.912580 13965 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:39.966171  7233 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:39.979862  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:39.980778  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.986048  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.986801  7233 raft_consensus.cc:2393] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 18: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:39.987416  7237 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:39.930603 14022 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:39.989588  7232 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:39.993975  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:39.994894  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.965790 13911 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:39.996587  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.009717  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:39.974102 13888 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:40.009622  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 97795.2us
I20260709 13:19:40.010500  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:40.011078  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.011458  7230 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:40.012527  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:39.939309 13990 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.013139  7233 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 13.
I20260709 13:19:40.018375  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.015954  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:39.984378 14021 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:40.019673  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 97041.7us
I20260709 13:19:39.990396 13848 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:40.025107  6593 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:19:39.993484 14022 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.027464  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:40.029820  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:40.033305  7233 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:40.034859  6593 pending_rounds.cc:85] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 10
I20260709 13:19:40.034947  6590 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:19:40.035305  6593 pending_rounds.cc:107] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:19:40.035567  6593 raft_consensus.cc:2889] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:40.036064  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:40.036638  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.037209  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.039037  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:40.042263  7230 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:40.043766  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:40.046742  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:40.047358  6590 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:40.028972 14020 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.055562  6592 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:40.056564  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:40.059975  6520 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:40.057556  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:40.060290  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.063014  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.063647  7237 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:40.081517  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.082226  7236 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:40.036583 14022 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:40.084033  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.087229  6516 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:40.085254  6487 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:40.021804 13911 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:40.088948 14032 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:19:40.089293 14025 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:19:40.089143  7236 raft_consensus.cc:1275] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:19:40.053478 13914 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:40.095103  7237 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:40.029477 14021 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.106001  6590 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
W20260709 13:19:40.112344  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:40.115139 14029 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:19:40.116041 13990 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.083992 14031 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:19:40.117347 13965 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:40.118820 14026 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:19:40.120112 13911 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.098726 13914 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.122107 14021 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:40.122941  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.144822  7233 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
W20260709 13:19:40.170878  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 94214us
I20260709 13:19:40.080224 14020 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.175673  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2884) took 1730 ms. Trace:
I20260709 13:19:40.176126  6522 rpcz_store.cc:276] 0709 13:19:38.444887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.445313 (+   426us) service_pool.cc:224] Handling call
0709 13:19:38.445649 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.510091 (+ 64442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.510121 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:38.510162 (+    41us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:38.510193 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.175425 (+1665232us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b5400216bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:40.175469 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.175600 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147781376}
I20260709 13:19:40.175983  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:40.170383 13914 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:40.187620  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:40.187961  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:40.188289  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:40.190069  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:40.192834  6591 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:40.173375 14018 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.163892 13848 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.206719  6592 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:40.206943 14051 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:40.207790 14051 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.215171  6520 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:40.164672 14030 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.060s
I20260709 13:19:40.225854  6520 raft_consensus.cc:1240] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
W20260709 13:19:40.211246 14023 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:19:40.194885 14040 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:40.210855 13983 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:40.259387  6518 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
W20260709 13:19:40.261909  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 176715us
W20260709 13:19:40.264799 14012 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.015s	sys 0.023s
I20260709 13:19:40.267378  6520 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:40.177716 14041 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
W20260709 13:19:40.248688 14029 log.cc:927] Time spent T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:19:40.270916 14012 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.016s	sys 0.023s
I20260709 13:19:40.255571 14040 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:40.370810  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 375499us
I20260709 13:19:40.372897  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:40.376307  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:40.381572  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.383884  7229 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:40.224792 14018 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.385735 13677 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:40.383177 14036 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:40.382408 14040 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:40.387485 14028 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 13:19:40.387765 13976 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.001s
I20260709 13:19:40.384869 13911 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:40.384348 14051 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.383229 14038 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:40.397538 14036 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:40.503304  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.401378 13677 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.485394 14036 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:40.515544  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 137277us
I20260709 13:19:40.408897 14017 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:40.419376 14038 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:40.519934 14028 log.cc:927] Time spent T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:19:40.553012  7234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.002s	sys 0.010s
W20260709 13:19:40.553503  7234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.002s	sys 0.010s
I20260709 13:19:40.395567 13848 consensus_queue.cc:1048] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.121s
I20260709 13:19:40.523031 14038 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:40.563581  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:40.552291 13960 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:40.570638  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.562441 13911 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.580320 13911 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.586362 14003 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:40.592232  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:40.597980  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.598191  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.599087  6520 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:40.604545  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.605386  6517 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:40.611538 14045 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:40.612154 14045 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:40.613251 14045 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.604473 14064 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 13:19:40.615886 14045 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:40.619153  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 380160us
I20260709 13:19:40.599674 14020 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.630966 14017 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.631995 13983 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:40.632741 13983 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:40.634879 14020 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:40.635381 14020 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.635773 14020 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:40.637363  7200 leader_election.cc:336] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Timed out: RequestConsensusVote RPC to 127.5.72.130:37977 timed out after 1.721s (SENT)
I20260709 13:19:40.639472 13888 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.639611 14018 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.640002 13888 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:40.643690  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:40.652185  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 389196us
W20260709 13:19:40.644455  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.602082 13960 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.647636 13943 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:40.661351  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:40.663146  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:40.665454  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:40.666703  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:40.669498  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 110450us
I20260709 13:19:40.672559  7235 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:19:40.659412 13908 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:40.677464  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.679647  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.679901  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:40.680158  7236 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:19:40.678721  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:40.662961 13943 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:40.685863  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.681919 13822 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:40.688270 13775 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:40.675078 13848 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.694564  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:40.695891  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:40.698163  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:40.700743  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:40.702003  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 108830us
I20260709 13:19:40.702292  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:40.706991 14011 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:40.707479 14003 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:40.707410  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:40.708197 14003 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.708779 14011 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:40.709367  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:40.710372 14011 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:40.711006  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:40.711666 14003 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.711992 14011 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:40.706115  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:40.712836 14011 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:40.712954  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:40.714051 14015 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:40.715288 14015 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:40.713703  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:40.699486  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:40.716632 14068 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:40.716321 14015 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.717249 14068 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:40.717613 14015 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:40.718102 14068 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.704905  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:40.698887 13768 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:40.724445  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.703159 13775 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:40.747040 14058 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:40.747576 14058 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:40.747953 14058 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:40.723577 14070 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:40.751168 14070 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.753407 14070 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.723075 14067 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:40.683405 14036 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.754861 14055 raft_consensus.cc:2764] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:19:40.755612 14059 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:19:40.756680 14036 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.757594 14071 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:40.758136 14071 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.758878 13775 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:40.760284 14071 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.760859 14071 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:40.761322 14071 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:40.762744 14022 consensus_meta.cc:287] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.666s	user 0.001s	sys 0.013s
I20260709 13:19:40.763186 14022 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.765245 14022 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.765933 14067 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:40.704125 14021 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.766899 14021 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.767268 14039 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:40.767751 14039 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:40.768922 14021 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:40.768615 14039 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.769485 14021 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:40.769960 14021 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.771701 13960 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:40.772333 13943 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.773277 14070 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:40.773592 13943 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:40.775771  7235 raft_consensus.cc:2393] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:40.775885 14069 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.776304 14014 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:40.776536 14069 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.786629 14021 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:40.787746 14003 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.789114 14003 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:40.790783  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.794077 14021 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:40.794488 14021 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:40.794652 13888 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:40.701515 13677 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.698367 13848 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.801626 13880 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:40.802276 13880 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.802781 13880 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:40.803994 14049 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.320s	user 0.014s	sys 0.017s
I20260709 13:19:40.725450 14030 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 13:19:40.823537  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:40.824194  7234 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:40.794950 14021 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:40.826406  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:40.827610  7235 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:40.836975 13848 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:40.837626  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1452) took 1995 ms. Trace:
I20260709 13:19:40.838027  6664 rpcz_store.cc:276] 0709 13:19:38.841851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.842064 (+   213us) service_pool.cc:224] Handling call
0709 13:19:38.842364 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:39.190812 (+348448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:39.190842 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:39.190879 (+    37us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:39.190909 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.837321 (+1646412us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54000cb554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:40.837405 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.837554 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800890112}
I20260709 13:19:40.853035  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:40.854223 14073 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:40.854861 14073 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.856300 14069 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:40.857789 14073 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.852393 13677 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:40.858958 13677 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.860558 14071 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:40.861146 13677 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.853742  6664 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:40.863818  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:40.864408  6664 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:40.869102 14003 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.871299  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.872004  7231 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:38.974093  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:40.877875  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:40.887391  6663 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:40.887650  6671 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:40.890129  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:40.890455  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:40.891373  6669 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:40.894641  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:40.897526 13677 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:19:40.899487 14040 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.900120  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:40.900701  7231 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:40.901678 14040 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.902177  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:40.903079 14064 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:40.903450 14020 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.916616 14016 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:40.917502  6666 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:19:40.930207  6589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1558) took 2511 ms (client timeout 1721 ms). Trace:
W20260709 13:19:40.931021  6589 rpcz_store.cc:269] 0709 13:19:38.408030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.408219 (+   189us) service_pool.cc:224] Handling call
0709 13:19:40.918816 (+2510597us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b540023cdd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:40.918967 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1798980992}
I20260709 13:19:40.894052 14003 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:40.898646 13880 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.903494 14064 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.951067  6636 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:40.958988 14047 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.473s	user 0.009s	sys 0.003s
I20260709 13:19:40.960170 14072 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:40.964341 14020 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:40.938084 13846 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.965564 13846 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:40.965050 14064 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:40.993300  7231 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:40.939465 14003 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.943171 14073 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:40.940871 14016 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.000720  7236 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:40.952648 13677 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.994278  6636 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:19:41.008260  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 303391us
I20260709 13:19:41.009555  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:40.914237  7235 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 14.
W20260709 13:19:41.015232  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.019595 14075 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:41.020531 14075 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:41.021843 14075 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:41.003541  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 290672us
I20260709 13:19:41.029354  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:41.029569  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:40.994968 13571 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:40.996930 13990 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:41.032793  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 319139us
I20260709 13:19:40.940657 14038 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:40.998588 14003 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:41.035627  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.035709  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.049652  6667 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:41.038116  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:41.038072  6665 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:19:41.040041  6590 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:19:41.041679  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.050956  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.038760  6663 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:41.051767  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:40.998720 14073 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:41.002799 13848 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.053606  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.055511  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.043998  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.001425 13993 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:41.049660  6670 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:40.999567 13998 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.068856 13998 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.053843  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.071365  6594 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:41.056272  6593 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:19:40.996723 14020 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 13:19:41.058166  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.075172  6662 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 15.
I20260709 13:19:41.082789 14074 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:19:41.007627 14016 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.062093  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:41.025331 13677 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.064666  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.122939  6671 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:41.065112 13993 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.053588 13990 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.098225 13848 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.012019 14072 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:41.024984 13960 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.111100  6662 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:41.100181 14004 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.116655  7234 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:41.032192 13846 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.038571 14022 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:41.126973  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 64964.7us
I20260709 13:19:41.128571  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.131290 13983 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.137714 13960 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.138532 14004 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.139926  7230 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:19:41.132328 13696 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:41.048645 13571 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.128168 14017 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:41.211850  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.141235 14021 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:41.142045 14044 log.cc:927] Time spent T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.594s	user 0.008s	sys 0.000s
I20260709 13:19:41.142249 13993 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.184814 13880 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.148689 13990 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.150382 14073 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.132247 14070 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.153328 14064 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.181460 14072 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.256552 14072 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.257553 13960 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.218027 13696 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:41.184741  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:41.259195 13908 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.259044 13696 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.259723 13908 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:41.262326 14053 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.010s	sys 0.016s
I20260709 13:19:41.139765 13677 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:41.262744 14053 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.010s	sys 0.016s
I20260709 13:19:41.198835  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:41.209028  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 457943us
W20260709 13:19:41.263604  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.250375  6668 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:41.146950  6669 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 25.
W20260709 13:19:41.269378  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 86186.2us
I20260709 13:19:41.241838 14038 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:41.272019  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 73258.3us
I20260709 13:19:41.273769  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.277848  6514 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:41.278740  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.273499  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.274794  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.280036  6520 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:41.284268  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.285722  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:41.267091 14054 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.779s	user 0.006s	sys 0.007s
I20260709 13:19:41.290373  7233 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:19:41.278342  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:41.329174  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 50909.8us
I20260709 13:19:41.287971  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:41.209997 14058 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.298127  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.330150  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.343818  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:41.331646  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:41.266597 14076 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.218072 14022 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.288821  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.411803  7228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1821) took 1502 ms. Trace:
I20260709 13:19:41.414345  6588 raft_consensus.cc:2368] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18.
W20260709 13:19:41.413285  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:41.507588  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.353636 14014 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.508426 13838 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.509965 13771 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.509053 14014 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.510676 13771 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.513460 13771 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.509132 13838 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.517174 13838 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.300595  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.268831 14020 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.271301 14087 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.263780 14070 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.518262 14020 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.518486 14014 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.520668 14070 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.521132 14020 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.518572 14087 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.522080 13838 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:41.266675 13972 consensus_queue.cc:1059] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 13:19:41.414170  7228 rpcz_store.cc:276] 0709 13:19:39.909445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.909686 (+   241us) service_pool.cc:224] Handling call
0709 13:19:39.909968 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:39.910276 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:39.910326 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:39.910816 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:19:40.226516 (+315700us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:40.226574 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.226670 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.107866 (+881196us) spinlock_profiling.cc:243] Waited 8.5 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:41.349578 (+241712us) raft_consensus.cc:1685] finished
0709 13:19:41.349601 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.411728 (+ 62127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":586,"raft.queue_time_us":8757,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":7187,"spinlock_wait_cycles":19559552,"thread_start_us":299779,"threads_started":1,"wal-append.queue_time_us":469228}
I20260709 13:19:41.434917 14038 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:41.445699 13475 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:41.447970 14058 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.450938  7231 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
W20260709 13:19:41.451180 14084 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:19:41.250423 13880 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:41.415261 14080 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:19:41.491148  6521 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:41.491117  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1483) took 1468 ms. Trace:
I20260709 13:19:41.495270 14034 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:41.275780 14017 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:41.523643 13822 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:41.504289  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 234194us
I20260709 13:19:41.522637 13838 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.437009 13983 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.276003 13990 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.435164 14076 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.343428 13815 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.149s
I20260709 13:19:41.282871 13677 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:41.528240 14103 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.457407 14016 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.415655 13848 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.539865  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:41.540395  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:41.540959  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.541442  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.350747 13960 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:41.543484  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 2895) took 2272 ms. Trace:
I20260709 13:19:41.544201  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.277616 13775 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.528649 14015 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:41.537014  6522 rpcz_store.cc:276] 0709 13:19:40.022937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.184355 (+161418us) service_pool.cc:224] Handling call
0709 13:19:40.184708 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.191258 (+  6550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.191292 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:40.206491 (+ 15199us) log.cc:844] Serialized 26 byte log entry
0709 13:19:41.453908 (+1247417us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b180033a7e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c210d1
0709 13:19:41.454068 (+   160us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:41.454362 (+   294us) log.cc:844] Serialized 14 byte log entry
0709 13:19:41.454568 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.455558 (+   990us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.455601 (+    43us) raft_consensus.cc:1685] finished
0709 13:19:41.487145 (+ 31544us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.491059 (+  3914us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6209,"prepare.queue_time_us":895,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":104609,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":29638,"spinlock_wait_cycles":78498816,"thread_start_us":114420,"threads_started":1,"wal-append.queue_time_us":187423}
I20260709 13:19:41.552986  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:41.646320  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 93416.9us
I20260709 13:19:41.647226  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:19:41.555176 14045 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.939s	user 0.011s	sys 0.013s
I20260709 13:19:41.556555 13880 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.551091 13822 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:41.557756 14091 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:41.553983 13838 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.558516 13571 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.559067 14004 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.564392 13947 raft_consensus.cc:3055] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:19:41.518239  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 217883us
I20260709 13:19:41.575969 14087 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.665624 14096 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:41.666257 14096 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:41.576387  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.579934 14021 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.547736 13991 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:41.530223 14038 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.667428 14014 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.667995 13614 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:41.668135 14014 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.667407 13991 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.670140 14014 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.670394 14038 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:41.670735 14093 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:19:41.671211  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 1477) took 1674 ms. Trace:
I20260709 13:19:41.671581  6518 rpcz_store.cc:276] 0709 13:19:39.997279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.998012 (+   733us) service_pool.cc:224] Handling call
0709 13:19:41.671016 (+1673004us) spinlock_profiling.cc:243] Waited 9.72 ms on lock 0x7b540019b4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:41.671130 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22357888}
I20260709 13:19:41.672217 13991 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:41.674726  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.676781 13822 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.677867 13908 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.679145 13991 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:41.679634 13991 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.680294 14004 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:41.680784 14004 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:41.681483 14003 consensus_meta.cc:287] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.571s	user 0.005s	sys 0.003s
I20260709 13:19:41.681960 14003 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.684029 14003 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.681466 13991 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.685884  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:41.687589  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.688014 14076 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.688633  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.651316 13775 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.689214  6519 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:41.690662  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1763) took 1883 ms. Trace:
I20260709 13:19:41.690838 13775 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.653707 14015 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.655124 13983 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.692559 13908 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:41.695215 13983 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:41.583384 14088 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:19:41.568266 14020 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:19:41.643915  6521 rpcz_store.cc:276] 0709 13:19:39.265132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.265282 (+   150us) service_pool.cc:224] Handling call
0709 13:19:41.487141 (+2221859us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5800010598. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be05afc0 00007f04be045929 00007f04be0453d2 00007f04b75bcbd4
0709 13:19:41.490822 (+  3681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.536959 (+ 46137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.537166 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31139072}
I20260709 13:19:41.642545  6516 raft_consensus.cc:3060] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:41.698326 14046 log.cc:927] Time spent T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 1.180s	user 0.000s	sys 0.011s
I20260709 13:19:41.645711 14103 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.572294 13990 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.701298 13990 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.652424  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.701932  6520 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:41.702025 14103 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.645279 13960 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.703401 13947 raft_consensus.cc:740] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:41.704352 13947 consensus_queue.cc:260] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.698037 13983 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.705533 13947 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:41.694454 14015 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.703455 13960 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:41.695897 13908 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:41.597795 14064 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.665364  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:41.717697  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 52414.5us
I20260709 13:19:41.718904 13943 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:41.719276  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.655614 14016 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:41.724354 14016 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.724838 14016 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:41.725962  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.666754 14096 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.727802 14076 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.728307 14076 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.691038  7229 rpcz_store.cc:276] 0709 13:19:39.807733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.807868 (+   135us) service_pool.cc:224] Handling call
0709 13:19:39.808138 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.387121 (+578983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.690453 (+1303332us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400684cd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:41.690591 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323420544}
I20260709 13:19:41.730333 14076 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.722461 14072 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.719295 13943 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.732504  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.752149  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.734202  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.734790 14039 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.736227 13908 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.743060 14004 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.743381  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:41.745160  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 55009.5us
I20260709 13:19:41.769354  6670 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:19:41.587275 14022 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.746563  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.695254  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.748723  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.749629  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:41.778594  6664 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:41.778894  6517 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.750351  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.779685  6522 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:41.751027  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:41.751366  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.780339  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:41.776337  7237 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:19:41.733515  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.755385  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.656538 14075 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.759004 13943 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:41.759218  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.760002 13932 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.752667 14103 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.794943  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.796550  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.759514 14039 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.753077 14076 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.762264 13838 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.763283  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.764187  6597 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:41.764730  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:41.765954  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 62901.2us
I20260709 13:19:41.767365  6595 raft_consensus.cc:2410] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:19:41.769626  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:41.781314  7231 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:41.783587  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.806103  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:41.806885  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.756498  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.812302 14021 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:41.812707 14021 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.813076 14021 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.774523  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.815322  6670 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:41.818205  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.818924  6670 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:41.820152  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.820864  6670 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:41.821874  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:41.779925  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 84725us
I20260709 13:19:41.788475  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.827957  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.829478  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.830847  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.832219  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.752225  6516 raft_consensus.cc:2468] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:41.787107  6562 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:41.794220  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.833796  6667 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:41.834662  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:41.798574  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.800010  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:41.835860  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.836925  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.837985  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.839110  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.838503  6665 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:41.840724  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:41.851408  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.785909 14011 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:41.804474  6593 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:19:41.853509  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47123.9us
I20260709 13:19:41.794997 14086 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:41.855556  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49550.4us
I20260709 13:19:41.855720  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.855813  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:41.854380  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.859982  6671 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:41.793996 13822 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.859884  7237 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:41.857398  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.860877  6670 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:19:41.861230  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 74241.5us
I20260709 13:19:41.871466  7231 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:41.861577  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.885715  6562 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:41.886226  7228 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:41.886062  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:41.850798 13880 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.892905  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:41.897221  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.898867  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.899431  7231 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:41.899673  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:41.900516  7233 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:41.902091  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.903085  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.903999  7234 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:41.853807 13983 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:41.855553 14011 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.893806  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.853713 14022 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:41.895787  6561 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:41.801764 13932 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.890194 13614 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:41.909121 13880 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.917047  7234 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:41.795586 14075 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:41.861526 14088 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:19:41.858227 14086 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.911651 13983 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.923739 14086 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.923981 13838 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.924387 13990 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:41.894029 14021 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.924814 13990 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.925220 13990 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.925396  6593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:41.926146  6593 raft_consensus.cc:606] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:41.926646 14021 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:41.926813 13935 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:41.892937 14004 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:19:41.927306 13935 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:41.927712 13935 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:41.886771 13887 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:41.913198  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.892985 13571 raft_consensus.cc:2804] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:41.924057 13983 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:41.938265  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1777) took 1964 ms. Trace:
I20260709 13:19:41.938717  7230 rpcz_store.cc:276] 0709 13:19:39.973578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.076591 (+103013us) service_pool.cc:224] Handling call
0709 13:19:40.078211 (+  1620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.209067 (+1130856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.938031 (+728964us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54005a3554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:41.938204 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440844032}
I20260709 13:19:41.939766 14100 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:19:41.940140 14034 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:41.940321 13838 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:41.940721 14034 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:41.941227 14034 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:41.941872 13838 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:41.942811  6669 raft_consensus.cc:2410] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:41.943833 14075 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:41.942296 13838 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:41.920113 14106 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:19:41.920164 14096 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:41.921237 14105 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:41.921128  7233 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:41.893401 13991 raft_consensus.cc:3060] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:41.802947 14103 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.911270 14064 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:41.934589  6488 leader_election.cc:400] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18.
I20260709 13:19:41.967845  6488 leader_election.cc:403] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.853475 14076 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.944383  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:41.944561  6515 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:41.946744  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.968288  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.968765  7234 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:41.969029  7230 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:41.918159  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:19:41.969871  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 51852.6us
I20260709 13:19:41.972270  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:41.913300 14011 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:41.921831 13460 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.912403 13822 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.946379 13887 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.950692  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.951295  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:41.977829  7231 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:41.952006 13838 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:41.954969 13998 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.956548 13614 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:41.957091 13947 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.988049  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:41.967684 14064 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:41.994084  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:41.962168  6517 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:41.962699 14039 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.913131 14022 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:41.946146 13571 raft_consensus.cc:697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:41.920410 14111 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:19:41.976225  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.978919  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.977310  7228 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:41.981674  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:41.984323  7235 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:41.984894  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.007699  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.985683  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.008507  6590 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:41.976258 14041 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:41.989264  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.989341  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.011579  6593 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:41.989507  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.013901  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:41.960013 13990 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.974363 14062 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:41.996492  6516 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:41.999238  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.000818  7231 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:41.996683  6637 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:42.040866  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44260.7us
I20260709 13:19:41.980621 13887 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.980875 14004 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:42.043303 14004 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.043694 14004 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:42.042848 13614 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.047755 14096 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.047258 13571 consensus_queue.cc:237] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:41.982421 14003 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:42.051621 14003 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:41.992069 13775 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:42.052070 14003 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:42.014758  6589 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:42.053464  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.015564  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.055335  6593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:42.056144  6593 raft_consensus.cc:606] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.028137  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.058111  7234 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:42.028362  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.029752  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:41.995153 13947 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.060747 14096 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:42.033039  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.060752 13947 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.061592  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.062077  6596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:42.062793 14070 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:42.063241 14070 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.063619 14070 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.064417 14076 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:41.983239 13460 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.065315 14086 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.062544  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.067596  6671 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:42.067775 13460 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.068539 13983 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:42.037720  7198 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.762s (SENT) [suppressed 33 similar messages]
I20260709 13:19:41.982564 13837 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:19:42.041563  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.042761 14075 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:42.073475  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.000373 14064 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:42.076828  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.095892  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.068491 13947 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:42.113024  6593 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:42.087237  7234 raft_consensus.cc:2468] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:42.088384  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:42.089316  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.124205 14086 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:42.092061  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 50572.7us
I20260709 13:19:42.125325  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.003273 14022 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:42.101544  6589 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:42.081408  6590 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:19:42.142972  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.144430  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.145135  6590 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:42.146114  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.147403  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.069736 13822 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.040145 13838 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:42.124555  6513 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:42.159860 13998 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:42.072939 14062 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.162458 14062 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:42.162425 13998 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:42.168649 13998 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.054347 13943 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.003708 14017 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:19:42.141520  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.101235 14068 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:42.180549  6562 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.172289  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:42.183826 14074 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:42.184298 14074 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:42.184662 14115 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:19:42.184679 14074 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:42.185045 13983 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:42.186875  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:42.190994  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 107775us
I20260709 13:19:42.102730 14075 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:42.088253 13935 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.088740 14038 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 13:19:42.220245  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.221109  6517 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:42.160842 13947 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:42.193576 13822 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:42.203691 14079 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:42.239246  6596 raft_consensus.cc:606] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.239917  6596 raft_consensus.cc:614] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:42.243937 13935 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.195744 13960 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.247213 13887 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.248412 13960 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.249293 13887 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.251902 13991 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:42.254659  6597 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:42.254917  6593 raft_consensus.cc:1240] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:42.257313  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.258070  6670 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:42.237197 14079 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:42.266059 14079 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.267340 14079 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:42.268581  6670 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:42.219774 14068 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:42.272236  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:42.282202  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 244480us
I20260709 13:19:42.234968 13822 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.220983 14017 consensus_queue.cc:1059] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:19:42.285761 13822 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:42.296936  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.281962 13935 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:42.277815 13991 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:42.279238 13887 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:42.278491 13990 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.310511 13947 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:42.292856 14016 consensus_meta.cc:287] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.004s
W20260709 13:19:42.377403  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 196934us
I20260709 13:19:42.333606 14103 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:42.366513 14112 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:19:42.335843 13935 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:42.336263 13887 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.380599  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.299372 14076 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:19:42.303278 14034 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.336097 13991 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.402326 13614 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:42.392905 14031 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.411409 14076 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:42.415827 14076 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:42.416400  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.416739 14076 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.417925 14076 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.420498 14034 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.422858  6516 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:42.423317  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3168) took 1860 ms. Trace:
I20260709 13:19:42.423678  6595 rpcz_store.cc:276] 0709 13:19:40.563144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.563297 (+   153us) service_pool.cc:224] Handling call
0709 13:19:42.423115 (+1859818us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002f7bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:42.423239 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2766270848}
I20260709 13:19:42.424819  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.426380 13887 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.351387 14075 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.432766 14074 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.434046  6663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1473) took 1394 ms. Trace:
I20260709 13:19:42.434428  6663 rpcz_store.cc:276] 0709 13:19:41.040142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.061955 (+ 21813us) service_pool.cc:224] Handling call
0709 13:19:41.757100 (+695145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.757291 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.757317 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:41.757357 (+    40us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:41.757387 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.757436 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.433879 (+676443us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54001e0ad0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:42.433989 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598039936}
I20260709 13:19:42.435940  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.436620  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.437013  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.437563  6596 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:42.437682  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.441388  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.442059 13932 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.442514  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.443063  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.443835  6513 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.414583 13943 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.411912 13614 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:42.448136 13614 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.450096  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.451660  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:42.455610  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.456372  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:42.459213  6515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2899) took 2599 ms (client timeout 2222 ms). Trace:
W20260709 13:19:42.459604  6515 rpcz_store.cc:269] 0709 13:19:39.860219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.860369 (+   150us) service_pool.cc:224] Handling call
0709 13:19:42.459015 (+2598646us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b5400045954. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:42.459105 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4791150848}
I20260709 13:19:42.448028  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.460717  6519 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:42.463208  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.465269 13991 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.465967  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.466581 14045 raft_consensus.cc:3055] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:42.466995 14045 raft_consensus.cc:740] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:42.467422  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.467792 14045 consensus_queue.cc:260] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.468415  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.468954 14045 raft_consensus.cc:3060] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:42.403129 13960 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.470367 14003 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.471561  6591 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:42.471161 13960 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.473254  6591 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:42.475030 13960 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.369867 14016 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.476387  6591 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:42.477274  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.479458 14011 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.480481  6589 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:42.482117 14011 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:42.483728  7237 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:42.485663 14003 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.487076  6593 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.488559 13880 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:42.490746 14112 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:19:42.492234 14074 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.494505  6668 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:42.443132  6514 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:19:42.497982  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:42.500123 14101 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:19:42.501209  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.503125 13887 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:42.505084 13887 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.452965  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.402463 14004 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.425851  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.505472 13887 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:42.514813  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.535742  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.438381  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.533303  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:42.542461  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.545401  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.543676  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.498812 14031 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.549088  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.550310  6592 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:42.532720 14016 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:42.557559  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.559139  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.560284  6517 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:42.561269  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.561477  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:42.562896  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.559147  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.558395  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.537163 14041 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 17. Current term is 18.
W20260709 13:19:42.567293  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 176844us
I20260709 13:19:42.568933  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.570235  6593 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:42.538430 14071 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:42.571365  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.571874  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.562553  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.575536 14086 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:42.575824  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.581190  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:42.581832  6588 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:42.583070  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.585078  6593 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:42.579171 14086 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:42.586786 14086 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.587327 14004 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.588891 14086 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.590553 14011 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:42.591188  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.591416 14011 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:42.592028 13908 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:42.592185 14011 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:42.592425 13908 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:42.592630 14011 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.592473  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.593191 14011 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.593338 13908 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.594244 14104 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:42.594635 13908 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.571215 13982 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:42.595530 13982 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.568126 14034 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.595971 13982 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.596199 14034 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.510036 13935 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.598009  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.547859 14102 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:42.600685 14122 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:19:42.546540 13838 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:42.556342 14072 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:42.616032 14022 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.617744  6594 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:42.620051 14022 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.622696 13822 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:42.624938 14070 consensus_meta.cc:287] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.002s
I20260709 13:19:42.625455 14070 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.626549  6513 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:42.567879 13914 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:42.628901 14036 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:42.629489 14036 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.630724 14062 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.608686  6417 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 10 to 18, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 18 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.632957 14062 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.634794 14031 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.635099  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.634972  6519 raft_consensus.cc:2368] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23.
W20260709 13:19:42.637553  6415 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 331) took 69410 ms (client timeout 15000 ms). Trace:
I20260709 13:19:42.639068 14073 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:42.640051  6520 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:42.579952 14071 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:42.640832 14071 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:42.643241 13822 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:42.644248  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 50237.8us
W20260709 13:19:42.645143  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 69414.9us
I20260709 13:19:42.657609  6637 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.647125  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.650523  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.658579  7237 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:42.658669  7232 raft_consensus.cc:1240] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:42.658756  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.652514  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.661959  6593 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:19:42.656843  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.669735  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.670871  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.672645  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.672991  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.675809  6562 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:42.677469  6562 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.667292  6417 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:42.680495  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:42.688628  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.692770  6515 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:42.699203  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.700502  6665 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:42.700623  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.654942 13914 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:42.704766  6562 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.703038  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.700778 14086 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:42.638019  6415 rpcz_store.cc:269] 0709 13:18:33.227163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.227420 (+   257us) service_pool.cc:224] Handling call
0709 13:19:42.637478 (+69410058us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:34.863904 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:34.963304 (+ 99400us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:34.963368 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:19:34.963389 (+    21us) tablet.cc:662] Decoding operations
0709 13:19:34.964144 (+   755us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:34.964256 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:34.964287 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:19:34.964862 (+   575us) tablet.cc:701] Row locks acquired
0709 13:19:34.964909 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:19:34.965227 (+   318us) write_op.cc:270] Start()
0709 13:19:34.965394 (+   167us) write_op.cc:276] Timestamp: P: 1783603174965141 usec, L: 0
0709 13:19:34.965445 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:19:35.006649 (+ 41204us) log.cc:844] Serialized 1679 byte log entry
0709 13:19:35.692451 (+685802us) op_driver.cc:464] REPLICATION: finished
0709 13:19:35.990436 (+297985us) write_op.cc:301] APPLY: starting
0709 13:19:36.033107 (+ 42671us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:36.435232 (+402125us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:36.435348 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:36.437752 (+  2404us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:36.438083 (+   331us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:19:36.438143 (+    60us) write_op.cc:312] APPLY: finished
0709 13:19:36.438475 (+   332us) log.cc:844] Serialized 49 byte log entry
0709 13:19:36.443340 (+  4865us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:36.452431 (+  9091us) write_op.cc:454] Released schema lock
0709 13:19:36.469907 (+ 17476us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:36.563709 (+ 93802us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:36.564977 (+  1268us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":637566,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":199507,"apply.run_cpu_time_us":17198,"apply.run_wall_time_us":616166,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20839,"num_ops":4,"prepare.queue_time_us":493054,"prepare.run_cpu_time_us":28291,"prepare.run_wall_time_us":358671,"raft.queue_time_us":160127,"raft.run_cpu_time_us":10077,"raft.run_wall_time_us":108090,"replication_time_us":691125,"thread_start_us":404254,"threads_started":4,"wal-append.queue_time_us":358516,"wal-append.run_cpu_time_us":29949,"wal-append.run_wall_time_us":2186520}]]}
I20260709 13:19:42.706032  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.679088 13947 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:42.708369  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.688694 14101 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:42.716114  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.731945  6519 raft_consensus.cc:2368] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:42.735509  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.719385  7228 raft_consensus.cc:2368] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23.
I20260709 13:19:42.684643 13983 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:42.714247  6662 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:42.738267  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.707823  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.727862  6665 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:42.728838  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.685981 14022 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:42.729811  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.739549  6670 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:42.741887  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:42.734014  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.744776  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.745550  6667 raft_consensus.cc:2368] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:42.749250  6487 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.752066  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.752760  6662 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:42.753229  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.753798  7237 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:42.754544  7234 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:42.754302  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.744596  6417 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 12 to 13, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 13 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:42.756045  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.756855  6670 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:42.758148  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.721496 14072 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:42.721429 14003 raft_consensus.cc:2804] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:42.792053  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.793581  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.794194  6667 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:42.794335  7237 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:42.721843 13571 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:42.784233  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.784849  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:42.791824  6417 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:42.795702  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.797379  6636 leader_election.cc:400] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23.
I20260709 13:19:42.802912  6636 leader_election.cc:403] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:42.804088  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.805531  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.756049 14014 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:19:42.701800 13614 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.761229 14015 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.727397 14004 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 13:19:42.751621 14101 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:42.809118  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.809514  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.810554  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.812011  6668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2742) took 1003 ms. Trace:
I20260709 13:19:42.832551  6668 rpcz_store.cc:276] 0709 13:19:41.808635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.831941 (+ 23306us) service_pool.cc:224] Handling call
0709 13:19:42.811824 (+979883us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b540011e9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:42.811945 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1521429888}
I20260709 13:19:42.814404  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:42.762389 14017 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:42.844362  6588 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:42.846585  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.849009  6417 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 10 to 14, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 14 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.817662  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:42.761420 14034 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.769338 14079 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:42.782727 14072 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.817730  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.807428 14096 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:42.742137 13947 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.830796  7237 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:42.850551  6667 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:19:42.808207 14003 raft_consensus.cc:697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:42.811246 13983 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:42.861029  6417 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 12 to 18, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 18 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.862234  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.864660  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.791766 13627 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:19:42.892237 14003 consensus_queue.cc:237] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.892594  6417 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 12 to 14, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 14 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.886390 14096 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.894426  6417 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.896456 13982 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.897385 14096 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:42.897639  6663 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:42.857527 14036 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.898159 14004 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:42.830874 14015 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.897940  6417 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 14 to 15, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 15 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.899307 13947 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.900034 13982 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:42.878456  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 176631us
W20260709 13:19:42.883687  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.860353 13571 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:42.849566 13996 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:42.885744  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:42.888724  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 94565.7us
I20260709 13:19:42.824467 13991 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:19:42.905027  6517 raft_consensus.cc:2368] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:42.877869 14072 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:42.876879 13990 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:42.897267 13887 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.902161  6519 raft_consensus.cc:2393] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:42.824210 14068 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:19:42.876911  6665 raft_consensus.cc:2468] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:42.850024 14123 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:42.891631  6563 leader_election.cc:400] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23.
I20260709 13:19:42.920150  6563 leader_election.cc:403] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:42.907866  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:42.906368  7237 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:42.921083  6520 raft_consensus.cc:2410] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:19:42.921365  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:42.921486  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 104254us
I20260709 13:19:42.923060  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:42.889559 14064 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.762s	user 0.011s	sys 0.002s
W20260709 13:19:42.916911  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.924108  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.924880  7231 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:42.900535 14015 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:42.890185 14070 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.928665  7237 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.924724 13614 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:42.929888  6596 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 17: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:42.931527  6667 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:19:42.931660 13992 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:42.931833 13887 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:42.932117 13914 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23.
I20260709 13:19:42.904469 13996 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:42.925798  6417 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.934965  6563 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.935119  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.919950 14123 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.926846 13571 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:42.944638  6563 leader_election.cc:400] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:42.945812  6563 leader_election.cc:403] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:19:42.947062  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:42.948338  6637 leader_election.cc:400] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:42.948704  6637 leader_election.cc:403] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:19:42.945339  7198 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:42.951694  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:42.938045 13880 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:19:42.939158 13990 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:19:42.951159 14068 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:19:42.956756 14127 raft_consensus.cc:2804] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:42.957393 14127 raft_consensus.cc:697] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:42.945322 13571 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:42.958425 14127 consensus_queue.cc:237] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.945582 13947 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:42.959184 13571 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:42.959777 14131 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:19:42.943455 13983 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:42.960465 13990 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:42.955199 14036 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.963021  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:42.963883 14015 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:42.964380 14015 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:42.965354 14015 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.967231  6561 leader_election.cc:400] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:42.968173  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.969310 14004 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:42.959671 13947 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.970249 13947 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:42.970981 14076 consensus_meta.cc:287] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.002s
I20260709 13:19:42.971657  7233 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
W20260709 13:19:42.972462 14128 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:19:42.960937 13990 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:42.904271 14101 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:42.974403  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:42.928086 14064 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:42.945847 13614 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:42.983628  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:42.985962 14096 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:42.974421 14136 log.cc:927] Time spent T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:42.987527  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:42.988376 14036 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:42.981387  7237 raft_consensus.cc:2468] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:42.945554 14123 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:42.993860  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:43.004403  7237 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:43.004738  7231 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 21. Current term is 22. Ops: []
W20260709 13:19:42.983757  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75881.6us
I20260709 13:19:43.005731  7237 raft_consensus.cc:1275] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:43.005983  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:42.960472 13571 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:43.010427  6590 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:19:42.957412 14068 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:43.003027  6517 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.018689  6664 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:19:43.019089  6664 pending_rounds.cc:85] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 12
I20260709 13:19:43.019299  6664 pending_rounds.cc:107] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:19:43.019528  6664 raft_consensus.cc:2889] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:43.020831  6664 pending_rounds.cc:107] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:19:43.021179  6664 raft_consensus.cc:2889] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:42.991356 13991 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.013592  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:42.991765 14149 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:43.009846 14146 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:43.022909 14144 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:43.036991  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:43.055943  7200 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:43.055105 14144 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.056723 14076 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:43.057142  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:43.037577 14015 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:43.057883 14076 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:43.029942 14071 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.057439 13990 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.023777 14070 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.015228 13949 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:43.017680 14068 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:43.060163  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:43.061223 14151 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:42.994387 13996 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.057183 14144 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:43.072698  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3169) took 1097 ms. Trace:
I20260709 13:19:43.078740  6517 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:43.078899  6594 rpcz_store.cc:276] 0709 13:19:41.975371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.975920 (+   549us) service_pool.cc:224] Handling call
0709 13:19:43.072497 (+1096577us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b5400210f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:43.072642 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407144832}
I20260709 13:19:43.079933  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.080799  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.081166  6594 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:43.081663  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:43.073217  6593 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:43.076231 13982 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.076233  7237 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:43.085708  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:43.084735  7229 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:43.082696 13990 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:43.090970 14153 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:43.095376  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:43.096623 14045 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:43.105422  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:43.089324 14151 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.067220 14011 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.058887 14146 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.069825 14070 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:43.109814  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.106071  6519 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:43.093130 13996 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:43.117116  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:43.089078 13982 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.121865  7230 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:43.124655  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1482) took 1332 ms. Trace:
I20260709 13:19:43.125065  6664 rpcz_store.cc:276] 0709 13:19:41.792549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.793126 (+   577us) service_pool.cc:224] Handling call
0709 13:19:41.793345 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.021661 (+1228316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.124440 (+102779us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000cba54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:43.124600 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817063168}
I20260709 13:19:43.129405  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:43.133596  7234 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:43.134338  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:43.149855  6668 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:43.141674  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:43.094121 14150 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:19:43.155817  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:43.122699 13627 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:43.102937 14152 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:19:43.179476  6515 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:43.183908  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:43.184638  6517 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:43.155508 14149 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:43.182379  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:43.168202 14017 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.186877 14149 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:43.188727 14073 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:43.189288 14073 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:43.190335  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:43.103305 14068 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:43.142061 14064 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:43.198489  6668 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:43.174835 14071 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.190151 14073 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:43.239722  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 84052us
I20260709 13:19:43.210484 14145 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:43.193208 14015 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 13:19:43.290771  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49214.9us
W20260709 13:19:43.223146 14155 log.cc:927] Time spent T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:19:43.211560 14011 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:43.334061  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 159478us
I20260709 13:19:43.214808 13880 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23.
I20260709 13:19:43.333508  6517 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
W20260709 13:19:43.295274 14156 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:19:43.298978 14145 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.291615 14019 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:43.219579 14045 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.117s
I20260709 13:19:43.366994  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:43.374922 13982 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.361030 14019 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:43.379206  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:43.336790 14160 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:43.352864 14145 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:43.389523  6594 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:43.347695 14159 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:19:43.386471 13627 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:43.409619  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54701.5us
W20260709 13:19:43.408619 14138 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.499s	user 0.002s	sys 0.002s
I20260709 13:19:43.420892  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:43.423857  6591 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:43.423549 14019 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:43.429445  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.382706 13947 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.421936 14146 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:43.451216  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 260938us
I20260709 13:19:43.423439 13627 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.456226  6667 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:43.456169  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:43.466768  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:43.467689  6669 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:43.467820  6515 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:43.393571 14160 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:43.481691 14072 consensus_meta.cc:287] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.003s
I20260709 13:19:43.492461 14072 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.489434 14096 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
W20260709 13:19:43.491859 14086 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.003s
I20260709 13:19:43.575585 14086 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.495678 14160 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.500190  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3180) took 1498 ms. Trace:
I20260709 13:19:43.579000  6592 rpcz_store.cc:276] 0709 13:19:42.002951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.007423 (+  4472us) service_pool.cc:224] Handling call
0709 13:19:43.500014 (+1492591us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54002f6cd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:43.500135 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241890560}
I20260709 13:19:43.580302  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:43.581431  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.468132 14019 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:43.502445  6562 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:43.582317 14019 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:43.582420  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 80050.1us
I20260709 13:19:43.583276 14019 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:43.583695  6562 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:43.584565  6562 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:43.502421  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:43.585807  6593 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:43.586596  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.587244  6593 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:43.588078  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.589134  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:43.589651  6593 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:43.487741 13947 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.466671 14172 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.522775 13460 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:43.529775 14076 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:43.532291  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.534228  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:43.535579 14130 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.939s	user 0.005s	sys 0.008s
I20260709 13:19:43.536103  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:43.536350  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:43.591735  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:43.539825  6666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3244) took 1771 ms. Trace:
I20260709 13:19:43.592298  6589 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.541018  6588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3157) took 1020 ms. Trace:
I20260709 13:19:43.594450  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:43.594206  6588 rpcz_store.cc:276] 0709 13:19:42.520728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.606045 (+ 85317us) service_pool.cc:224] Handling call
0709 13:19:42.606296 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.849390 (+243094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.540727 (+691337us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54003a2354. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:43.540956 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547035008}
I20260709 13:19:43.541991 14022 raft_consensus.cc:993] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:19:43.543476  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:43.545037  6596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 3153) took 1177 ms. Trace:
W20260709 13:19:43.553858 14126 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.959s	user 0.000s	sys 0.002s
I20260709 13:19:43.554416  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:43.563285  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 58581.1us
W20260709 13:19:43.565918  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 55734.7us
I20260709 13:19:43.570283  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:43.598994  6517 raft_consensus.cc:2368] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:43.571836 13914 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:19:43.572404 14123 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.569s	user 0.005s	sys 0.005s
I20260709 13:19:43.566869 13995 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:43.574322  7235 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:43.574585 14151 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:43.537479  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:43.591646  6519 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:43.596187  6516 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:43.596134  6596 rpcz_store.cc:276] 0709 13:19:42.367809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.438147 (+ 70338us) service_pool.cc:224] Handling call
0709 13:19:43.544850 (+1106703us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54003bd8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:43.544979 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892715904}
I20260709 13:19:43.570633  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:43.601322  6593 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:43.656530  7229 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:43.644744  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:43.646669  6671 raft_consensus.cc:1240] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:19:43.592353  6666 rpcz_store.cc:276] 0709 13:19:41.768315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.768427 (+   112us) service_pool.cc:224] Handling call
0709 13:19:42.483129 (+714702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.483307 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:42.647828 (+164521us) log.cc:844] Serialized 14 byte log entry
0709 13:19:42.942011 (+294183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:42.942078 (+    67us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:42.942136 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.942246 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.539593 (+597347us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54001cacd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:43.539748 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":300419,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":50371,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":1597740928,"thread_start_us":457870,"threads_started":3,"wal-append.queue_time_us":259340}
I20260709 13:19:43.645891  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:43.659926  6517 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:43.606094  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 71929.4us
I20260709 13:19:43.590893 13992 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.668514  6593 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:43.673702  6589 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:43.598563 14169 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.593638 14076 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:43.679646  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 78152.6us
I20260709 13:19:43.593479 13571 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:43.597990 13990 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:43.648628 14030 raft_consensus.cc:2764] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:43.691924 14041 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:43.692039 14167 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:43.693320  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:43.679422 13914 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:43.701014 14070 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.702708  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 2925) took 1033 ms. Trace:
W20260709 13:19:43.707481 14125 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.111s	user 0.002s	sys 0.001s
I20260709 13:19:43.707333  6516 raft_consensus.cc:1275] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:19:43.708240 14076 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:43.709568 14144 consensus_meta.cc:287] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.004s
I20260709 13:19:43.725411 13914 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:43.600607 13880 raft_consensus.cc:2804] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:43.718344  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:43.675899 13822 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.598423 14172 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.726332  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:43.650447 13947 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:19:43.655774 14123 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.749079  6516 pending_rounds.cc:85] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 14
I20260709 13:19:43.607455 13998 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.111s
I20260709 13:19:43.792528  6516 pending_rounds.cc:107] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:19:43.793107  6516 raft_consensus.cc:2889] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:43.747396  6520 rpcz_store.cc:276] 0709 13:19:42.669202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.669496 (+   294us) service_pool.cc:224] Handling call
0709 13:19:43.702479 (+1032983us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400044054. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:43.702643 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575927040}
I20260709 13:19:43.680565 14158 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:19:43.667781 13995 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.681784 13992 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:43.778106  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 59850.4us
I20260709 13:19:43.676522 14151 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.798415  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:43.801096  6670 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:43.711964 14145 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.807224  6670 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:43.804857  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 78819.9us
I20260709 13:19:43.772892 13627 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:43.808259 13627 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.775580 14101 raft_consensus.cc:1081] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:19:43.811096  6666 raft_consensus.cc:1240] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:43.708267 14169 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.766203 13990 consensus_queue.cc:1048] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 13:19:43.881577  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:43.866657  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.833642  6518 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:19:43.845039  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:43.886049  6666 raft_consensus.cc:2393] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:43.775933 14144 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.802896 14076 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:43.813313  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 222868us
I20260709 13:19:43.890746  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:43.796823 13880 raft_consensus.cc:697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:43.892875  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1504) took 2051 ms. Trace:
I20260709 13:19:43.893756  7230 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:19:43.782392 14086 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.810633 13627 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:43.895824  7236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 2007) took 1998 ms. Trace:
I20260709 13:19:43.900434  6562 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:43.897217  6521 rpcz_store.cc:276] 0709 13:19:41.841507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.841685 (+   178us) service_pool.cc:224] Handling call
0709 13:19:41.842074 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.842344 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.842383 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:42.249790 (+407407us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b2c000014c8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b797aa11 00007f04b798a372 00007f04b90bca01 00007f04b908143a 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2
0709 13:19:42.250333 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.511830 (+261497us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:42.511906 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.512002 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.706890 (+1194888us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:43.892600 (+185710us) raft_consensus.cc:1685] finished
0709 13:19:43.892632 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.892796 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73422592,"thread_start_us":261014,"threads_started":1,"wal-append.queue_time_us":345415}
I20260709 13:19:43.873190 13949 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.865386 14015 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.806216 14073 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:43.797902 13822 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.908720  7236 rpcz_store.cc:276] 0709 13:19:41.897225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.898719 (+  1494us) service_pool.cc:224] Handling call
0709 13:19:42.256008 (+357289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:42.256415 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.256454 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:42.256989 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.549165 (+292176us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:42.549213 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.549283 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.553193 (+1003910us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:43.711445 (+158252us) raft_consensus.cc:1685] finished
0709 13:19:43.711465 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.895658 (+184193us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54007a3d50. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:43.895772 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15018,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4786,"spinlock_wait_cycles":874784512,"thread_start_us":290209,"threads_started":1,"wal-append.queue_time_us":337735}
I20260709 13:19:43.952459 13995 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:43.925899  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.954362 14123 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.955093 14123 raft_consensus.cc:993] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:19:43.845395 13914 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.955574 14123 raft_consensus.cc:1081] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:19:43.956030 13914 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:43.957990 13992 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:43.958046 14015 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.918932 13996 consensus_queue.cc:1059] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:19:43.961902 13996 raft_consensus.cc:3055] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:43.962257 13996 raft_consensus.cc:740] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:43.963075 13996 consensus_queue.cc:260] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.954236 14180 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:19:43.969507 13998 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:43.970664 14015 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:43.970945  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.971206 13949 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:43.943217 14178 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:19:43.972201  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.944473 14151 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:43.947671 14172 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:43.949216 14169 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.950774 14144 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:43.951133 13880 consensus_queue.cc:237] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:43.973497  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:43.954072 13822 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:43.887240 13571 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:43.971698  6597 raft_consensus.cc:2393] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 16: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:43.975189  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 74760.5us
I20260709 13:19:43.958086 14073 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:43.941571  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2765) took 1229 ms. Trace:
I20260709 13:19:43.999542  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.000662  6589 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:19:43.940037 14145 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:43.999514  6662 rpcz_store.cc:276] 0709 13:19:42.711877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.737965 (+ 26088us) service_pool.cc:224] Handling call
0709 13:19:43.941386 (+1203421us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b540011e4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:43.941504 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31518592}
I20260709 13:19:43.997347  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.977659  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:44.004211  6593 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:44.005745  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:43.905876 14072 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:43.896291 14104 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:19:44.001991 14073 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:44.014659  6597 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:43.901655 14127 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.105s
I20260709 13:19:44.005849  6588 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
W20260709 13:19:44.023087  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 214635us
I20260709 13:19:44.031824 13627 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.034230 13627 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:44.039116 14185 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.097s	user 0.002s	sys 0.000s
I20260709 13:19:44.040581 13571 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.054039  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:44.056042  6662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73"
dest_uuid: "53fee9894731461a821a131be24dbbca"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:53226
I20260709 13:19:44.056811  6662 raft_consensus.cc:606] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:19:44.057283  6662 raft_consensus.cc:614] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:44.061900  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.063050  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 155868us
W20260709 13:19:44.065958  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 444215us
I20260709 13:19:44.070541  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.071293  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:44.071904  6667 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.072880  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.073707  6662 raft_consensus.cc:2410] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:19:44.074096  6668 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:44.059513 14165 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:44.076362  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.063486  6563 leader_election.cc:400] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:44.080667  6563 leader_election.cc:403] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:44.082581  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:44.083422  6521 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 22: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:44.083856  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.084913 14186 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19.
I20260709 13:19:44.056671 14190 consensus_queue.cc:1059] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:19:44.087692  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:44.091498  6667 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:44.085628  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:44.107666  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:44.115403  7228 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:44.116330  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:44.116950 14141 log.cc:927] Time spent T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 1.225s	user 0.007s	sys 0.009s
I20260709 13:19:44.117216 14145 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:44.117863  7228 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:44.088878  7230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:19:44.119029  7230 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:44.119383  7230 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:44.120100  7198 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:44.120858 14068 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.121110 14145 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:44.107934  7232 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:19:44.121896 14145 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:44.122056  7232 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:44.083452 13992 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.122748 14058 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:44.123163 14058 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:44.122967 13992 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.107344 14196 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.123972 14058 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:44.124722 14196 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.125486 14058 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:44.105309 13949 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:44.123169 14145 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.122522  7232 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.115396  7235 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:44.156632 13992 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:44.163866 14196 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:44.169776 14166 raft_consensus.cc:3055] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:44.170255 14166 raft_consensus.cc:740] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:44.171253 14166 consensus_queue.cc:260] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.172451 14166 raft_consensus.cc:3060] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:44.209964  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.217520  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 635643us
I20260709 13:19:44.223232  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.224193  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.225579  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.225740  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.227566  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.240334  6516 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
W20260709 13:19:44.240821  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 155181us
I20260709 13:19:44.228216  6519 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:44.229305  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.249734  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.230187  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "921c423406da4f289884afca51ca198f"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:44.252079  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:44.230324  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:44.251255  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.231572  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.226594  6520 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:44.253490  6593 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:44.234970  6517 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:44.253643  6597 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:44.252410  6589 raft_consensus.cc:606] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Received request to transfer leadership to TS 921c423406da4f289884afca51ca198f
I20260709 13:19:44.230195  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.254544  6589 raft_consensus.cc:614] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:44.255389  6514 raft_consensus.cc:2410] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:19:44.231112  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:44.254515  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.256505  6596 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:19:44.259482  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:44.275307 14070 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.276221  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 3161) took 1321 ms. Trace:
I20260709 13:19:44.275701  7230 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.276583 14070 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:44.276885  6594 rpcz_store.cc:276] 0709 13:19:42.954267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.081510 (+127243us) service_pool.cc:224] Handling call
0709 13:19:44.275748 (+1194238us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54002f62d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:44.275991 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619857664}
I20260709 13:19:44.279547  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1803) took 1121 ms. Trace:
I20260709 13:19:44.283010 14072 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:44.291833  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3193) took 1517 ms. Trace:
I20260709 13:19:44.292379  6591 rpcz_store.cc:276] 0709 13:19:42.774104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.774285 (+   181us) service_pool.cc:224] Handling call
0709 13:19:42.774582 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.425611 (+651029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.291215 (+865604us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b540034fb54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:44.291590 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480948992}
I20260709 13:19:44.301519  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3149) took 2305 ms. Trace:
I20260709 13:19:44.302058  6590 rpcz_store.cc:276] 0709 13:19:41.995761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.162327 (+166566us) service_pool.cc:224] Handling call
0709 13:19:42.162705 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.011291 (+848586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.300762 (+1289471us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b54002fb7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:44.301297 (+   535us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948804352}
I20260709 13:19:44.303128 14167 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:44.304077 14167 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:44.306859  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2756) took 1599 ms. Trace:
I20260709 13:19:44.308020 14167 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:44.309736  7233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46680 (request call id 2026) took 1580 ms (client timeout 1984 ms). Trace:
W20260709 13:19:44.310264  7233 rpcz_store.cc:269] 0709 13:19:42.729429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.729556 (+   127us) service_pool.cc:224] Handling call
0709 13:19:44.309275 (+1579719us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b540027d8d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:44.309499 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532685568}
I20260709 13:19:44.307366  6671 rpcz_store.cc:276] 0709 13:19:42.706845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.706958 (+   113us) service_pool.cc:224] Handling call
0709 13:19:42.745167 (+ 38209us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.658931 (+913764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.306211 (+647280us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54001cb1d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:44.306652 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967516032}
I20260709 13:19:44.307906  6669 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:44.280180  7234 rpcz_store.cc:276] 0709 13:19:43.158298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.164615 (+  6317us) service_pool.cc:224] Handling call
0709 13:19:43.165487 (+   872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.166331 (+   844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.166391 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.176246 (+  9855us) log.cc:844] Serialized 26 byte log entry
0709 13:19:43.453307 (+277061us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:43.460013 (+  6706us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.460236 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.460333 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.164285 (+703952us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b18002a7840. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:44.168795 (+  4510us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b18002a7840. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:44.278857 (+110062us) raft_consensus.cc:1685] finished
0709 13:19:44.278914 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.279231 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4092,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":2678,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":4620,"spinlock_wait_cycles":12696192,"thread_start_us":247657,"threads_started":1,"wal-append.queue_time_us":279062}
I20260709 13:19:44.320518 14196 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:44.321053  6588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:44.322875  6588 raft_consensus.cc:606] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:44.323406  6588 raft_consensus.cc:614] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:44.314476  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:44.325344  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:19:44.328966  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 54857.4us
I20260709 13:19:44.324828 13914 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:19:44.331130 14180 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:44.298285 14075 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:44.347587 14017 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:44.348119 14017 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:44.349488 14017 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.355762  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:44.356494  7230 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:44.360178  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:44.369263 14070 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:44.372012  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:44.372403  6668 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:19:44.374665 14196 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:44.375227 14196 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:44.375694 14196 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:44.380571 14039 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:44.381492 14039 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:44.382726 14075 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:44.362334 14186 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:19:44.412189  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.416752  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:44.417260 13908 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:44.417539  6668 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.418095 13908 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.420755 13908 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:44.427178 14180 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:44.427838 14180 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:44.428349 14180 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.428500  7234 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:44.433172 13848 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:44.433805 13848 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:44.435850  6596 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:44.442039 13880 raft_consensus.cc:2764] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:19:44.442987 14022 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:44.443636 14022 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.446280 14022 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:44.433243  6514 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
W20260709 13:19:44.506525  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 74136.2us
I20260709 13:19:44.507222  6668 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:44.452545 14039 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.508301 14039 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:44.513777  6662 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:19:44.453295 14017 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:44.516929  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 102904us
I20260709 13:19:44.461359 14104 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:44.518934 14104 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:44.520514 14104 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.522130 14104 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:44.462098 14019 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:44.523494 14073 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.441571 13848 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:44.474887  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.476022  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.482939  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76570.8us
I20260709 13:19:44.483768  7234 raft_consensus.cc:1240] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:44.487768  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:44.489315  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:44.489322 14030 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.497180 13996 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:44.499656  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:44.501068  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.501137  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:44.501900  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:44.501996  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:44.502842  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.447199  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:44.504221  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:44.528806  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:44.533466  6669 raft_consensus.cc:2393] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:44.542954 14030 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.545754 14030 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:44.570384  7229 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:44.571681  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.580890 13992 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:44.582540 14166 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:19:44.585132 14167 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.610538  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:44.613332  7233 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:44.613927  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.614714  6669 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:44.617022 14167 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:44.618805 14182 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.619362 14182 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:44.620759  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 131724us
W20260709 13:19:44.623272  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 75162.8us
W20260709 13:19:44.625314  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 121279us
I20260709 13:19:44.626991 14145 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.627516  7237 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:44.628162  7237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1802) took 1471 ms (client timeout 1798 ms). Trace:
W20260709 13:19:44.628569  7237 rpcz_store.cc:269] 0709 13:19:43.156504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.156930 (+   426us) service_pool.cc:224] Handling call
0709 13:19:44.386822 (+1229892us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b58001c2998. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:44.627100 (+240278us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b540046c4a8. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee5e924 00007f04be047227 00007f04be046ab8 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:44.628092 (+   992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11012736}
I20260709 13:19:44.629499 14152 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:44.631655 13848 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:44.635543  6592 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:44.637558  6592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55992 (request call id 3165) took 1212 ms. Trace:
I20260709 13:19:44.637972  6668 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:44.638746  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:44.626320  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:44.614317  7232 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.627614 14145 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.637939  6592 rpcz_store.cc:276] 0709 13:19:43.424806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.581193 (+156387us) service_pool.cc:224] Handling call
0709 13:19:44.637372 (+1056179us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002fdfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:44.637503 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422524800}
I20260709 13:19:44.644148  6419 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:44.645067 14145 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:44.645253 14196 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.650919 14072 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:44.605912 14070 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:44.649832  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:44.639288 14186 consensus_queue.cc:1048] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:19:44.663715  7232 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:44.662773  6668 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:19:44.670898 14123 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:19:44.673454 14182 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:44.675305 14030 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:44.676116 14030 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:44.677574  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.664865 14019 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:44.679535 14019 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:44.680056 14019 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:44.691136 14214 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
W20260709 13:19:44.703912  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46040.2us
I20260709 13:19:44.659159  6419 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:44.713953  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 56610.2us
W20260709 13:19:44.731570  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 138169us
I20260709 13:19:44.716811 14073 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:44.739007  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 112791us
I20260709 13:19:44.741591  6671 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
W20260709 13:19:44.746325  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57105.3us
W20260709 13:19:44.749776  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.762969  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.770639  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:44.771587  6671 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:44.773919  6419 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 19, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 19 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.784559  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:44.765393 14030 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:44.778553 14167 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:44.798010  6670 raft_consensus.cc:2393] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:19:44.812269  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.802126  6419 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 6 to 17. New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.804982 14073 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:44.810700 14153 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:44.820371 14058 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.011s
I20260709 13:19:44.816859 14196 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:44.847678  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:44.831015 14039 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:44.850481  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 93218.4us
I20260709 13:19:44.821648 14079 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:19:44.854795  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:44.852833  6419 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 20, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:44.851205 13996 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:44.871184  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:44.876359  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:44.879976  6514 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:44.880318  6669 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:44.865868 13996 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.882185 13996 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:44.889451  6521 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:19:44.890098  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:44.891991 14220 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:19:44.894054 14223 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:19:44.895987 14224 log.cc:927] Time spent T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:19:44.897199 14222 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:44.900744 14180 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:44.901753  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:44.902946 14180 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:44.903641  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:44.904383  6513 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:44.906735  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.849586 14153 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:44.878499 14221 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:19:44.906775  6419 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 20, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:44.912912 13914 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:44.913564  6419 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 13 to 16, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 16 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
W20260709 13:19:44.914496  6417 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 253) took 40200 ms (client timeout 15000 ms). Trace:
I20260709 13:19:44.918372  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:44.919842  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.923939  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.924492  6596 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:44.931875  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:44.932135 13572 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:19:44.932839  6592 raft_consensus.cc:2368] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:44.934697  6521 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.935067  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.936565  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.937651  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.944590  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:44.952191  7237 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:44.919838 14180 consensus_queue.cc:1059] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 13:19:44.956179  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.951282  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:44.937702 14039 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:44.940305 14153 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:44.967322  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:44.918778 14167 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:19:44.981683  7230 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:44.965075  6419 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 15 to 18, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 18 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.987937 14070 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.988601 14070 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.989200  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:44.989905  6596 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:44.990262  6596 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:44.990789  6592 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:44.991228  6592 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:44.992190  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 128997us
I20260709 13:19:44.966008 14039 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:44.999754  6592 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:44.967334 13880 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:45.015743  6637 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:44.993206 14072 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:45.017975  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:45.011452  6419 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 18, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 18 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:45.005187  6521 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:45.020889  6595 raft_consensus.cc:1275] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:45.018787 13848 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.044385  6592 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.982208 14019 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:45.051153  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 127755us
I20260709 13:19:45.048369 14212 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.050514 14153 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:45.053251 14068 raft_consensus.cc:3055] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:45.053923 14068 raft_consensus.cc:740] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:45.053543 14212 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.055079 14068 consensus_queue.cc:260] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.057034 14068 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:45.049142 14180 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.048491  6419 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 18 to 19, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 19 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
W20260709 13:19:45.058410  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.063491  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 57474.7us
I20260709 13:19:45.057818 14180 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.067054 14070 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:44.914994  6417 rpcz_store.cc:269] 0709 13:19:04.656714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.678163 (+ 21449us) service_pool.cc:224] Handling call
0709 13:19:44.851332 (+40173169us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:43.362880 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:43.464658 (+101778us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:43.468392 (+  3734us) write_op.cc:435] Acquired schema lock
0709 13:19:43.469820 (+  1428us) tablet.cc:662] Decoding operations
0709 13:19:43.475417 (+  5597us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:43.475485 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:43.475514 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:19:43.476352 (+   838us) tablet.cc:701] Row locks acquired
0709 13:19:43.476371 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:19:43.476522 (+   151us) write_op.cc:270] Start()
0709 13:19:43.476642 (+   120us) write_op.cc:276] Timestamp: P: 1783603183476467 usec, L: 0
0709 13:19:43.476666 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:19:43.477319 (+   653us) log.cc:844] Serialized 3954 byte log entry
0709 13:19:43.710109 (+232790us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b
0709 13:19:44.115055 (+404946us) op_driver.cc:464] REPLICATION: finished
0709 13:19:44.166241 (+ 51186us) write_op.cc:301] APPLY: starting
0709 13:19:44.167101 (+   860us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:44.174213 (+  7112us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:44.174236 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:44.178610 (+  4374us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:44.178882 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:19:44.178926 (+    44us) write_op.cc:312] APPLY: finished
0709 13:19:44.179270 (+   344us) log.cc:844] Serialized 97 byte log entry
0709 13:19:44.179472 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:44.179600 (+   128us) write_op.cc:454] Released schema lock
0709 13:19:44.179671 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:44.179989 (+   318us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:44.180006 (+    17us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3371331,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":47391,"apply.run_cpu_time_us":12115,"apply.run_wall_time_us":15200,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45907,"num_ops":10,"prepare.queue_time_us":308299,"prepare.run_cpu_time_us":16297,"prepare.run_wall_time_us":546338,"raft.queue_time_us":74584,"raft.run_cpu_time_us":9443,"raft.run_wall_time_us":35631,"replication_time_us":638261,"spinlock_wait_cycles":48283520,"thread_start_us":663380,"threads_started":4,"wal-append.queue_time_us":464272}]]}
I20260709 13:19:45.066203  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:45.066283 14180 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.071921  6419 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 20, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 20 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:19:45.084211  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 73287.5us
I20260709 13:19:45.057528 14212 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.089350  6419 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 15, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 15 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:45.091042  6419 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:45.093897  6419 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:45.095698  6419 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 10 to 11, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 11 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:45.097806 13996 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.103565  7232 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:45.105508 13996 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.101619  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.101442  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.063122 14072 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.114347  7235 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:45.112650  6664 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:45.050925 13880 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.073748 13848 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:45.125490  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.128561  6513 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:45.137125  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:45.146420 14064 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 13:19:45.150656 14014 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
W20260709 13:19:45.151293  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 85258.8us
I20260709 13:19:45.151476  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.150681 14187 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:45.152702 14019 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.150740 14096 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:45.155120 13880 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.159400 14072 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:45.162230 13848 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.153038 14187 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:45.166973  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:45.164461  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:45.125299 14070 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:19:45.175431 14019 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.166163  6592 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
W20260709 13:19:45.175957 14175 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:45.169961 14039 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.217047  6666 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.217633 14187 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:45.229526  6419 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 17, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:45.140269 14182 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:45.227500  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:45.235590  7229 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:45.237460  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 83825.5us
I20260709 13:19:45.129966 14153 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:45.140005 14196 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:45.240478  6595 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:45.223606 14166 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.260736  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:45.244256  6419 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:45.264420  6594 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:45.266986  6590 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:45.233055 14019 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.277045  6592 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:19:45.280236  6595 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:19:45.280740  6595 pending_rounds.cc:85] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 12
I20260709 13:19:45.252882 14072 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:45.281008  6595 pending_rounds.cc:107] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:19:45.288066  6595 raft_consensus.cc:2889] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:45.288959  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:45.253796 14196 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.295475  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:45.296643  7232 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:45.280233  6419 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 13 to 16, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 16 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:19:45.294773  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 130482us
I20260709 13:19:45.271544 14182 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.284338 13908 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:45.307144  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53417.6us
I20260709 13:19:45.308203  7200 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:45.309108 14104 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:45.320209 14167 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:45.320971 14167 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.321650  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:45.322392  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:45.323163  6589 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:45.324105  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:45.325335  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:45.325973  6597 raft_consensus.cc:2376] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 921c423406da4f289884afca51ca198f in term 23. Re-sending same reply.
I20260709 13:19:45.328349  6515 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:45.328239  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:45.335395  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:45.336371  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:45.336584  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.337452  6517 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:19:45.338706  6516 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:45.339164 14182 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.339321  6514 raft_consensus.cc:1275] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:45.340445 14039 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:45.280318 14166 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.341511  6520 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:45.343544 14058 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:45.344518  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.345078  6513 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:45.346099  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.346994  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:45.349498  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:45.338315  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:45.353921  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.311518  6419 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 12 to 17, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:45.362860  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:45.363831 13914 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:45.372471  6596 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.373621  6596 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:45.375221 14017 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:45.375681 14017 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:45.376592 14017 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.378105 14017 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:45.394078 14022 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:45.395143 14022 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:19:45.395386  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.395470 14022 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:45.396507 14022 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.397198 14180 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:45.397662 14022 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:45.402968 14235 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:45.404138 14068 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:45.404501 14068 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:45.405479 14068 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.407096  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:45.408237 14123 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:45.408718 14123 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:45.409366 14167 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.409664 14123 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.378925 14182 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:45.414088 13880 consensus_queue.cc:1048] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:45.415030 14182 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:45.416121 14182 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.420387  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:45.421504 14102 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:45.421976  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:45.423368  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:45.429347 14145 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:45.429819 14145 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:45.431135 14166 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.430797 14145 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.431756  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:45.432719 14102 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:45.435372  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:45.431879 14145 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:45.438556  6596 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
W20260709 13:19:45.466681  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 116696us
I20260709 13:19:45.469957  6521 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:19:45.480897  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 171152us
I20260709 13:19:45.467347  6666 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
W20260709 13:19:45.485041  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.480346 14096 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:19:45.488462 14243 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:45.488736  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:45.489693  6668 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:45.469527 14072 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:45.441526 13914 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:45.500336  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:45.496253 13848 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 13:19:45.507146  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:45.536543  6588 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:45.468685 13880 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:45.508687 14187 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.510828 14150 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:45.496616 14167 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:45.525146  6419 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 13, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 13 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:45.528179  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.517436 14180 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.485194 14166 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:45.542280  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 261395us
I20260709 13:19:45.544353  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.469797 14070 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:19:45.489435 14075 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.516798 14243 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:45.548677  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 210488us
I20260709 13:19:45.548722  6669 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:45.516820 14238 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.552951  6419 catalog_manager.cc:5697] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 9 to 15, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 15 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:19:45.535400 14186 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.547303 14068 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.541052 14235 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.553574 14196 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.589242  6486 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:45.594723 14186 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.603667 14075 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.605365 14186 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.606163 14186 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.607287 14186 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.609910 14186 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.618700 14238 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.621146 14238 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.622011  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.623378 14180 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.625558 14180 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.557128 13880 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.627296 13880 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.629427 13880 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.630101  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:45.635758 14187 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.636756 14243 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.638980 14019 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.560858 14150 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.650545 14150 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:45.655593  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 155375us
I20260709 13:19:45.595733 13460 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:45.659929  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 66755us
I20260709 13:19:45.606217 14075 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.658375  6419 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 18, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 18 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:45.668407 14212 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.668972 14212 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:45.672469  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 153077us
W20260709 13:19:45.673327  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.670817  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 169346us
I20260709 13:19:45.675176  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:45.676656  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:45.687244  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 98105.6us
I20260709 13:19:45.662575 13460 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.683657  7229 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:45.665984 14039 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.591284 14166 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.695216  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:45.695399  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:45.665329 14235 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.700923  7233 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:45.706151  6669 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:45.694367  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:45.710163 14167 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:45.712265 14167 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:45.697854 13996 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:45.700404 14039 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.702723  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:45.691767 14017 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.708453  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:45.696789 13460 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:45.720149  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.696355 14187 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:45.741308 14257 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.746191 14235 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.721474  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:45.748684 14017 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.750213 14255 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.752168  7236 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:45.745759  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:45.710326 14245 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.763831 14073 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:45.778968 14073 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.779398 14073 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:45.714771 14079 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:19:45.770232 14039 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:45.770744  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 95640.2us
I20260709 13:19:45.785543  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:45.770937 14237 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:19:45.771610 14167 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.789238 14167 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.785697 13460 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.791751 14167 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.772275 14245 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.792908 13460 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:45.793663 13460 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:45.794353 13460 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:45.794905 14245 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.773264 14166 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.795322 13460 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:45.773836  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79544us
I20260709 13:19:45.799059  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:45.799891 13848 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:45.800297 13848 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.800573  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:45.774367 14235 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:45.801211 14235 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.801569 14079 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.801764 14235 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:45.803249  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:45.773928 14196 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:45.802165 14079 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.804816 14196 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:45.719156 14212 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.806139 14196 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.807575 14196 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.808073 14079 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:45.775553  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.776270  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:45.775846 14123 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.800702 13848 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:45.810490  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 89230.1us
I20260709 13:19:45.776964 14239 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:45.813797  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:45.836625  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:45.836354  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 338638us
W20260709 13:19:45.839618  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54179.5us
I20260709 13:19:45.762812 14187 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.864032  6636 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:45.865602 14096 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:45.866182 14096 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.866642 14096 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:45.870756  6486 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:45.860226 14079 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.875932 14079 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.845953 14257 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.879642 14167 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:45.880210 14167 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:45.881177 14167 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:45.889597  6592 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:45.838845 14237 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:45.905028  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64663.1us
I20260709 13:19:45.905815  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:45.841938 14212 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:45.878448 14079 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:45.845500 14039 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.939543 14039 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.846032 14030 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.906662  6637 leader_election.cc:400] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17.
I20260709 13:19:45.938464  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:45.856434 14187 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:45.821168 14255 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.950505  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.909073 14237 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.949426  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.953526  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:45.954543  6666 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:45.952451  7198 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:45.937017 14212 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:45.961760  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 56015.2us
I20260709 13:19:45.944780 13996 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:45.856856 14123 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.941326 14239 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.946897 14075 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:45.955013 13914 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.947515 14079 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.968016  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:45.954412 14237 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.979192  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.980345  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.957559 14058 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:45.964340 14150 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:45.964782 14212 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:45.984387  6596 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:45.985355 13914 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:45.965344 14145 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:46.000838  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.011941 13995 consensus_queue.cc:1048] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:19:46.013536 14058 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.015676 14257 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:46.023694  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 187198us
I20260709 13:19:46.015775 14058 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.025817  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.029848  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.035614  6596 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:45.971985 14030 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.042441  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.982210 14075 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.008309 14150 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:46.046914  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:45.992990 14079 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:46.060233  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75641.7us
I20260709 13:19:45.983959 14239 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:46.074345  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.008035 14145 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.084537  6668 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:46.074239  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:46.104640  6667 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:19:46.060242 14039 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.105070  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:46.106683  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.107666  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.107995  6666 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:46.039784 14073 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.105067  6667 pending_rounds.cc:85] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 15
I20260709 13:19:46.112893  6667 pending_rounds.cc:107] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:19:46.113188  6667 raft_consensus.cc:2889] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:46.114163  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.116191  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:46.116528 14268 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:46.116204  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.119030  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.120177  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.121289  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.122047  6666 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:46.122725  6518 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:46.123066  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.123286  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.123955  6670 raft_consensus.cc:2410] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:19:46.124110 14260 log.cc:927] Time spent T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:46.124727  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.122776  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.123376  6663 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:46.081346 13996 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:46.081277 13995 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:46.111822  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:46.129462  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.095412 14248 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.217092  6662 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:46.130033 13908 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:46.129696 14186 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.191756  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:46.195894 14262 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:19:46.199627  6667 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
W20260709 13:19:46.201257  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 196461us
I20260709 13:19:46.201350 14017 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.183050  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.194248  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.240567  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.126055 14180 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.132746  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.127135  7229 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:46.191159  6671 raft_consensus.cc:2368] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22.
I20260709 13:19:46.203351  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.105225 14235 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.244791  6590 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:46.242960  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.243220  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.171379  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.253118  7237 raft_consensus.cc:2393] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:46.244684  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.254024  6670 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:19:46.244840  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.249069  6663 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:46.245100  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:46.241274  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:46.237659 14039 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:46.259783  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.248390  7233 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
W20260709 13:19:46.261700  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 261298us
I20260709 13:19:46.273829  6515 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:46.273438  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:46.280702  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.283120  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.283826  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.284528  7230 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:46.280184  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:46.183473 13996 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.282559  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.240746 14248 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.207724 13995 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.289616  7232 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:46.290793  6665 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:46.246030 14186 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.258628 14268 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:46.284198 13914 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.203896 14150 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:46.313915  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.262344 14255 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.349121  6521 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:46.350641  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.273187 14180 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.347635 14019 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:46.365418  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.366187  6590 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:46.313263 14123 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.281970 14039 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.351552 13914 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:46.375005  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.347575 14017 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:46.293802 14096 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.381160  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:46.384516  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.371652 14019 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:46.397855  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.402304  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.402923  7232 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:46.381124 14145 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:46.378919 14017 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:46.381174 14075 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.410768  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.411834  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.388641 14079 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.387737 13908 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:46.416334  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.395234 14030 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.397949 14073 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:46.382189 13914 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.435767  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:46.439144  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.447284 14235 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.448414 14096 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.449952 14039 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:46.450112 14180 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.450976 14180 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.451665 14180 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.420042 14017 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:46.430893  6665 raft_consensus.cc:2393] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 16: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:46.472494  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.486433 13914 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:46.446416 14196 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.004s
W20260709 13:19:46.496155  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.495553 13908 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.471992 13996 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:46.512174  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101457us
I20260709 13:19:46.513257  6561 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:46.460263 14186 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.455406 14248 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.454598 14180 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.461045 13995 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.552282  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.553289  6670 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
W20260709 13:19:46.564564  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.572494  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.575438  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.430003 14268 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:46.578706  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 298573us
W20260709 13:19:46.567317 14266 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:19:46.564950 14243 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.588382  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.531934 14196 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:46.593595  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.567277 14270 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:46.577993 14265 log.cc:927] Time spent T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 13:19:46.594367  6664 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:46.567270 14273 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:46.611379  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 98280.2us
I20260709 13:19:46.612818  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:46.614182  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:46.615803  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.578006 14279 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:46.577982 14282 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:46.610535 14166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.023s	sys 0.041s
I20260709 13:19:46.564675 14019 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.612304 14273 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.635180  6486 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:46.629012 14212 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.001s
W20260709 13:19:46.629196 14166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.023s	sys 0.042s
I20260709 13:19:46.641242  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:46.631285 14279 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:46.666090  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79773.3us
I20260709 13:19:46.666226  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:46.667454  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.671835  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.672698  6513 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:46.629688 14280 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:46.607170 14243 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.639969 14196 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:46.668076  6519 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:46.636705 14273 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:46.667207  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:46.679219 14079 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:46.679513  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.679729 14022 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:46.680231  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.707145  6513 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:46.680913  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:46.680241 14277 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:19:46.680253  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.676136  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:46.682395  6665 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:46.685369  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:46.695420 14293 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:46.695703 14281 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:46.632690 14276 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.077s
W20260709 13:19:46.716140  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49147.8us
I20260709 13:19:46.674527 14150 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:46.630398 14282 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.700532 14279 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.665660 14186 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.723909  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.616451 14270 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.727066  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:46.728811  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 87679.8us
I20260709 13:19:46.736716  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:46.735875  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 59822.5us
I20260709 13:19:46.711127 14079 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:46.736047  6594 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:46.739322  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:46.648044 14212 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.753391  6516 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:46.744009 14196 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.751281  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:46.770442  6519 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:46.772163 14196 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.726294 14022 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:46.773958 14022 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.776885  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:46.777451  6521 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:19:46.778308  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:46.768375  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.759907  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.727763 14167 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:46.716202 14145 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:46.743451  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.785840  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:46.780810  6664 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:46.789738  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:46.761783 13848 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.906s	user 0.006s	sys 0.003s
I20260709 13:19:46.790886 13848 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.791862 14243 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:46.793234  6593 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:19:46.768882 14237 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.776s	user 0.009s	sys 0.001s
I20260709 13:19:46.794957 14237 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.795718  6671 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:46.747655 14286 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:46.797246 14286 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.797640 14286 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:46.740212 14079 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:46.747884 14272 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:46.803005 14272 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.803472 14272 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:46.756811 14282 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.805924  6592 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:46.786876 14167 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:46.788728 14187 consensus_meta.cc:287] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.831s	user 0.012s	sys 0.001s
I20260709 13:19:46.806638 14167 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:46.806819 14187 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.809046 14187 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:46.809283 14283 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.287s	user 0.002s	sys 0.000s
W20260709 13:19:46.765197 14295 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.061s	user 0.000s	sys 0.002s
I20260709 13:19:46.810974  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.725649 14281 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.813166 14281 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:46.737464 14186 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.815246 14186 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.815804  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.816672 14212 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.805311 14150 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:46.818918 14150 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.821303 14150 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.821622 14017 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.759112 14270 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.810420 14284 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:46.823554 14284 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:46.823992 14284 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.826061  6518 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:46.817853  6519 raft_consensus.cc:2468] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:19:46.828025  6593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3191) took 1013 ms. Trace:
W20260709 13:19:46.828280  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 48262us
W20260709 13:19:46.829583  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.832284  6521 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:46.836571  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.833523  6562 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:46.828486  6593 rpcz_store.cc:276] 0709 13:19:45.814963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.815120 (+   157us) service_pool.cc:224] Handling call
0709 13:19:45.842401 (+ 27281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.793629 (+951228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.827732 (+ 34103us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54002fe9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:46.827969 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2186077952}
I20260709 13:19:46.839282  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:46.844236  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 54217us
I20260709 13:19:46.851220  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:46.848204  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:46.788991 14145 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:46.866546  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.873517  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:46.875592  6589 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:46.886421  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 147218us
I20260709 13:19:46.768401 14239 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.887650  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:46.884212  6637 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:46.888257  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 54857.1us
I20260709 13:19:46.827898 14278 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:19:46.891887  6637 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:46.895152  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47181.7us
I20260709 13:19:46.897416  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.898280  7230 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:46.903614  7233 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:46.905141 14297 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:46.905624 14297 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.905972 14297 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:46.907179  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:46.907923  7237 raft_consensus.cc:3060] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:46.909353 13848 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.911027  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:46.863276 14182 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:19:46.883489 14096 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.915249 14096 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.883865 14276 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:46.917620 14096 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.917526 14276 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.919934 14276 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.920150 14285 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.920710 14285 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.900107 14257 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:46.921756 14017 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.921952 14257 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.922413 14257 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:46.923153 14237 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.923452  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:46.927145  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.927806  6514 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 26.
I20260709 13:19:46.928371  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:46.928961  6662 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:46.928869  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:46.870239 14186 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.933816 14186 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.935599 14285 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:46.936017 14079 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.937177  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.876854 14145 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:46.940303  6589 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:46.937990  6665 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:19:46.901844  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:46.914721 13914 raft_consensus.cc:2804] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:46.949386  7237 raft_consensus.cc:2468] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:46.949635 13914 raft_consensus.cc:697] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:46.950151  7235 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:46.950742 13914 consensus_queue.cc:237] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:46.956055  6667 raft_consensus.cc:2393] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:46.957149  6515 raft_consensus.cc:1240] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:46.959125 14079 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:46.960162  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:46.971016  6514 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:19:46.937247 14268 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.980378 14268 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.899821 14255 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:46.983026 14255 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.983577 14255 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:46.989037 14278 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:46.989562 14273 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:46.989734 14278 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:46.991021 14299 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:19:46.994000  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:46.996096  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 108906us
I20260709 13:19:47.012449  6595 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:47.013131  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:46.996843 13848 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:46.964687 14186 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.011935  6662 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:47.016525  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.019737  6521 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:47.021482  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.033147  6520 raft_consensus.cc:2368] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23.
I20260709 13:19:47.012276 14235 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.040686 14237 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:47.043831 14297 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.045106  6522 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:47.045588 14288 raft_consensus.cc:2804] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:47.046188 14288 raft_consensus.cc:697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:47.047183 14288 consensus_queue.cc:237] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.049226 14254 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:19:47.022778 14243 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:47.052304  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:47.056588  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 43556.5us
I20260709 13:19:47.055583  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:47.057233  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 203976us
I20260709 13:19:47.058442  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.059616  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.060815  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:47.058043  6487 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.132:36575 timed out after 1.788s (SENT) [suppressed 12 similar messages]
I20260709 13:19:47.064838  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.065467  6594 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:47.065279  6595 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 26.
I20260709 13:19:47.064765  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.022491 14150 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 13:19:47.068284  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.075462 14296 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:47.076026 14296 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.062569 14167 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.079869 13848 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:47.059474 14285 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:47.080572 14167 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.080929 14285 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:47.081897 14285 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.085691 14278 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.067128 14254 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:47.093328  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.094312  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.061581 14248 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:47.095359 14248 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:47.095741 14286 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.096246 14248 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.096726  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:47.097797 14286 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.056510 14243 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:47.098490 14019 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:47.098335  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.101145 14273 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.061904 14239 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:47.053968 14235 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.059932 14079 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:19:47.107678  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.108381  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.120456  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.106511 14243 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:47.128682  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.053522 14257 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.070201 14022 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:47.136804  6487 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.072979 14268 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:47.074447 14237 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:47.133615  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.143283  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.078473 14296 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.151292  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.144446  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.152388  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.153281 14186 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:47.153683 14186 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:47.154052 14186 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:47.154734  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.160729  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.163223 14030 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:19:47.164084 14257 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.164901  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.166651 14186 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.152413  6670 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:47.169512  6595 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:47.140597  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.140480  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.141206  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.136686  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:47.141332  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.171053  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:47.172483  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.103654 14297 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.171203  7229 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:47.174644  6561 leader_election.cc:400] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22.
I20260709 13:19:47.176321  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.131672 14254 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:47.136982 14150 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.195425 14150 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.196859  7235 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:19:47.197850 14150 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.194000 14255 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.200667 14182 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:47.202350 14073 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:47.203472  6597 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:47.204790  6597 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:47.180570 14278 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:19:47.206449 14255 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.183873  6589 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:47.185189  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.185703  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.212455  7234 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.212677  6588 raft_consensus.cc:2393] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:47.215626  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:47.216933 14237 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:47.185917  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.217459 14237 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.218067  7228 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.218323 14237 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:47.220134  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.186436  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.186512  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.221748  6591 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:19:47.189208 14291 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:47.180343 14180 consensus_queue.cc:1059] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:19:47.223047 14180 raft_consensus.cc:3055] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:47.223446 14180 raft_consensus.cc:740] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:47.224288 14180 consensus_queue.cc:260] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.225517 14180 raft_consensus.cc:3060] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:47.177870 14187 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.226873 14187 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.136210 13996 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:47.228951 14187 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.136698 13848 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:47.178099 14281 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:47.229879 14281 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.230288 14281 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.230450  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.230839  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.231282  7229 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:47.175004 14270 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:47.233187 14270 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:47.233202  6592 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:47.233685 14270 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.234866  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.217456 14235 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:47.237002  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 85784.5us
I20260709 13:19:47.238452  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.240206  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.241935  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.245296  6595 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:47.246023  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.248584  6594 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:47.250105  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.253451  6520 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:47.173626 14286 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:47.253865  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 70054.5us
I20260709 13:19:47.258065  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.255259  6592 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:47.267551  6594 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:47.271250  7229 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:47.272332  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.273468  6596 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:47.276938  6562 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:47.279938  6597 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:47.135528 14272 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.255671 14058 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:47.294781  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.137841 14145 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.279412  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.298995  6518 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:47.292423  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.300483  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.253130 14294 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:47.177781 14257 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:19:47.302644  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.303426  6591 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:47.261363 14079 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:47.306949  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.307888  6594 raft_consensus.cc:2368] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23.
I20260709 13:19:47.302837  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.280751 14096 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.228968 13996 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.291249 14235 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.324942  7231 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 18: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:47.317438  7232 raft_consensus.cc:2368] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:47.320437  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:47.313030  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:47.298380 14255 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:47.320437  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.295107 14058 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.333536  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.334699  6516 raft_consensus.cc:1240] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:19:47.325695  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.335841  6520 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:47.262894 14073 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:19:47.336203  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.337042  6594 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:47.335834  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.332401  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:47.339897  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.343041  6589 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 26: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:47.344574  6670 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:47.345661 14245 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:47.346731 14245 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:47.347105 14245 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:47.324620 13848 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:19:47.347913 14245 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.349117 14186 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.350323  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.326702 14239 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.350818 14212 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:47.324939 14096 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.351099 14239 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:47.352610  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 85837us
I20260709 13:19:47.342290  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:47.351339 13996 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.359844  7234 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:47.371129  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.351231 14212 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:47.336575 14058 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:47.359776  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.373843  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:47.378305  6596 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
W20260709 13:19:47.340710 14284 consensus_meta.cc:287] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.003s
I20260709 13:19:47.374670  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:47.372470  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:47.375526 14245 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:47.385429 14311 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:19:47.388376 14237 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.389695  6588 raft_consensus.cc:2368] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 18. Current term is 19.
I20260709 13:19:47.391147 14096 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.391745 14212 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.393101 14212 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.395972 14272 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.397235 14186 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:47.377609  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.338091 14255 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:47.330227 14235 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.404842  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.406773 14235 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.409716  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.410910 14255 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.414063  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:47.414143  7228 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
W20260709 13:19:47.414938  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 393970us
I20260709 13:19:47.417709  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.418534  6513 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:47.419152  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:47.419700  6520 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:47.420411  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.418386  6668 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:47.420753  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:47.421281  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 43757.3us
I20260709 13:19:47.422024  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.423187  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.415359  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:47.423859  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.424510  6665 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:47.429280  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:47.430334  6487 leader_election.cc:400] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:47.430696  6487 leader_election.cc:403] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:47.431706  6513 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:47.422435  6515 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:47.434720  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.435316  6668 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
W20260709 13:19:47.437726  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 63155.9us
I20260709 13:19:47.440243  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.440490  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.437764  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.443367  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.444455  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.444911  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.430727 14166 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:47.445626  6666 raft_consensus.cc:2393] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:47.379010 14297 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:47.448022  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.449383  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:47.449584  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:47.444022  6667 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:47.400193 14237 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.448359  6562 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:47.456466  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.458137  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:47.461680  7233 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:47.462497  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.463129  7237 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.463526  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:47.513226  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.513090 14287 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:47.519263 14150 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:47.520255 14282 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:47.520521 14019 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:47.513712 14039 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:19:47.514346  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.432885 14280 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:47.515108  6668 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:47.523823  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.524125  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.430987 14182 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:47.525194  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 76904.5us
I20260709 13:19:47.534727  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:47.450078 14235 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:47.527981  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62236.3us
I20260709 13:19:47.530483  6671 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:47.532060  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:47.541685  7200 leader_election.cc:400] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23.
I20260709 13:19:47.442268 14276 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:47.454823 14257 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:47.546320  7230 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:47.542433  7200 leader_election.cc:403] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:47.541719  6669 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:47.527457  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71061.5us
I20260709 13:19:47.436393 14272 consensus_queue.cc:1059] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:19:47.461573 13908 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:47.404681 14284 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.558233  6666 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:47.466996 14297 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:47.442420 13996 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.097s
I20260709 13:19:47.534737 14254 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:47.611853  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 51727.9us
W20260709 13:19:47.618052  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 157596us
I20260709 13:19:47.618599  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.619349  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:47.605436  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53826.4us
I20260709 13:19:47.620244  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.620519  6488 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:47.620919  7229 raft_consensus.cc:2368] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20.
I20260709 13:19:47.534077 14182 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.535523 14280 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:47.604610  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 56068.9us
I20260709 13:19:47.536146 14196 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:47.606027  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 45550.7us
I20260709 13:19:47.621449  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.618803  6637 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.640594  6637 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:47.536739 14166 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:47.635135  6664 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:47.641657  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:47.547290 14022 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:47.642026  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.609476 14268 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:19:47.574796 14235 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:47.645650  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.648602  7237 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:47.650097  7200 leader_election.cc:400] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23.
I20260709 13:19:47.664453 14321 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:47.674671  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 120430us
I20260709 13:19:47.692391  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:47.629523 14290 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:47.559249 14276 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.637696 14280 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:47.635744 14182 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:47.639261 14196 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.724759  6671 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:47.732285  6669 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:47.665216 14145 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:47.760048  6665 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:47.761085  6668 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:47.697723 14245 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:47.683605 14039 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:19:47.721212 14022 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:47.750043  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 109557us
I20260709 13:19:47.685458 14272 consensus_queue.cc:1059] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 13:19:47.740406 14288 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:47.747298 14243 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.000s
W20260709 13:19:47.757031 14281 consensus_meta.cc:287] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.001s
I20260709 13:19:47.746485 14045 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:47.742703 14290 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:47.819173  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 189731us
I20260709 13:19:47.821442  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:47.818223 14045 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.823781 14045 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:47.825254  6590 consensus_meta.cc:287] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.000s
I20260709 13:19:47.825817  6590 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:47.750599 14196 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:47.846653  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:47.772728 13995 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:19:47.820083 14239 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.788444 14245 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:47.887415  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.866685 14167 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:47.846329 14254 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:47.754886 14297 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.817914 14281 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.823068 14243 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:47.899650  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 103839us
I20260709 13:19:47.894819 14239 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:47.914975  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 222204us
I20260709 13:19:47.917263  6670 raft_consensus.cc:2393] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 26: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:47.917585  6663 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:47.867491 14166 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.926409  6520 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:47.872586 14235 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.914901 14284 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:47.922339 14290 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.933884 14276 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:47.940723 14180 consensus_meta.cc:287] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.000s
W20260709 13:19:47.943080 14319 log.cc:927] Time spent T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:19:47.915949 14330 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:19:47.945473 14245 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.943161  6597 raft_consensus.cc:2410] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 13:19:47.949699 14270 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.716s	user 0.006s	sys 0.004s
I20260709 13:19:47.956296 14243 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:47.957993 14281 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:47.961253 14058 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.000s
I20260709 13:19:47.961723 14058 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.911536 14022 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:47.928921 14239 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:48.000468 14239 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:47.926108 14317 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:47.931324 14248 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17.
I20260709 13:19:47.950114 14270 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.015421 14270 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:47.924319 14328 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:19:48.017449 14328 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:48.018007 14328 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:48.019336 14328 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:48.023068  6664 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:48.027426 14254 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:47.954456 14212 consensus_meta.cc:287] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.000s
I20260709 13:19:48.037456  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:48.046370  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:48.047042  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:48.061282  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:48.062109  6665 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:48.057224  6562 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:48.034601 14276 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.065325  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:48.061910  6637 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:48.075817  7229 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:48.065798  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:48.066145  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:48.077080  7232 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:48.084607  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.093513  6665 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:48.095484  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:48.096083  6668 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:48.103055  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:48.106974  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 411597us
I20260709 13:19:48.116359  7200 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:48.070720 14279 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.121433  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:48.098563 14284 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:48.018978 14280 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.123445 14058 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:48.098790 14064 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:48.121778  7233 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:48.136140  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:48.136765  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:48.137243  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:48.138520 14326 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:48.141814  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:48.118822 14328 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:48.144372  6595 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:48.140769  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:48.142721  6521 raft_consensus.cc:1275] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:48.142364  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:48.144006  6592 raft_consensus.cc:2393] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:48.131641 14282 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:48.142362  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.145023  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:48.147951  6519 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:48.154249  6590 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:48.144976  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:48.145197  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.134213 14284 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.162235  7231 raft_consensus.cc:2368] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 22. Current term is 23.
I20260709 13:19:48.108373 14276 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.171741  6514 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:48.170578  6590 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 27.
I20260709 13:19:48.170976  6666 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:48.117822 14196 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.150887 14064 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:48.176513 14180 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:48.176466  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:48.187352 14321 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.187691  6519 raft_consensus.cc:2368] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 26. Current term is 27.
I20260709 13:19:48.176986 14282 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:48.188239 14064 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.188972 14282 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.177315 14280 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:48.177891 13996 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:48.178354 14272 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:48.153151 14326 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.158275 14281 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:19:48.171169 14338 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:19:48.196986 13996 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:48.182140  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:48.201674 14167 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:48.201464 13996 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.202064 14167 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.202420 14167 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:48.148151 14235 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:48.175441 14284 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:48.187577 14150 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:48.204566 14150 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:48.205454 14150 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.206590 14150 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:48.206851  6591 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:48.207157 14145 raft_consensus.cc:3055] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:48.194554  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:48.207557 14145 raft_consensus.cc:740] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:48.199363 14326 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:48.208410 14145 consensus_queue.cc:260] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.209849 14145 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:48.211220 14323 log.cc:927] Time spent T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.386s	user 0.003s	sys 0.000s
I20260709 13:19:48.193859 14182 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.227466 14196 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:48.237006  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1890) took 1338 ms. Trace:
I20260709 13:19:48.237473  7230 rpcz_store.cc:276] 0709 13:19:46.899099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.940733 (+ 41634us) service_pool.cc:224] Handling call
0709 13:19:47.546143 (+605410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:47.546783 (+   640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.236807 (+690024us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b540028cdd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:48.236943 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366590848}
W20260709 13:19:48.266294  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 74476.3us
I20260709 13:19:48.268989  6664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 3339) took 1118 ms. Trace:
W20260709 13:19:48.269923  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 83755.6us
I20260709 13:19:48.275291  6637 leader_election.cc:400] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 26. Current term is 27.
I20260709 13:19:48.275812  6637 leader_election.cc:403] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 13:19:48.273591  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.269508  6664 rpcz_store.cc:276] 0709 13:19:47.128861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.128997 (+   136us) service_pool.cc:224] Handling call
0709 13:19:48.247279 (+1118282us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54001dced4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:48.247409 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129780992}
I20260709 13:19:48.287053  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:48.291786  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.293349  6487 leader_election.cc:400] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20.
I20260709 13:19:48.293763  6487 leader_election.cc:403] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:48.293062  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:48.297609  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:48.298641 14341 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:48.299880 14276 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:48.301326  7230 raft_consensus.cc:1240] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:48.292706 14317 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:48.305480 14285 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.292855 14281 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:19:48.292547 14196 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:48.312784 14196 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.313246 14196 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:48.321179 14182 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:48.335781  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.339732  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:48.334894 14073 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:19:48.345736  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 152878us
I20260709 13:19:48.346597  6594 raft_consensus.cc:1275] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:48.348191  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.354182  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.356699  6487 leader_election.cc:400] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 22. Current term is 23.
I20260709 13:19:48.365335 14030 consensus_queue.cc:1059] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:48.366391 14212 raft_consensus.cc:3055] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:48.342811 14045 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.366778 14212 raft_consensus.cc:740] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:48.367627 14212 consensus_queue.cc:260] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.368366 14045 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:48.369120 14212 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:48.369719  6522 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: [22.19-22.19]
I20260709 13:19:48.346151 14281 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:48.343200 14019 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:48.369513  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.372583  6594 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:48.376394  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 231245us
I20260709 13:19:48.377398  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:48.377622  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:48.378872  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.381747  6517 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:48.386005  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:48.358767 14073 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:48.360888 14239 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:48.361447 14058 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.393817  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:48.397090 14257 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:48.384158 14166 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20.
I20260709 13:19:48.392766 14254 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.384243 14349 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:48.411901  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.408283  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:48.370595 14281 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.426684  6589 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:48.404752 14186 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:19:48.411443 14150 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:48.430608  6591 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:48.426819  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:48.428624  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:48.432593  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.432958  6589 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:48.429721  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:48.433832  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:48.434514  6591 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
W20260709 13:19:48.435443  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.416602 14257 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:48.421627 14235 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:48.423174 14268 consensus_meta.cc:287] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.002s
I20260709 13:19:48.436167  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:48.433770  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:48.442045  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:48.417215 14349 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:48.444116  6595 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:48.423976 14353 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:48.447461  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:48.448185  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:48.450037  7228 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 17: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:48.453250  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:48.453263  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:48.420171 14284 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.466609  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:48.480214 14343 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.480805 14343 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.466943 13914 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:48.466693 14279 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:48.453603 14268 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23.
I20260709 13:19:48.482116 13914 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:48.482669 14279 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:48.483141 14279 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:48.452804 14321 consensus_queue.cc:1048] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:19:48.483256 14257 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:48.484074 14279 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.485467 14279 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.483230 13914 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.487510 13914 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:48.469133  6589 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:19:48.438174 14358 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:48.490602 14358 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.440003 14235 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.491052  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.491328 14235 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:48.491914  6589 raft_consensus.cc:2368] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20.
I20260709 13:19:48.426841 14073 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.496014 14073 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:48.497692 14317 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 26. Current term is 27.
I20260709 13:19:48.445451 14338 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:48.498654 14338 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.499034 14338 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:48.495361  6637 leader_election.cc:400] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20.
I20260709 13:19:48.492758  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.504158  6637 leader_election.cc:403] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:19:48.479302 14357 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:19:48.481710 14284 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:48.513825  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.514585  6589 raft_consensus.cc:2368] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 26. Current term is 27.
I20260709 13:19:48.515687  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.519065  7234 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:48.445626 14186 consensus_queue.cc:1243] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer 921c423406da4f289884afca51ca198f log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:48.449591 14349 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:48.541498  6664 raft_consensus.cc:2410] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:19:48.449793 14328 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:48.493037 14348 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:19:48.523981  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:48.438304 14280 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.073s
W20260709 13:19:48.529821  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.511560 14285 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:48.533851 14350 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.163s	user 0.003s	sys 0.000s
I20260709 13:19:48.535928 14294 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:48.537698 14346 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:19:48.541586  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:48.544597  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 49373.1us
I20260709 13:19:48.561313  6562 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.561856  6637 leader_election.cc:400] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 26. Current term is 27.
I20260709 13:19:48.562700 14022 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:48.565207  6597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3220) took 1264 ms. Trace:
I20260709 13:19:48.565595  6597 rpcz_store.cc:276] 0709 13:19:47.300457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.301101 (+   644us) service_pool.cc:224] Handling call
0709 13:19:48.565006 (+1263905us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54002f67d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:48.565152 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416999808}
I20260709 13:19:48.566749 14328 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:48.567306 14328 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.569509 14328 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:48.572572  6594 raft_consensus.cc:1275] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:19:48.573450  6594 pending_rounds.cc:85] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 14
I20260709 13:19:48.573669  6594 pending_rounds.cc:107] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:19:48.573900  6594 raft_consensus.cc:2889] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:48.574651  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:48.491104 14358 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:48.522847 14284 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20.
I20260709 13:19:48.553261  6522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1557) took 1132 ms. Trace:
W20260709 13:19:48.553694 14354 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:19:48.513330 14167 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.593242  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:48.582170  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:48.583536  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:48.595261  7229 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:48.479702 14196 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.578118  6522 rpcz_store.cc:276] 0709 13:19:47.416908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.417785 (+   877us) service_pool.cc:224] Handling call
0709 13:19:47.447245 (+ 29460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:48.370139 (+922894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.547802 (+177663us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54001e9454. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:48.548862 (+  1060us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120877952}
I20260709 13:19:48.541278 14180 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:19:48.607688  7232 raft_consensus.cc:2393] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:48.610728  7235 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:48.570163 14022 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.588688 14353 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:48.634907  6671 raft_consensus.cc:1240] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:48.623826  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2794) took 1208 ms. Trace:
I20260709 13:19:48.585556 14343 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:48.761152  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.627627  7229 raft_consensus.cc:2468] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:19:48.591403 14145 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.629456  6596 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:48.629863  6516 raft_consensus.cc:1240] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:48.644553  6670 rpcz_store.cc:276] 0709 13:19:47.415902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.422759 (+  6857us) service_pool.cc:224] Handling call
0709 13:19:48.623649 (+1200890us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54003fc854. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:48.623772 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1134386432}
W20260709 13:19:48.625310  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 165406us
W20260709 13:19:48.845821  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.886385  6588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1631) took 1592 ms (client timeout 1702 ms). Trace:
I20260709 13:19:48.572139 14294 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:48.885058  6636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.887684  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 247315us
W20260709 13:19:48.854768 14359 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
W20260709 13:19:48.906076  6588 rpcz_store.cc:269] 0709 13:19:47.294091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.294351 (+   260us) service_pool.cc:224] Handling call
0709 13:19:48.885990 (+1591639us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b54002f5dd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:48.886252 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181086976}
I20260709 13:19:48.928229  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:48.933109  7232 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:19:48.935662  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:48.941563  6636 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:48.947188  6636 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:48.907176 14022 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:48.910647 14333 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:48.974538  6588 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:48.975147  6588 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:48.904299 14186 consensus_queue.cc:1059] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 13:19:48.929960 13848 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.976063  6588 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:48.983634 14257 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.983839  6588 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.984251 14257 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:48.988204 14367 log.cc:927] Time spent T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:19:48.989749 14363 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.381s	user 0.000s	sys 0.001s
I20260709 13:19:48.991555 14287 raft_consensus.cc:3055] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:48.992092 14287 raft_consensus.cc:740] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:48.993106 14287 consensus_queue.cc:260] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:48.994942 14257 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:48.960363 14073 raft_consensus.cc:2804] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:48.957594 14328 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:49.003865  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48652.9us
I20260709 13:19:49.007712  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:49.003927  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 58709.2us
W20260709 13:19:48.994012 14212 consensus_meta.cc:287] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.624s	user 0.007s	sys 0.000s
I20260709 13:19:48.995574 14186 raft_consensus.cc:3055] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:49.021479  7200 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:48.995487 14257 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:19:49.004741 14280 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.015s
I20260709 13:19:49.013000 14186 raft_consensus.cc:740] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:49.012778 14073 raft_consensus.cc:697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:49.051800 14167 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:49.057049 14235 consensus_meta.cc:287] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.565s	user 0.004s	sys 0.004s
I20260709 13:19:49.052582 14073 consensus_queue.cc:237] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.057512 14235 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.058676  6593 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:49.059403 14294 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:48.978135 14022 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.051717 14186 consensus_queue.cc:260] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.061141 14186 raft_consensus.cc:3060] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:49.061465 14022 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:49.073834 14338 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.574s	user 0.004s	sys 0.003s
I20260709 13:19:49.007853 13848 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.076413 13848 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.028162 13996 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:19:48.981626 14333 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.087884 14333 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:49.089110 14281 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:49.089705 14356 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:49.089941 14196 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:49.090658 14356 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:49.091245 14356 raft_consensus.cc:697] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:49.092321 14356 consensus_queue.cc:237] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.095456 14373 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:49.096035 14373 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:49.095727 14358 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.074316 14338 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.133952 14381 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:49.145705  7229 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:19:49.145637 14381 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.146215  6520 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:49.146787 14358 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.136719  6588 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:19:49.126720  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:49.148614 14381 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:49.105377 14349 consensus_meta.cc:287] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.000s
I20260709 13:19:49.148238  6588 pending_rounds.cc:85] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 9
I20260709 13:19:49.149607  6588 pending_rounds.cc:107] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:19:49.149484 14349 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.149987  6588 raft_consensus.cc:2889] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:49.130895  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.151603 14349 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.138167 14235 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:49.133139  7199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.001s	sys 0.005s
W20260709 13:19:49.153861  7199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.002s	sys 0.006s
I20260709 13:19:49.148120 14338 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:49.160573  6561 leader_election.cc:336] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.670s (SENT)
I20260709 13:19:49.161170  6561 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:49.162319  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.163048  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:49.164572  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:49.142076  7200 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:49.172626 14351 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.805s	user 0.007s	sys 0.005s
I20260709 13:19:49.185191 13908 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.186966 14235 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:19:49.189409  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:49.190320  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:49.191042  6515 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 23: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:49.191478  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:49.193856 14326 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:19:49.194397  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:49.194628  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:49.195247 14326 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:49.200577  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.201473  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:49.202211  6588 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:49.202432 14288 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.203156 14288 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:49.205657  7234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1840) took 1587 ms (client timeout 1988 ms). Trace:
W20260709 13:19:49.206086  7234 rpcz_store.cc:269] 0709 13:19:47.618611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.637148 (+ 18537us) service_pool.cc:224] Handling call
0709 13:19:49.205510 (+1568362us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54002e4ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:49.205604 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816071424}
I20260709 13:19:49.209069  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:49.209446  6588 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:49.210660  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.210927  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:49.211107 14096 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.211591 14096 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.213833  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:49.215799 14366 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 13:19:49.151391  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1903) took 1085 ms. Trace:
I20260709 13:19:49.216274 14237 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:49.216445  7237 rpcz_store.cc:276] 0709 13:19:48.066146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.074992 (+  8846us) service_pool.cc:224] Handling call
0709 13:19:48.112870 (+ 37878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:48.240451 (+127581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:48.240482 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:48.252766 (+ 12284us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.398471 (+145705us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:48.398512 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.151024 (+752512us) spinlock_profiling.cc:243] Waited 79.8 ms on lock 0x7b5400646c54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:49.151073 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.151121 (+    48us) raft_consensus.cc:1685] finished
0709 13:19:49.151142 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.151331 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9977,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":183533952,"thread_start_us":111390,"threads_started":2,"wal-append.queue_time_us":157278}
I20260709 13:19:49.220072  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:49.221294 14353 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.222111 14182 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.222865 14182 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.225162 14182 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:49.193904  6522 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:19:49.225842  6522 pending_rounds.cc:85] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 12
I20260709 13:19:49.226094  6522 pending_rounds.cc:107] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:19:49.226317  6522 raft_consensus.cc:2889] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:49.227880  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:49.230003  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.230305  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.231820  6662 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:19:49.198241  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:49.232522  6518 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:49.233537  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.235133  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:49.144448 14145 consensus_queue.cc:1059] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.091s
I20260709 13:19:49.236882  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:49.239456  6662 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:19:49.215471 14096 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:49.227819 14182 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:49.240337 14017 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:49.241079 14017 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.241789 14317 consensus_queue.cc:1048] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:49.244313  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1559) took 1826 ms. Trace:
I20260709 13:19:49.247296  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:19:49.253047  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.253628  6516 rpcz_store.cc:276] 0709 13:19:47.417627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.419224 (+  1597us) service_pool.cc:224] Handling call
0709 13:19:48.236617 (+817393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.902538 (+665921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.243430 (+340892us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b540019afd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:49.244068 (+   638us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b540019afd0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90808ad 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:49.244241 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729205888}
W20260709 13:19:49.277863  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 105682us
I20260709 13:19:49.259301  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.280646 14030 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:49.282971  6593 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:49.260185 14317 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:49.271052 14365 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 13:19:49.286026  6665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45444 (request call id 2809) took 1225 ms. Trace:
I20260709 13:19:49.286386  6665 rpcz_store.cc:276] 0709 13:19:48.060930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.112288 (+ 51358us) service_pool.cc:224] Handling call
0709 13:19:48.113792 (+  1504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:48.114488 (+   696us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:48.250024 (+135536us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b1800455140. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:19:48.347575 (+ 97551us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.624249 (+276674us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.624484 (+   235us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:48.855404 (+230920us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.855571 (+   167us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:48.855617 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.855709 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.260744 (+405035us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b1800455140. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f94522 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:49.262169 (+  1425us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:49.285839 (+ 23670us) raft_consensus.cc:1685] finished
0709 13:19:49.285862 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.285972 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12131,"prepare.queue_time_us":69352,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":51630,"raft.queue_time_us":20541,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":328966,"spinlock_wait_cycles":144422272,"thread_start_us":309554,"threads_started":2,"wal-append.queue_time_us":561719}
I20260709 13:19:49.288998 14317 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:49.289979 14317 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.295516  6671 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:49.287485  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:49.299422 14180 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:49.299872 14282 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:49.300578 14324 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:49.300969 14324 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:49.301690 14324 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.302860 14324 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:49.302538  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:49.304898 14282 consensus_queue.cc:1059] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:49.305924 14282 raft_consensus.cc:3055] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:49.306269 14282 raft_consensus.cc:740] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:49.306478 14017 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.307111 14282 consensus_queue.cc:260] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.308393 14282 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:49.309891  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:49.304960 14039 consensus_queue.cc:1059] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:49.312670  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:49.313400  7233 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:49.313912 14272 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:19:49.314566  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 52705.1us
I20260709 13:19:49.316040  7228 raft_consensus.cc:1275] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:19:49.316360  6561 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:49.316471  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.317730  7228 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:49.317883  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:49.319792 14254 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:49.320010  7228 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:49.320694 14254 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:49.321409 14180 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:49.322728  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.186014 13908 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.324564 14290 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:49.454281 13908 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.222060 14353 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.460839 14353 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.328137  6593 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:49.329725  6592 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:19:49.336409  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:49.339913  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:49.343856  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.464316  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:49.303910  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:49.312830 14039 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:19:49.322270  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:49.372165  6662 raft_consensus.cc:1240] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:49.385414  7233 raft_consensus.cc:1275] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:19:49.387393  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:49.394399  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.395810  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:49.398463  6665 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:49.478086  7237 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:49.399562  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.478912  6671 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:49.402122  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:49.406549  6663 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:49.403921  6636 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.205773 14288 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:49.324558 14287 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:49.420799  6596 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:19:49.421484  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.423632  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.431420  6519 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: [22.19-22.19]
I20260709 13:19:49.432459  6588 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:49.436640  6594 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:49.438311  6670 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:49.469821  7236 raft_consensus.cc:2410] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:19:49.470059  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:49.470074  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.470561  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:49.489566  6589 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:49.118916  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:49.519346  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 115534us
W20260709 13:19:49.539371  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 176376us
I20260709 13:19:49.119170  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:49.537253  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:49.520977  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 126880us
W20260709 13:19:49.538544  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 215909us
I20260709 13:19:49.543274  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:49.533644  6518 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:49.544628  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:49.549566  7237 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
W20260709 13:19:49.551550  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.097503 14280 raft_consensus.cc:3055] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:49.557940  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.520550 13996 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.536696 14276 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:49.577775  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:49.565939  6520 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:49.579005  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.530519 14102 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:49.542892 14324 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:49.585858  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:49.530522 14030 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.064s
I20260709 13:19:49.606109  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:49.579542 14349 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:49.617671  6589 raft_consensus.cc:606] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:49.556077 14254 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:49.618130 14294 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:49.618520 14254 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.619345 14254 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:49.605432  6595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3229) took 1167 ms. Trace:
I20260709 13:19:49.618214  6589 raft_consensus.cc:614] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:49.569893 14280 raft_consensus.cc:740] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:49.584697 14248 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:19:49.583845 14377 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:49.624888  6595 rpcz_store.cc:276] 0709 13:19:48.438483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.454880 (+ 16397us) service_pool.cc:224] Handling call
0709 13:19:48.455157 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.455340 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:48.512245 (+ 56905us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.274340 (+762095us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b180048cfa0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b 00007f04b8c20d23 00007f04b8c2123a 00007f04b908ba2a
0709 13:19:49.605037 (+330697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:49.605107 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:49.605160 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.605238 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.605372 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68572,"prepare.queue_time_us":56733,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":15844,"spinlock_wait_cycles":21185920,"thread_start_us":261791,"threads_started":1,"wal-append.queue_time_us":1090777}
I20260709 13:19:49.584487 14017 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:49.657497  6589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542a321aae654f82803540bffcf7968f"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:49.584514 14096 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.572065 14182 consensus_queue.cc:1048] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:19:49.097106 14373 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.673321  6589 raft_consensus.cc:606] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.583016 13996 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.584975 14326 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.080s
W20260709 13:19:49.682395  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 100325us
I20260709 13:19:49.588747 14380 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.083s
I20260709 13:19:49.632408 14102 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:49.673074 14395 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:49.671394 14391 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:49.676573 14383 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:49.686287  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 112909us
I20260709 13:19:49.688231 14096 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.675839 14248 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:49.675549 14377 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.725117 13996 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.743863 14237 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:49.745496 14397 log.cc:927] Time spent T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:19:49.748798 14145 raft_consensus.cc:3055] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:49.758740  6663 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:19:49.817255  6516 pending_rounds.cc:85] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 15
I20260709 13:19:49.818008  6516 pending_rounds.cc:107] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:19:49.818264  6516 raft_consensus.cc:2889] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:49.760434  7198 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:49.761477 14396 log.cc:927] Time spent T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:19:49.767452 14321 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.761528  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:49.769730 14326 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.771401  6590 raft_consensus.cc:1240] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: [22.19-22.19]
I20260709 13:19:49.772583  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.777504  6520 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:49.778254 14102 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.816232 14073 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:49.844372 14073 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.845614 14102 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:49.784539 14383 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.788228 14280 consensus_queue.cc:260] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.797511  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1913) took 1207 ms. Trace:
W20260709 13:19:49.797986  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65520.3us
I20260709 13:19:49.799785  6588 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
W20260709 13:19:49.861168  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 100830us
I20260709 13:19:49.862210  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:49.832525  6594 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:49.741101 14272 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.094s
I20260709 13:19:49.864359  6563 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.865818  6520 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:19:49.866214  6520 pending_rounds.cc:85] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Aborting all ops after (but not including) 10
I20260709 13:19:49.866420  6520 pending_rounds.cc:107] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:19:49.866655  6520 raft_consensus.cc:2889] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:49.850666  6588 pending_rounds.cc:85] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 16
I20260709 13:19:49.870383  6588 pending_rounds.cc:107] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 20.17
I20260709 13:19:49.870708  6588 raft_consensus.cc:2889] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:49.875041  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.876817  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.877611  6595 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
W20260709 13:19:49.880930  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 300963us
I20260709 13:19:49.814328 14145 raft_consensus.cc:740] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:49.848325  7231 rpcz_store.cc:276] 0709 13:19:48.590234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.590448 (+   214us) service_pool.cc:224] Handling call
0709 13:19:48.594314 (+  3866us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.139810 (+545496us) spinlock_profiling.cc:243] Waited 51.7 ms on lock 0x7b18004b4cc0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8c294bc 00007f04b5f7748b 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04b9079060 00007f04b9079bfb 00007f04b908464b
0709 13:19:49.139967 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.140016 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.140484 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:19:49.535272 (+394788us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:49.535322 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.535611 (+   289us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.797239 (+261628us) raft_consensus.cc:1685] finished
0709 13:19:49.797260 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.797446 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31312,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":123371,"spinlock_wait_cycles":118968064,"thread_start_us":620759,"threads_started":2,"wal-append.queue_time_us":441495}
I20260709 13:19:49.848184 14280 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:49.850798 14239 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.846575 14073 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:49.855907 14377 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:49.882258  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.848026 14337 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:49.906399  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 62322.3us
I20260709 13:19:49.839280 14378 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:49.906632  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 329384us
I20260709 13:19:49.813807 14237 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.847741 14383 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.872982 14288 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.886400 14384 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:19:49.930689  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 48521.3us
I20260709 13:19:49.895684 14393 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.896018 14327 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:19:49.941087  6520 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:19:49.941649  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:49.942679  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.943171  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:49.944523  6594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "627436404a154d80a882f4900dff2948"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:45976
W20260709 13:19:49.945068  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 61650.4us
I20260709 13:19:49.884032 14379 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:19:49.881230 14321 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.990953  6594 raft_consensus.cc:606] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Received request to transfer leadership to TS 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.991405  6594 raft_consensus.cc:614] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:49.991298  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:49.947140  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:49.856585 14326 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:49.940434  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:49.935904 14019 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:19:49.994609  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47527.1us
I20260709 13:19:49.929284 14239 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.850930 14248 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.932055 14317 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.928270 14412 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:49.931504 14343 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:19:50.020560  6562 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:49.948768 14102 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:49.948351 14338 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.971792 14255 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:49.973254 14288 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.027285 14411 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:50.029158 14239 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:49.942909 14378 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.995728 14372 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:50.063390 14401 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:50.068996 14326 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:49.949007 14167 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:19:50.037931  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a72647203b143708b26072192a9c102"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:49.933233 14039 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:49.997376 14384 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:49.997144 14383 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:50.020851  6636 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:50.076694  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 75992.7us
W20260709 13:19:50.042758  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 58996us
I20260709 13:19:50.049917  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.031340 14373 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:50.043591 14255 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:49.944060 14337 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.036733  6562 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:50.041008 14414 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.132870 14414 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.084298  7235 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:50.076186 14039 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.135375 14237 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.050151 14182 consensus_queue.cc:1048] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.086s
I20260709 13:19:50.001143 14254 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.054149 14381 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:50.070649 14378 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.132288 14337 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:50.137091 14409 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:19:50.137213 14337 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:50.135717 14414 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.138599 14414 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.139107 14334 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:50.139674 14334 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:50.139961 14381 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.140202 14334 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:50.140442 14381 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:50.056970 14102 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.142818 14102 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.144168 14254 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.122247  6488 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:50.145699  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:50.146365 14294 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.147004 14294 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.147867 14406 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:19:50.149922 14294 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:50.150398 14294 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.151515 14384 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.154752 14096 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:50.155483 14096 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.156062 14096 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.158862 14376 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:50.162256 14290 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:19:49.850912 14282 consensus_queue.cc:1175] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "921c423406da4f289884afca51ca198f" responder_term: 23 status { last_received { term: 20 index: 18 } last_committed_idx: 18 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: [22.19-22.19]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:49.985343 14393 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.169715  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:50.152403 14294 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.040648 14180 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:50.181033  6520 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
W20260709 13:19:50.186437  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 243348us
I20260709 13:19:50.035562 14405 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 13:19:50.204651  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 168018us
I20260709 13:19:50.208084  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:50.218142  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.218756  7200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.223099  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:50.223613  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:50.224231  6590 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:50.223630  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:50.055392 14296 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 13:19:50.228839  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.229669  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:50.055929 14338 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.231524  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:50.232095  6596 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:19:50.042825 14317 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:50.071456 14391 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.154s
I20260709 13:19:50.037233 14412 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.233680  6589 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:50.230607 14182 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:50.225283 14237 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:50.275187  6591 raft_consensus.cc:606] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:50.275650  6591 raft_consensus.cc:614] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:50.241648 14096 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.269423 14237 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:50.179795 14381 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:50.071956 14411 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:50.297984  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 220247us
I20260709 13:19:50.249183 14294 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:19:50.270047 14254 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:50.319435  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.332872  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:50.331310  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:50.346484  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.351331  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:50.316715 14287 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.836s	user 0.007s	sys 0.006s
W20260709 13:19:50.354377  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 334123us
I20260709 13:19:50.369966  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.348028 14353 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.370873  6563 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:50.454213  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 83396.5us
I20260709 13:19:50.454385 14378 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.457412 14255 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.359612  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.354897 14321 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:50.335099 14254 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:50.464102 14412 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.467218 14381 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.385783  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.386106  6513 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:19:50.402798 14237 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:50.473903 14418 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.153s	user 0.004s	sys 0.000s
I20260709 13:19:50.477974 14288 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.466326 14254 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.404270 14411 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.404855 14376 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:50.386473 14401 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:50.408149 14338 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.421149 14393 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.424783  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.426105  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.431856  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.436095 14430 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:50.441553 14372 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:50.389758 14235 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:50.444236  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.492599 14376 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.358175  7237 raft_consensus.cc:1275] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:50.457587  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:50.531011  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 73593.6us
I20260709 13:19:50.354866  7231 raft_consensus.cc:1275] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:50.499295  7233 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:50.501037  7236 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:50.510264  6670 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:50.512579  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.493191 14288 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.527377  6665 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:50.446676 14317 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:50.539137  6591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a758d1835d6148668ab495504660d72a"
dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
mode: GRACEFUL
new_leader_uuid: "53fee9894731461a821a131be24dbbca"
 from {username='slave'} at 127.0.0.1:45976
I20260709 13:19:50.539976  6591 raft_consensus.cc:606] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Received request to transfer leadership to TS 53fee9894731461a821a131be24dbbca
I20260709 13:19:50.540426  6591 raft_consensus.cc:614] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:50.527715  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.540899  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.541052  6662 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 17.
I20260709 13:19:50.541492  7231 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:50.535765 14235 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:50.550257  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:50.550994 14145 consensus_queue.cc:260] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.551879 14235 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.552016 14145 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:50.547747 14237 consensus_queue.cc:1048] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:19:50.557039 14039 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.561132  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:50.536198 14376 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:50.535863 14393 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.539112 14254 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:50.573683  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.574535  7229 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:50.584662  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:50.586444  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:50.601627  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:50.607641  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:50.529009 14372 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.611622  7237 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:50.557147 14334 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.614984  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.615753  6669 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:50.570013 14317 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:50.543745 14288 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:50.620517  7198 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:50.621690  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:50.609651 14338 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:50.611925 14353 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:50.528431 14430 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.613179 14377 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.620131 14317 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.606777 14393 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.623624 14411 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.648504  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:50.628425 14427 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:50.655081 14411 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.609596 14424 consensus_queue.cc:1048] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 13:19:50.654601 14255 raft_consensus.cc:993] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: : Instructing follower 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:19:50.655454 14427 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:50.657140 14411 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.624652 14431 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.657163 14255 raft_consensus.cc:1081] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Signalling peer 64b95375974b48d18c596a23c9cac75c to start an election
I20260709 13:19:50.658277 14431 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.657356 14427 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:50.660490 14431 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.638023 14353 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.661571 14353 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:50.666986 14427 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.670167  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:50.628643 14401 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:50.626999 14338 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:50.674710  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 328791us
I20260709 13:19:50.646483 14317 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:50.684482  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:50.685515  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:50.686336  6517 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:50.687899  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:50.679666 14401 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:50.694868  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 308150us
I20260709 13:19:50.702395  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.710202  6670 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 17.
I20260709 13:19:50.704090  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.680527 14338 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:50.711419  6663 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:50.712406  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:50.718511 14401 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:50.709211 14429 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:50.685124 14431 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
W20260709 13:19:50.747697 14439 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:50.717903 14407 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 13:19:50.736959 14429 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:50.850693  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 106437us
I20260709 13:19:50.827593 14039 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.830452 14282 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.853771  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:50.829782 14237 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.830731 14373 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:50.867796  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:50.870082  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:50.833849 14254 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.859692  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 94617.3us
I20260709 13:19:50.842840 14372 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.872203  7198 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:50.861248 14429 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:50.866459  7231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "542a321aae654f82803540bffcf7968f"
dest_uuid: "64b95375974b48d18c596a23c9cac75c"
 from {username='slave'} at 127.0.0.1:46680
I20260709 13:19:50.892161  7231 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:50.892572  7231 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:50.878419 14412 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:50.915153 14437 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:50.916160 14430 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.918824  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:50.875980 14282 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.893316 14427 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.873091 14039 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.878522 14373 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.895556 14372 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:50.875976 14237 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:50.882515 14254 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.008371 14254 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.918308 14412 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.009680 14412 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.006351 14237 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.020159 14039 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.021519 14237 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.022087 14237 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.024878 14237 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.022097 14373 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.025046 14372 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.027452 14372 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:51.035547 14441 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:19:51.035931  7231 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.036743  7233 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:19:51.037146  7233 pending_rounds.cc:85] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 15
I20260709 13:19:51.037384  7233 pending_rounds.cc:107] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:19:51.037622  7233 raft_consensus.cc:2889] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:51.040148 14353 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.041098  7231 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.001219 14413 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:51.053172 14413 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.055485 14413 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.073449 14372 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.075590 14334 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.045706 14412 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.082991 14412 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:50.995664 14288 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:51.087774 14420 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.778s	user 0.003s	sys 0.000s
I20260709 13:19:51.089107  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1579) took 1305 ms. Trace:
I20260709 13:19:51.089454  6516 rpcz_store.cc:276] 0709 13:19:49.784063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.792988 (+  8925us) service_pool.cc:224] Handling call
0709 13:19:49.804819 (+ 11831us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.818795 (+ 13976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.818902 (+   107us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.819306 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.000150 (+180844us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:50.000202 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.000391 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.087127 (+1086736us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:51.088800 (+  1673us) raft_consensus.cc:1685] finished
0709 13:19:51.088828 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.089040 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8946560,"thread_start_us":180502,"threads_started":1,"wal-append.queue_time_us":447548}
I20260709 13:19:51.099995 14379 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:51.100634 14393 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.101320 14288 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.101905 14288 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.103034 14282 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:50.989939 14384 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.103860 14384 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.104159 14288 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:51.105214 14442 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:19:51.105997 14384 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.110369 14338 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.111341 14372 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.100666 14379 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.118187 14379 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:51.119858 14450 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:19:51.042397 14353 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:51.155148 14417 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.975s	user 0.001s	sys 0.005s
I20260709 13:19:51.156466  7230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 2122) took 2038 ms. Trace:
I20260709 13:19:51.156840  7230 rpcz_store.cc:276] 0709 13:19:49.118169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.165486 (+ 47317us) service_pool.cc:224] Handling call
0709 13:19:49.165780 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.406650 (+240870us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b18003c98e0. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f943f3 00007f04b5f967c1 00007f04b5f96779 00007f04b5f96709 00007f04b5f966d1 00007f04b5f959cd 000000000038f354 000000000038f199 00007f04b5f7783a
0709 13:19:49.715566 (+308916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.715602 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.716165 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:19:49.930667 (+214502us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:49.930726 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.930847 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.154506 (+1223659us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:51.156250 (+  1744us) raft_consensus.cc:1685] finished
0709 13:19:51.156272 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.156406 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22586,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":476976640,"thread_start_us":535067,"threads_started":2,"wal-append.queue_time_us":324567}
W20260709 13:19:51.159786 14435 log.cc:927] Time spent T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.316s	user 0.002s	sys 0.002s
I20260709 13:19:51.163287 14254 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.164028 14254 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.168697 14386 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.169390 14386 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.018442 14145 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.171277 14145 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.173014 14386 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.173535 14145 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.176632 14102 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.177241 14102 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.178790 14384 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:51.179333 14102 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.179307 14384 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.166461 14254 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.182142  6592 raft_consensus.cc:1240] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:19:51.182361 14384 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.182935 14384 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.183493 14254 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.183447 14384 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.184262 14254 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.200063 14254 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:51.179929 14376 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.002s
I20260709 13:19:51.200891 14386 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:51.203258 14386 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.166491 14317 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.208734 14384 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.218228 14451 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:51.178035 14429 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.233855  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:51.235838  6487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.005s	sys 0.008s
W20260709 13:19:51.236231  6487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.005s	sys 0.008s
W20260709 13:19:51.236515  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 526295us
I20260709 13:19:51.227909 14424 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.239352 14454 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.239681 14424 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.240095 14454 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.224223 14451 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:51.202327 14376 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.268309 14414 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.245803  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:51.268828  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:51.282302 14386 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.291663  6595 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 17.
I20260709 13:19:51.261581 14412 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.268776 14451 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:51.261457  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:51.291435 14414 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.245396  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:51.269464 14317 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:51.241806  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 347022us
I20260709 13:19:51.241894 14334 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:51.243726  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:19:51.296945  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 53335.2us
I20260709 13:19:51.292605 14412 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.298470  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:51.299279  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.295166  6596 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:19:51.244697  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:51.297540 14334 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:51.301497  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:51.302856  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.303432  6665 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:51.302243  6662 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:51.304603  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.301625 14421 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:51.305303 14421 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.305655 14421 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:51.305962  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.306540  6665 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:51.307984  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.295186 14414 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.308705  6665 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 16.
I20260709 13:19:51.308362 14412 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.310263  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.312279  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.313668  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.315335  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.324004  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.327018  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.241659 14413 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.328441  6663 raft_consensus.cc:2410] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 28 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:19:51.192868 14401 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.330422  6666 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:51.330813  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.338932  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:51.340798  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 103720us
I20260709 13:19:51.341516  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:51.345937  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.346668  7236 raft_consensus.cc:2368] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19.
I20260709 13:19:51.347087  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.347641  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.347843  6670 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:51.354276  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:51.355424  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:51.356048  6596 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 27.
I20260709 13:19:51.345634  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.362679  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.335273 14255 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:51.370075  7198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.365667  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:51.368391  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:51.373279 14412 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.373989 14412 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:51.374737 14317 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:19:51.376658  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.379923  6522 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:19:51.377583  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.382740  6513 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:51.386026  6515 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:51.387214  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:51.388865  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 499130us
I20260709 13:19:51.374559 14401 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.392241  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.379447 14317 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.402966 14317 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:51.368934 14145 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.333010 14413 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.363862 14280 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.389931  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:51.405905  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.410382  6519 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:51.370911 14461 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:51.417954  6594 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.015s	sys 0.014s
W20260709 13:19:51.427196  6594 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.015s	sys 0.015s
W20260709 13:19:51.469352  7200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.002s	sys 0.019s
W20260709 13:19:51.469866  7200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.002s	sys 0.020s
W20260709 13:19:51.470193  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 587331us
I20260709 13:19:51.472074  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.472972  7234 raft_consensus.cc:2410] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 24 because replica has last-logged OpId of term: 22 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 18.
I20260709 13:19:51.473892  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.474265  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.474500  6515 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:51.380625 14255 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.488380  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.489392  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.491956  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.498973  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.492355 14436 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.492616  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.492969  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.493258  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.493603  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.494490  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.495395  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.442138  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3251) took 1233 ms. Trace:
I20260709 13:19:51.502523  6517 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:51.428864 14385 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.528008  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.504119  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:51.525781  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.429932 14381 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:51.530766  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.544476  7235 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 18: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:51.531076  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.546938  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:51.530328  6589 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.015s	sys 0.023s
I20260709 13:19:51.432085 14338 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:51.541390  6486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.020s	sys 0.045s
W20260709 13:19:51.547888  6589 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.015s	sys 0.024s
W20260709 13:19:51.548087  6486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.020s	sys 0.046s
W20260709 13:19:51.548575  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 694820us
I20260709 13:19:51.542095  6522 raft_consensus.cc:2393] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:51.502390  6594 rpcz_store.cc:276] 0709 13:19:50.208979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.220705 (+ 11726us) service_pool.cc:224] Handling call
0709 13:19:50.221055 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.308695 (+ 87640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.308739 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.335618 (+ 26879us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.687567 (+351949us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:51.427665 (+740098us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.440004 (+ 12339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.441871 (+  1867us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.441918 (+    47us) raft_consensus.cc:1685] finished
0709 13:19:51.441931 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.442073 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102405,"prepare.queue_time_us":396951,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":89924,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":15595,"thread_start_us":1015036,"threads_started":2,"wal-append.queue_time_us":514967}
I20260709 13:19:51.549619  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.549140  6486 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:51.557539  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.560513  7235 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:51.411463 14145 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.565032  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.421350 14280 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.569878  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.572964  7200 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:51.573354  7233 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:51.570652  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:51.551074 14441 log.cc:927] Time spent T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:51.576648  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:51.577227  6517 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:51.578598  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.578879  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.432214 14254 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:51.582683  7233 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:51.582858  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:51.586962  6522 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.586527 14254 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.555411 14429 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.591579 14343 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:51.598163 14343 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.591275 14272 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.599874 14272 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.600524 14145 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:51.601876 14272 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.602175 14280 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.607388  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.610577  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:51.530525 14436 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.585080  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:51.625980  6517 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
W20260709 13:19:51.583959  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 216134us
I20260709 13:19:51.618249  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:51.547667 14385 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.616791 14268 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:51.635519 14268 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:51.580757 14454 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:51.636286  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 51258.8us
I20260709 13:19:51.584481 14424 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.643891  6520 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:51.639132  7200 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:51.577210 14470 consensus_queue.cc:1059] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:19:51.650276  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 307067us
I20260709 13:19:51.629634 14381 raft_consensus.cc:2764] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:51.653132  7231 raft_consensus.cc:2410] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:19:51.655507  6596 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:51.655928 14383 raft_consensus.cc:2804] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:51.661972  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.663476 14255 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.663597  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.664464  6594 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:19:51.665320  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:51.665637  6487 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:51.647701 14353 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:51.673022  6637 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.675089  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:51.689366  7228 raft_consensus.cc:2393] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:51.674737  6591 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:51.678221  6522 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
W20260709 13:19:51.647956  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 285359us
I20260709 13:19:51.691407  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:51.694443  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.695896  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.697328  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.690830  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:51.699460  6522 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:51.700152  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:51.648943  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.639073 14268 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.712016  6522 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:51.717258  6637 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.718744  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.720508  6637 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:51.722645  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:51.743714  7200 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:51.725416  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.757014  6515 raft_consensus.cc:2376] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19. Re-sending same reply.
I20260709 13:19:51.762822  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:51.763149  7228 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.755640  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:51.765096  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.766891  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.715196 14167 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:51.687527 14353 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.714910 14383 raft_consensus.cc:697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:51.725854 14338 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:51.732086 14254 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.725713 14288 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:51.761010 14455 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:51.830468  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.863641  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.865916  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.687688 14474 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.191s
W20260709 13:19:51.868083  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 112537us
I20260709 13:19:51.732549 14424 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:51.766325 14377 raft_consensus.cc:2804] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:51.880081 14385 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.880280 14377 raft_consensus.cc:697] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:51.765714 14410 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:51.881587  7237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46666 (request call id 1937) took 1009 ms. Trace:
I20260709 13:19:51.881389 14377 consensus_queue.cc:237] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.881979  7237 rpcz_store.cc:276] 0709 13:19:50.871808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.871941 (+   133us) service_pool.cc:224] Handling call
0709 13:19:50.881155 (+  9214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.232631 (+351476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.232678 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.233266 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.564741 (+331475us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:51.564778 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.564836 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.881381 (+316545us) raft_consensus.cc:1685] finished
0709 13:19:51.881401 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.881530 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9418,"raft.queue_time_us":9804,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":15874,"thread_start_us":669363,"threads_started":2,"wal-append.queue_time_us":488372}
I20260709 13:19:51.883090 14383 consensus_queue.cc:237] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.723003 14321 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:51.863660 14353 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:51.830224 14334 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:51.846689 14456 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.726168 14472 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:51.878737 14030 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:51.879127 14288 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:51.861728  6518 raft_consensus.cc:2393] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 23: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:51.891137  6515 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 14.
I20260709 13:19:51.847961 14413 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.830802 14460 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.870652 14454 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:51.919538  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.925586  7228 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:51.927696  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.872385 14255 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:51.933419  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:51.935698  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.936807  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.950587  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:51.961925  6517 raft_consensus.cc:2368] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19.
I20260709 13:19:51.964576  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.928586 14239 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:51.961033  7237 raft_consensus.cc:2393] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 18: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:51.928812 14406 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.929133 14471 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.968205  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:51.916512 14470 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:19:51.974305  7228 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:51.943323 14472 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.963527 14288 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:51.990849  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 270386us
I20260709 13:19:51.990495  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.992607  7233 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:51.994259  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:51.962954 14255 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:51.995244  6637 leader_election.cc:400] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19.
I20260709 13:19:51.930094 14456 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.934252 14030 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.005451  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:51.986393 14436 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:51.944154 14460 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:51.999549  7228 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:52.010768  7198 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:51.988709 14413 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:52.022046  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:52.023715  7228 raft_consensus.cc:2393] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:51.976379 14239 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.039175 14239 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.041536 14268 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.043999 14030 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:52.045293 14473 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:19:52.045125  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:52.018765 13908 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:52.049141  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:52.050827 14288 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:52.051733 14477 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:19:52.052587 14475 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:19:52.053522  6562 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:52.055397 14487 consensus_queue.cc:1059] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:52.007079 14255 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:51.977632 14406 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.054410  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:52.050350  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:52.114720  7231 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:19:52.056895 14487 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:51.980525 14471 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:52.122262  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 67940.3us
I20260709 13:19:52.121430  6589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55992 (request call id 3258) took 1411 ms. Trace:
I20260709 13:19:52.123914 13995 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:52.129263 13995 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.131310 14479 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:52.131757 14479 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:52.132155 14479 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:52.132182 14235 raft_consensus.cc:3055] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:52.132941 14235 raft_consensus.cc:740] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:52.036542 14463 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:52.139927 14463 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.140327 14463 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:52.120630 14487 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:52.144057 14487 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:52.144841 13995 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.152176 14471 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.154724 14479 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.157368 14479 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.163666 14423 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.164302 14423 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.167205 14423 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.168391 14460 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.036062 14413 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.171984 14456 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.031011 14280 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.173920 14280 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.176455 14280 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:51.997359 14472 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.179841 14472 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.181699 14485 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.182262 14321 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:52.191043 14334 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.191395 14321 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.191637 14334 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.192988 14102 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:52.193718 14321 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.193537 14102 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:52.132526 14469 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 13:19:52.187929  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:52.195731 14102 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:52.196528  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:52.197754  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:52.189724  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:52.202847  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:52.203693  6520 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:19:52.205799  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
W20260709 13:19:52.120535 14468 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
W20260709 13:19:52.132230 14421 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.826s	user 0.000s	sys 0.011s
W20260709 13:19:52.299558  6561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.009s	sys 0.016s
W20260709 13:19:52.330626  6561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.010s	sys 0.016s
I20260709 13:19:52.212106 14166 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.230086  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:52.319027  6563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.005s	sys 0.021s
W20260709 13:19:52.365793  6563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.006s	sys 0.021s
W20260709 13:19:52.366241  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 677510us
I20260709 13:19:52.117657 14484 raft_consensus.cc:3055] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:52.126492  6589 rpcz_store.cc:276] 0709 13:19:50.709742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.711060 (+  1318us) service_pool.cc:224] Handling call
0709 13:19:50.711435 (+   375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:50.719859 (+  8424us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:50.846233 (+126374us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.874672 (+1028439us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:51.893431 (+ 18759us) log.cc:844] Serialized 52 byte log entry
0709 13:19:51.918340 (+ 24909us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:51.929074 (+ 10734us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.929288 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.929426 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.121154 (+191728us) raft_consensus.cc:1685] finished
0709 13:19:52.121182 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.121358 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92456,"prepare.queue_time_us":374722,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":2092,"raft.queue_time_us":256751,"raft.run_cpu_time_us":457,"raft.run_wall_time_us":26302,"thread_start_us":1108037,"threads_started":3,"wal-append.queue_time_us":784512}
W20260709 13:19:52.321503  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 107278us
I20260709 13:19:52.321861  6518 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
W20260709 13:19:52.324743  7229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.001s	sys 0.025s
I20260709 13:19:52.401819  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:52.319339  7237 raft_consensus.cc:1240] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:52.320158  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 119216us
W20260709 13:19:52.138423 14317 consensus_meta.cc:287] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.001s
W20260709 13:19:52.331483  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 640107us
I20260709 13:19:52.412015  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:52.413326  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.322929 14102 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:52.400861  6563 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
W20260709 13:19:52.401435  6636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.012s	sys 0.027s
I20260709 13:19:52.420483  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:52.325630 14376 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:52.329645 14272 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.022s	sys 0.012s
I20260709 13:19:52.435752  6563 leader_election.cc:403] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:52.449398  6561 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:52.404601 14343 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.410884 14430 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:19:52.435760  6636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.013s	sys 0.027s
W20260709 13:19:52.496871  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47593us
I20260709 13:19:52.508996  7233 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:52.510231  6590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55990 (request call id 3253) took 1264 ms. Trace:
I20260709 13:19:52.510774  6590 rpcz_store.cc:276] 0709 13:19:51.245679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.245796 (+   117us) service_pool.cc:224] Handling call
0709 13:19:51.246102 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.247534 (+  1432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.247587 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.248165 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.566586 (+318421us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:51.566652 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.566744 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.509932 (+943188us) raft_consensus.cc:1685] finished
0709 13:19:52.509966 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.510148 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15988,"raft.queue_time_us":2800,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":17619,"thread_start_us":317527,"threads_started":1,"wal-append.queue_time_us":424055}
I20260709 13:19:52.414572 14030 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.182310 14485 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:52.523677  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 156663us
I20260709 13:19:52.539170  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:52.524291 14379 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:52.494138 14479 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:52.525161  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:52.525609  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.401151 14484 raft_consensus.cc:740] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:52.552415  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.436841 14460 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:52.553831  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.406493 14421 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.526371  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.492465 14102 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.437194 14472 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:52.527103  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.528828  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.529336  6594 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:52.530284 14449 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.531448 14438 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:52.322956 14429 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.012s	sys 0.014s
I20260709 13:19:52.534462 14483 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.437608 14287 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.021s	sys 0.050s
I20260709 13:19:52.537220  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:52.435079 14321 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.590781 14321 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.591426  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:52.591749  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:52.592834  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60540 (request call id 3019) took 1226 ms. Trace:
I20260709 13:19:52.593246  6516 rpcz_store.cc:276] 0709 13:19:51.366445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.366584 (+   139us) service_pool.cc:224] Handling call
0709 13:19:52.591556 (+1224972us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b5800010598. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5ca22 00007f04bee75c09 00007f04bdfcfee0 00007f04be0990ab 00007f04be099131 00007f04be0470a0 00007f04be046984 00007f04b75c16d4 00007f04b75c1662
0709 13:19:52.592732 (+  1176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8301952}
W20260709 13:19:52.596419 14451 consensus_meta.cc:287] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 1.304s	user 0.014s	sys 0.002s
I20260709 13:19:52.596877 14451 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:52.492419 14272 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.023s	sys 0.012s
I20260709 13:19:52.540503  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:52.599728  6516 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:52.600454  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 3029) took 1027 ms. Trace:
I20260709 13:19:52.544552  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.600914  6516 rpcz_store.cc:276] 0709 13:19:51.573377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.594373 (+1020996us) service_pool.cc:224] Handling call
0709 13:19:52.599547 (+  5174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.600164 (+   617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.600390 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.560815 14472 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:52.601919 14472 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:52.602097  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.413247 14235 consensus_queue.cc:260] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.603067 14235 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:52.603298  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.540095  6563 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.525853  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.549839  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 3028) took 1007 ms. Trace:
I20260709 13:19:52.557934  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:52.442258 14386 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:52.415467 14166 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.609714  6665 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:19:52.435060 14317 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.558845 14460 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.551592 14030 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.587704  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.589900  6663 raft_consensus.cc:2410] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:19:52.589982 14413 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.556661 14334 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.501703 14324 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:52.545506 14430 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:52.612566  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 72592.3us
I20260709 13:19:52.674907  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.676185  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:52.613787  6588 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:52.517616 14376 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:52.608803  6520 rpcz_store.cc:276] 0709 13:19:51.542318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.542341 (+1000023us) service_pool.cc:224] Handling call
0709 13:19:52.549140 (+  6799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.549377 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.549435 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.549523 (+    88us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:52.549567 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.549637 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.549769 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:52.587123 14287 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.024s	sys 0.054s
I20260709 13:19:52.623780  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:52.550606 14377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.021s	sys 0.035s
W20260709 13:19:52.709731 14377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.022s	sys 0.035s
I20260709 13:19:52.519158 14476 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:52.546502 14471 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:52.710642  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.710806 14471 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.712083  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.713332  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.713541 14471 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:52.610558 14429 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.017s	sys 0.019s
W20260709 13:19:52.636337  7236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46666 (request call id 1938) took 1744 ms (client timeout 1567 ms). Trace:
W20260709 13:19:52.714232  7236 rpcz_store.cc:269] 0709 13:19:50.894036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.894195 (+   159us) service_pool.cc:224] Handling call
0709 13:19:52.636121 (+1741926us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54006a13d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:52.636270 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968670592}
I20260709 13:19:52.593185 14321 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.714851 14476 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.648376 14406 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.649921 14421 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:52.715896  6591 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:52.611467 14483 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.717885  7229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1876) took 1185 ms. Trace:
I20260709 13:19:52.623705 14470 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:52.584335 14288 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.719367  6596 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:52.719208 14288 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.721302 14288 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.656729 14166 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:52.657351 14255 consensus_meta.cc:287] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.531s	user 0.004s	sys 0.004s
I20260709 13:19:52.723237 14255 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.723774  6636 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:52.724695  6514 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:52.725485 14255 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.660337 14317 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 28 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:52.725411  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48592 (request call id 3216) took 1058 ms. Trace:
I20260709 13:19:52.606323 14438 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.662860 14485 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.667824  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.697404  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.726919  6514 rpcz_store.cc:276] 0709 13:19:51.666678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.724164 (+1057486us) service_pool.cc:224] Handling call
0709 13:19:52.724482 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.725142 (+   660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.725348 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.718580 14483 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.718320  7229 rpcz_store.cc:276] 0709 13:19:51.532923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.552727 (+ 19804us) service_pool.cc:224] Handling call
0709 13:19:51.553026 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.594463 (+ 41437us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:51.814777 (+220314us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.402457 (+587680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.402518 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:52.402580 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.717592 (+315012us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b54006a3bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:52.717663 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.717816 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":995,"prepare.queue_time_us":368486,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":8976,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":27278,"spinlock_wait_cycles":94711040,"thread_start_us":775599,"threads_started":2}
I20260709 13:19:52.738252  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:52.746187  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.747731  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.739008  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.665733 14460 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:52.735322  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:52.749042  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 51742.4us
I20260709 13:19:52.775388  6595 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:52.775442  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.718835 14476 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:52.777076  6521 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:52.784821  6521 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1600) took 1051 ms. Trace:
I20260709 13:19:52.696810 14497 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:19:52.804080  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:52.734414 14487 consensus_meta.cc:287] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.590s	user 0.005s	sys 0.005s
I20260709 13:19:52.779583  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.780889  6594 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:52.782347  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.839677  6588 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:19:52.703632 14384 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:52.785274  6521 rpcz_store.cc:276] 0709 13:19:51.733729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.776185 (+1042456us) service_pool.cc:224] Handling call
0709 13:19:52.776900 (+   715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.784562 (+  7662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.784754 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.644129 14472 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:52.726913 14334 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.745653 14471 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.619535 14449 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.773185 14255 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:52.855337  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 311504us
W20260709 13:19:52.859515  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 80006.3us
I20260709 13:19:52.859858  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:52.862286  6563 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.865195  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:52.865944  6593 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:19:52.727548 14376 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:52.870730  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 147012us
I20260709 13:19:52.818512 14483 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:52.893714  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:52.816613 14451 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:52.834101 14479 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:52.845003 14384 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.857694 14485 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:52.898828  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:52.867324 14470 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:52.696177 14484 consensus_queue.cc:260] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:52.942464  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 81180us
I20260709 13:19:52.842227 14487 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.931816 14384 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:52.947613  6518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1596) took 1291 ms. Trace:
I20260709 13:19:52.930923 14497 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:52.857329 14334 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:52.960752  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62241.4us
I20260709 13:19:52.953418  6563 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:52.838802 14430 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:52.977533  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.018908  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 125356us
I20260709 13:19:52.889287 14255 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.865674 14471 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:52.939694 14483 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:52.954840 14484 raft_consensus.cc:3060] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:52.954334 14317 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:53.039587  6561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.72.131:38019 timed out after 1.881s (SENT) [suppressed 2 similar messages]
W20260709 13:19:53.047322 14503 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:19:53.044696 14430 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:53.048077 14449 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:53.048803 14494 log.cc:927] Time spent T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 13:19:53.053429 14463 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.913s	user 0.010s	sys 0.000s
I20260709 13:19:53.025455 14383 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:53.054198 14383 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:53.055161 14514 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:19:53.025131 14470 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:53.056489 14383 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.027315 14272 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:53.057339 14272 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:53.027181 14501 log.cc:927] Time spent T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.401s	user 0.001s	sys 0.000s
I20260709 13:19:53.027369 14521 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.006371 14485 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
W20260709 13:19:53.066562  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 113376us
I20260709 13:19:53.022289 14102 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.125337  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:53.055282 14520 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:53.054349 14030 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:53.120146 14499 log.cc:927] Time spent T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:19:53.120656 14383 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:53.127441 14379 consensus_meta.cc:287] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.000s
I20260709 13:19:53.010193 14497 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.053884 14463 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:53.143889  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 122170us
W20260709 13:19:53.146886  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 107330us
I20260709 13:19:53.132112 14520 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.132476 14438 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:53.140717 14505 log.cc:927] Time spent T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
W20260709 13:19:53.154174  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 170226us
I20260709 13:19:53.154523  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.172130  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:53.152055  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:53.158818 14334 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:53.159248 14386 consensus_meta.cc:287] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.004s
I20260709 13:19:53.159461 14412 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.160936 14376 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.134289 14474 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:53.163376 14506 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:53.164264  6515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59988 (request call id 1597) took 1506 ms. Trace:
W20260709 13:19:53.167814 14235 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.004s
I20260709 13:19:51.837850  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:53.150414 14372 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:53.150822 14287 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:53.183825  6515 rpcz_store.cc:276] 0709 13:19:51.657711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.767648 (+109937us) service_pool.cc:224] Handling call
0709 13:19:53.164041 (+1396393us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b54003f7fd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:53.164190 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38388096}
I20260709 13:19:53.172803  6671 raft_consensus.cc:3060] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:53.181406  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.173015  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.173447  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.174645  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.188031  6666 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:53.142911 14379 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:19:53.120113 14521 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.169476 14520 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:53.183269  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.155536  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.178148 14386 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:53.177565 14509 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.190047 14377 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:53.191547  6520 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1595) took 1536 ms. Trace:
I20260709 13:19:53.183173  6669 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 28 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 27. Current term is 28. Ops: []
I20260709 13:19:53.196215  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:53.210700  6520 rpcz_store.cc:276] 0709 13:19:51.654943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.696145 (+1041202us) service_pool.cc:224] Handling call
0709 13:19:52.705051 (+  8906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.937910 (+232859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.937955 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.938008 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:52.938050 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.191254 (+253204us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540019b9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:53.191322 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.191474 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473433344}
I20260709 13:19:53.197629  6670 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:53.183543  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:53.192267  6589 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 23.
I20260709 13:19:53.200109  6593 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 13:19:53.170269  6561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.202347  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.182839 14472 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.203642 14471 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.184511 14483 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.200418 14487 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.180673 14334 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.238945 14334 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:53.240741 14527 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:53.241218 14334 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.243543 14272 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:53.189785 14353 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 1.260s	user 0.012s	sys 0.000s
I20260709 13:19:53.245602  6516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60540 (request call id 3031) took 1623 ms. Trace:
I20260709 13:19:53.245422 14353 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.246009  6516 rpcz_store.cc:276] 0709 13:19:51.622431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.601917 (+979486us) service_pool.cc:224] Handling call
0709 13:19:52.602162 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.936960 (+334798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.937006 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.937061 (+    55us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:52.937115 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.245346 (+308231us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54003f6bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:53.245406 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.245542 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769438592}
I20260709 13:19:53.183099 14529 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:53.249873  6596 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:53.144909 14383 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.266984  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.268424  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.268573 14383 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.220929  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.224040  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.196316 14287 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.269548  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:53.269680  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 48903.1us
I20260709 13:19:53.269843  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.270224  6596 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:53.270587  6590 raft_consensus.cc:2393] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:53.270797 14287 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.271057  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.271395  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.272578  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.273625  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.217384 14509 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.206928 14528 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:53.217779 14377 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:53.250553  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.256006 14386 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.256317 14531 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:53.258131 14521 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.216949  6671 raft_consensus.cc:2468] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
W20260709 13:19:53.259057 14504 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:19:53.259586 14272 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.196014 14372 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.301854 14372 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.264014 14497 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.264526 14353 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.265300  7231 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:19:53.273555  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.198215 14317 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:53.304430 14317 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.184226 14485 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.275451  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.305864  6590 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:19:53.306823  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:53.307340  6590 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:19:53.307538 14334 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.308244 14334 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:53.309478  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.232050 14463 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.279989  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.280717  6516 raft_consensus.cc:1240] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:19:53.282435  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:53.285125  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.285872  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.317361  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.290845  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:53.276316 14530 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:19:53.299548 14272 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.217703 14532 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:53.324180  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.329350 14532 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.286393 14472 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:53.286545 14487 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:53.285169 14531 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:53.331624  6519 raft_consensus.cc:2410] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:53.286881 14414 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:53.335378  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.271872 14483 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:53.332186 14324 consensus_meta.cc:287] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.001s
I20260709 13:19:53.337486  6596 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:53.336135  6590 raft_consensus.cc:2468] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 28.
I20260709 13:19:53.333657  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:53.330104 14463 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:53.329784 14532 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:53.286820 14386 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:53.318560 14384 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.366163 14384 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.344157 14414 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.366978  6595 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 24: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:53.360925  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:53.361968  6590 raft_consensus.cc:1240] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:53.368162  6596 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:53.368920 14353 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.347576 14471 consensus_queue.cc:1059] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:19:53.369433 14353 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.370971 14521 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:53.371389 14521 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:53.371663 14353 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.372551 14521 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.373890 14521 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:53.376351  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:53.342943 14334 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.384862 14531 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:53.385522 14460 consensus_meta.cc:287] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
W20260709 13:19:53.385466  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 51882.4us
I20260709 13:19:53.407984  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:53.371335 14384 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.410673  6592 raft_consensus.cc:2468] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:53.358026 14324 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:53.393939  7234 raft_consensus.cc:1240] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:53.397460  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.420320  7228 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:53.399857  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.400529  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:53.403234  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 107938us
I20260709 13:19:53.361384 14523 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:53.398936  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:53.412348  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.412770  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.432636  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:53.433353  6515 raft_consensus.cc:3060] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:53.415167  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:53.422391  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.376945 14463 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.438216  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.387301 14538 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:53.388324 14372 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:53.383208 14353 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:53.430536  6665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45444 (request call id 2832) took 2113 ms (client timeout 1881 ms). Trace:
I20260709 13:19:53.424551  7229 raft_consensus.cc:2410] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:53.439800  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.440603  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.441630  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.450490  6662 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:53.421710 14539 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:53.443382  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:53.454597  6664 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.410127 14237 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:53.377568 14536 consensus_queue.cc:1059] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
W20260709 13:19:53.449115  6665 rpcz_store.cc:269] 0709 13:19:51.311703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.312092 (+   389us) service_pool.cc:224] Handling call
0709 13:19:53.413168 (+2101076us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540011dfd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:53.424891 (+ 11723us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2981910912}
I20260709 13:19:53.442482  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:53.426384 14495 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:53.453172  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:53.458642  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.459120  6669 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:53.413921 14460 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.404654 14520 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.434237 14272 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.457345  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:53.459865  6667 raft_consensus.cc:2368] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:19:53.425899 14383 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:19:53.450568  6670 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:53.454427 14539 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.428417 14384 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.468175  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.469282 14539 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:53.453269 14353 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.476648 14353 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.460246 14287 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:53.481420 14384 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.452032 14372 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.483081 14377 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.484480 14372 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.486315 14460 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.486378  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.487238  7232 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 24: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:53.482473 14386 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.490820 14509 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.491349  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.496518  6515 raft_consensus.cc:2468] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:19:53.497766  6669 raft_consensus.cc:2468] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:53.499404  6597 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:19:53.503219 14539 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.503898 14463 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.449990 14538 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.504850  6664 raft_consensus.cc:2468] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 17.
I20260709 13:19:53.505191 14539 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.506665 14538 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.508917 14483 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.511404 14520 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.512396  6671 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:53.486864  7235 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:53.720021 14461 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.720630 14461 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.723006 14463 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.723547 14463 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.725517 14463 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.732899 14532 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:53.741442 14268 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.734804 14539 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.743470  7235 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
I20260709 13:19:53.744877  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
W20260709 13:19:53.745913  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.736639 14461 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.745615 14539 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.756630  6514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 1599) took 2041 ms. Trace:
I20260709 13:19:53.757052  6514 rpcz_store.cc:276] 0709 13:19:51.715532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.738423 (+1022891us) service_pool.cc:224] Handling call
0709 13:19:52.738736 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.787637 (+ 48901us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:52.844747 (+ 57110us) log.cc:844] Serialized 14 byte log entry
0709 13:19:53.016233 (+171486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:53.016312 (+    79us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:53.016373 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.756420 (+740047us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b54003a75d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:53.756465 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.756578 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":69221,"prepare.run_cpu_time_us":145,"prepare.run_wall_time_us":4651,"raft.queue_time_us":4333,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":4453,"spinlock_wait_cycles":111827328,"thread_start_us":171027,"threads_started":1,"wal-append.queue_time_us":310554}
W20260709 13:19:53.760214  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 345086us
I20260709 13:19:53.762012  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:53.763257  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:53.764617  7200 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:53.765165 14255 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.765322 14268 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:53.766180  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:53.766389 14268 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:53.766963 14549 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:53.765579 14551 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:53.769982  7231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46680 (request call id 2165) took 1582 ms. Trace:
I20260709 13:19:53.774549 14549 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:53.776067  7200 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:53.777386 14549 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:53.777781 14549 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.778100 14549 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:53.770361  7231 rpcz_store.cc:276] 0709 13:19:52.187863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.196559 (+  8696us) service_pool.cc:224] Handling call
0709 13:19:52.196903 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.303451 (+1106548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.769739 (+466288us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000ba754. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:53.769927 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456277376}
I20260709 13:19:53.781939 14532 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:53.784555 14272 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:53.785084 14272 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:53.787400 14334 raft_consensus.cc:3055] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:53.787878 14334 raft_consensus.cc:740] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:53.788825 14334 consensus_queue.cc:260] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:53.789978 14334 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:53.792858  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.797451 14539 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.798159 14413 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.798671 14413 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:53.800868  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 316465us
I20260709 13:19:53.802717  6563 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.803885  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.805353  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.807912  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.808706  7231 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:53.804667  7232 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:53.810544  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:53.786095 14272 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.816172  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.815935  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.817543  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:53.819581  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.818554 14272 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:53.823276 14411 consensus_queue.cc:1059] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:53.824447  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:53.825222  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.825551 14436 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:53.825784  7232 raft_consensus.cc:2468] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:53.828723 14523 raft_consensus.cc:2804] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:53.816802 14413 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:53.829241 14523 raft_consensus.cc:697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:53.830963  7237 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 24: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:53.831477  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.836103  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.833537  6562 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:53.841859  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:53.816725 14516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.008s	sys 0.014s
I20260709 13:19:53.847399  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.847692 14479 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 13:19:53.851899  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:53.852175 14461 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:53.852571  7229 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:53.850369  6488 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.863817  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:53.840798 14384 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:53.848829 14516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.008s	sys 0.015s
I20260709 13:19:53.848076 14474 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:53.870517 14474 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:53.870934 14448 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:53.871546 14448 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:53.871755 14474 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.872486 14448 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:53.865047 14290 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:53.847602 14433 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:53.846710 14523 consensus_queue.cc:237] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.866183 14384 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:53.881352 14384 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:53.885149  6488 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.887212  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.888427 14290 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:53.890929 14519 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.723s	user 0.003s	sys 0.000s
I20260709 13:19:53.893659  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.894759 14290 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:53.895160 14290 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:53.895537 14290 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:53.898962  6597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55990 (request call id 3263) took 1374 ms (client timeout 1785 ms). Trace:
W20260709 13:19:53.899386  6597 rpcz_store.cc:269] 0709 13:19:52.524742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.524864 (+   122us) service_pool.cc:224] Handling call
0709 13:19:53.898778 (+1373914us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b54003bc4d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:53.898899 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2021746048}
I20260709 13:19:53.904147  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:53.905810 14376 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:53.906236 14376 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.906610 14376 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:52.948117  6518 rpcz_store.cc:276] 0709 13:19:51.655979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.722068 (+1066089us) service_pool.cc:224] Handling call
0709 13:19:52.722440 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.727635 (+  5195us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:52.727857 (+   222us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.944665 (+216808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.947254 (+  2589us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:52.947311 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.947398 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.947542 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43696,"prepare.queue_time_us":13519,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":2285,"thread_start_us":216405,"threads_started":1}
I20260709 13:19:53.907644 14549 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:53.913677  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 1677) took 1249 ms. Trace:
I20260709 13:19:53.914678  6561 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:53.914975  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.915805  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:53.915916  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.006834  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.008050  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.007424  6411 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 9 to 19, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 19 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:54.008399  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:54.009058  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.009179 14411 raft_consensus.cc:2804] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:54.009651 14411 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:54.010083 14411 raft_consensus.cc:3060] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.010145  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:53.955236  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:54.012442  6664 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:54.012621  6517 raft_consensus.cc:2393] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 24: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:54.012825  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:54.013334  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:54.013550  6670 raft_consensus.cc:2393] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:54.014148  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:54.014518  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.014711  6669 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:54.015111  6665 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:54.015556  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.016181  6519 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:54.017117  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.020274  6519 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:54.021492  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 105755us
I20260709 13:19:54.022481  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.023317  6588 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:19:54.023770 14554 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:54.024403 14554 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:54.024750 14554 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:54.024922  6588 raft_consensus.cc:1240] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 27. Current term is 28. Ops: [25.14-27.15]
I20260709 13:19:54.014430  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:53.957499  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 59937.2us
I20260709 13:19:53.984419 14019 raft_consensus.cc:3055] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:54.027756  6664 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:53.984805 14414 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:53.985152 14376 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:53.990855  6595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1683) took 1240 ms (client timeout 1582 ms). Trace:
W20260709 13:19:54.040794  6595 rpcz_store.cc:269] 0709 13:19:52.750282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.803736 (+ 53454us) service_pool.cc:224] Handling call
0709 13:19:53.963151 (+1159415us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540019d7d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:53.990788 (+ 27637us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222719744}
I20260709 13:19:53.990945  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:53.991667 14487 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:53.991701  6520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.992543  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.990244  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:53.993736  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.993810  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:53.997560  7233 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:54.034605  6669 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
I20260709 13:19:53.999861 14459 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:54.023730  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.029294  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:53.914101  6594 rpcz_store.cc:276] 0709 13:19:52.664688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.852344 (+187656us) service_pool.cc:224] Handling call
0709 13:19:52.852680 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.140526 (+287846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.140558 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:53.140601 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:53.140636 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.913455 (+772819us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54002f7bd4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:53.913506 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.913622 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497468800}
I20260709 13:19:53.952198 14549 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:54.009352  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:54.029047  6561 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:54.039130  6593 raft_consensus.cc:1240] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:53.904663 14290 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.059674  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.009816  6411 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 12 to 17, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 17 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:54.043401  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.045282  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.038951 14019 raft_consensus.cc:740] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:54.067211  6517 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 24: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:54.064062  6637 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:54.050575  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:54.056016  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 46747.5us
I20260709 13:19:54.068707  6664 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:54.069147  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:54.069600  6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:54.070802  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:54.056649  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:54.071200 14414 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:54.071727 14414 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:54.072114 14414 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.072320  6561 leader_election.cc:400] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:19:54.072808 14376 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.042104  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.073165  6561 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:54.071086  6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.058604  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:54.075218  6561 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:54.076069 14472 raft_consensus.cc:2804] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:54.076436 14479 raft_consensus.cc:2804] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:54.076725 14472 raft_consensus.cc:697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:54.077034 14479 raft_consensus.cc:697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:54.077775 14472 consensus_queue.cc:237] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.081601 14411 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.058367 14324 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:54.083179  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.051613 14255 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:54.083976 14255 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:54.084151 14413 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:54.084836 14290 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.084883 14413 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:54.085021 14255 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.079623  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:54.058478 14448 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:54.087996  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:54.078012 14479 consensus_queue.cc:237] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.092161  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.059795 14459 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.086380 14255 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:54.093307  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:54.094385  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.107335  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.109995  6486 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:54.110710  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.104828 14290 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:54.111719  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.099381  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.101069  6595 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:54.104297  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.060882 14471 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:54.069502  6637 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:19:54.115074  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 45678.2us
I20260709 13:19:54.116335  6666 raft_consensus.cc:2410] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:19:54.120101  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.123144  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.123769  6596 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:54.124986  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.125721  6590 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.113787  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:54.103838 14470 raft_consensus.cc:3060] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:54.141494  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.143858  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.144269  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.144871  6595 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:54.145812  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.135198  6590 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:54.147657  6589 raft_consensus.cc:1240] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:54.148931  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.145979  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:54.150758  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.073081 14384 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.153179  6595 raft_consensus.cc:2410] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:19:54.154541  6596 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:19:54.156498  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.116010 14290 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.116703 14401 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:54.162389  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.163257  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.152738  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:54.165593  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:54.171301  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.126780 14334 raft_consensus.cc:2804] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:19:54.128188 14376 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.179394  6595 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 24: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:54.162106 14516 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:54.189774 14334 raft_consensus.cc:697] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 28 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:54.195605 14552 consensus_queue.cc:1059] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:54.199048  6595 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:54.138586 14413 consensus_queue.cc:1059] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:19:54.203083  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:54.219322  6419 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 335) took 24680 ms (client timeout 15000 ms). Trace:
I20260709 13:19:54.159541 14506 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:54.200588  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.175351 14290 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.185581 14448 consensus_queue.cc:1059] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:19:54.182014 14401 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:54.249145  6590 raft_consensus.cc:2376] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18. Re-sending same reply.
W20260709 13:19:54.249755  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46754.3us
I20260709 13:19:54.251940 14334 consensus_queue.cc:237] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.189006 14538 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:54.265384  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.232945 14516 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:54.342140  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 76872.9us
I20260709 13:19:54.256855 14422 raft_consensus.cc:3055] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:19:54.253007 14412 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:54.252825 14552 consensus_queue.cc:1243] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:19:54.355613  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.155222 14019 consensus_queue.cc:260] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:54.245280  6419 rpcz_store.cc:269] 0709 13:19:29.626859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.645888 (+ 19029us) service_pool.cc:224] Handling call
0709 13:19:54.219219 (+24573331us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:46.409968 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:46.437763 (+ 27795us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:46.437831 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:19:46.437857 (+    26us) tablet.cc:662] Decoding operations
0709 13:19:46.439614 (+  1757us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:46.439703 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:46.439731 (+    28us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:19:46.441390 (+  1659us) tablet.cc:701] Row locks acquired
0709 13:19:46.441437 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:19:46.441615 (+   178us) write_op.cc:270] Start()
0709 13:19:46.441764 (+   149us) write_op.cc:276] Timestamp: P: 1783603186441555 usec, L: 0
0709 13:19:46.441803 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:19:46.484259 (+ 42456us) log.cc:844] Serialized 8564 byte log entry
0709 13:19:47.040965 (+556706us) op_driver.cc:464] REPLICATION: finished
0709 13:19:47.194977 (+154012us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04bcab7ef0
0709 13:19:47.392690 (+197713us) write_op.cc:301] APPLY: starting
0709 13:19:47.392857 (+   167us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:48.290715 (+897858us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:48.330728 (+ 40013us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:49.129981 (+799253us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:49.130439 (+   458us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 13:19:49.130511 (+    72us) write_op.cc:312] APPLY: finished
0709 13:19:49.158330 (+ 27819us) log.cc:844] Serialized 196 byte log entry
0709 13:19:49.252029 (+ 93699us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:49.252247 (+   218us) write_op.cc:454] Released schema lock
0709 13:19:49.252354 (+   107us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:49.253008 (+   654us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:49.253052 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":204432,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":329631,"apply.run_cpu_time_us":42865,"apply.run_wall_time_us":1860397,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96043,"num_ops":22,"prepare.queue_time_us":329766,"prepare.run_cpu_time_us":31701,"prepare.run_wall_time_us":244501,"raft.queue_time_us":217561,"raft.run_cpu_time_us":17316,"raft.run_wall_time_us":261487,"replication_time_us":598972,"spinlock_wait_cycles":25392896,"thread_start_us":888912,"threads_started":5,"wal-append.queue_time_us":273490,"wal-append.run_cpu_time_us":30840,"wal-append.run_wall_time_us":2405111}]]}
I20260709 13:19:54.270515 14553 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:54.362743  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 248991us
W20260709 13:19:54.342273 14557 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:19:54.268178 14401 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:54.360589  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:54.373766  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 219205us
I20260709 13:19:54.374416  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.341836 14538 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.375720 14538 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:54.376092  6488 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:54.378823 14535 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:54.379774 14535 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:54.384537  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.350060 14516 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:54.350932 14520 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:54.366909  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.393646  6597 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:54.369303  6590 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:54.256076 14506 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.355455 14483 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:54.354548 14422 raft_consensus.cc:740] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:54.389879  6593 raft_consensus.cc:1240] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:54.397357 14413 raft_consensus.cc:3055] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:19:54.364446  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 213553us
I20260709 13:19:54.423198  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:54.364588 14552 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:54.424657  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:54.427289  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.432240  6597 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 24.
I20260709 13:19:54.433068  7236 raft_consensus.cc:2393] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 24: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:54.364588 14454 consensus_queue.cc:1175] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "53fee9894731461a821a131be24dbbca" responder_term: 19 status { last_received { term: 19 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 19 index: 11 } }
I20260709 13:19:54.372943 14411 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.404548 14237 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:54.374593 14553 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.466094 14376 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:54.467700 14376 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:54.454929 14411 consensus_queue.cc:1059] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:19:54.459235 14560 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:54.459707 14422 consensus_queue.cc:260] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.460428 14290 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.463518 14384 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.424134 14483 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.425843 14413 raft_consensus.cc:740] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:54.432196 14459 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:54.472898  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 361951us
I20260709 13:19:54.452699 14568 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:54.466842 14553 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.463061 14506 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.499082  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:54.470978 14414 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.512176  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:54.481352 14376 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:54.507606 14422 raft_consensus.cc:3060] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:19:54.482807 14436 consensus_queue.cc:1175] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ceee54e49d041be9f7f48a2052709c5" responder_term: 28 status { last_received { term: 23 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 27. Current term is 28. Ops: [25.14-27.15]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:19:54.543299 14573 log.cc:927] Time spent T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:54.504951 14533 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:54.516146 14459 raft_consensus.cc:3055] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:54.560302 14463 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:54.558588 14459 raft_consensus.cc:740] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:54.262989  6411 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 15 to 18, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 18 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:19:54.653923  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.665623  6518 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:54.668953  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 294444us
W20260709 13:19:54.674839  6562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.677870  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 250468us
I20260709 13:19:54.680037  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.683202  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.687264  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.681267  6411 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:54.670444  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.565395 14272 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:54.699638 14272 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:54.700191 14272 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:54.706192  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.706030  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.709836  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.710740  6665 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:54.716784 14290 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.560425 14376 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.559958 14413 consensus_queue.cc:260] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.721028 14413 raft_consensus.cc:3060] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:54.724485 14459 consensus_queue.cc:260] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.725427 14414 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.725972 14568 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:54.727350 14568 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:54.725080 14506 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:54.754206  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 67148.4us
I20260709 13:19:54.757031 14459 raft_consensus.cc:3060] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:54.697606  6411 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 12 to 17, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 17 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:54.754562 14567 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:54.791039 14272 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.793844 14272 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:54.798920  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.801759  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:54.808214 14235 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.809223 14235 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:54.795672 14554 consensus_meta.cc:287] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.770s	user 0.007s	sys 0.005s
I20260709 13:19:54.816694 14272 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:54.822467 14554 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.843077 14483 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:54.818789 14567 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:54.820818 14580 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:54.845933 14580 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.839764  6411 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:54.859748  6671 raft_consensus.cc:3055] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:54.863874 14531 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:54.849423 14580 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:54.872023  6411 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 17 to 18, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 18 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:54.884873 14237 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:54.867573  6671 raft_consensus.cc:740] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:54.892957  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.890399 14554 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:54.894471 14237 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:54.898191  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 388786us
I20260709 13:19:54.915006  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:54.913904  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 459143us
I20260709 13:19:54.920073 14553 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:54.920475  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.921293  6665 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:19:54.923439  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 121963us
I20260709 13:19:54.936123  6671 consensus_queue.cc:260] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.921099 14553 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.939786  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:54.937024 14470 consensus_meta.cc:287] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.735s	user 0.006s	sys 0.012s
I20260709 13:19:54.942598 14412 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:54.943807  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.942505  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:54.951866  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.954348  6671 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:54.956017 14563 log.cc:927] Time spent T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.396s	user 0.005s	sys 0.000s
I20260709 13:19:54.959765 14474 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:54.960609 14474 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:54.962878  6665 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:54.944454 14235 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:54.967854 14533 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:54.968386 14533 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:54.969507  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.974109 14019 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.975510 14268 consensus_queue.cc:1059] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:54.982688 14551 raft_consensus.cc:3055] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:54.983155 14551 raft_consensus.cc:740] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:54.983129  6411 catalog_manager.cc:5697] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 13 to 17, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 17 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:19:54.984524 14551 consensus_queue.cc:260] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:54.986075 14551 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.987480  6665 raft_consensus.cc:1275] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:19:54.990883  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 478617us
I20260709 13:19:54.994540  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:54.996198  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:54.998309 14334 consensus_queue.cc:1059] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
I20260709 13:19:54.999171  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:54.999850 14412 raft_consensus.cc:3055] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:55.000279 14412 raft_consensus.cc:740] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:55.001026  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.004135 14412 consensus_queue.cc:260] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.006045 14412 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:55.004788 14474 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:55.006255  6696 heartbeater.cc:646] Failed to heartbeat to 127.5.72.190:46511 (6 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.72.190:46511 timed out after 15.000s (SENT)
W20260709 13:19:55.013203 14401 consensus_meta.cc:287] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.014s
I20260709 13:19:55.014914 14533 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.016642  6671 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:19:55.021521 14536 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:19:55.024008  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.028861  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.027679 14237 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.030828  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.028882  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.031725 14237 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.032264  6593 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:55.033420  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.034257  7233 raft_consensus.cc:2368] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 23. Current term is 24.
I20260709 13:19:55.035143 14235 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:55.035243  7236 raft_consensus.cc:1240] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:55.036020 14235 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:55.038164  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:55.039286 14474 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:55.039706 14474 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:55.040057 14474 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:55.044490 14533 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.044654  7200 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:55.045706 14539 raft_consensus.cc:2804] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:55.046226 14539 raft_consensus.cc:697] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:55.047467 14539 consensus_queue.cc:237] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.051343  7237 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:55.052296  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.052392 14235 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:55.053421  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.055889  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.056797  6664 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 19: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:55.056804  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.058759  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.059026  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:54.952648 14470 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:55.061275  6513 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:19:55.067306  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:54.986044  6411 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 18 to 20, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.077059  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:55.075255  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:55.078132 14536 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:55.078753  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:55.018803 14401 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.078791 14536 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:55.079718 14334 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:55.081296  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:55.082324  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 108954us
I20260709 13:19:55.086445  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:55.066937  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.086500 14536 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.092316  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.092566  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.092795 14553 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.093307  6666 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:55.087288 14334 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:55.094305 14334 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:55.076709  6411 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.095978  6411 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 13 to 18, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 18 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.064886 14474 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.099606  7237 raft_consensus.cc:2393] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:55.100150  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:55.102775 14255 consensus_meta.cc:287] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: Time spent flushing consensus metadata: real 0.991s	user 0.019s	sys 0.003s
I20260709 13:19:55.135423  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.107664  6517 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:19:55.111605  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.113345 14237 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:55.135654  6487 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:55.114348  6591 raft_consensus.cc:2468] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:55.139317  6592 raft_consensus.cc:1275] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 13:19:55.153442  6594 raft_consensus.cc:2410] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:19:55.145829  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.132125  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:55.129304  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:55.126590  6519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:55.155615  6664 raft_consensus.cc:1275] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:19:55.155861  6519 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:55.134713  6670 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:19:55.122850  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:55.127507  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:55.123641  6522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:55.152680 14376 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:55.140413  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:55.158897 14376 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:55.118049  6411 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 11 to 16, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 16 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.159319 14413 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:55.128265 14538 consensus_meta.cc:287] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: Time spent flushing consensus metadata: real 0.752s	user 0.018s	sys 0.000s
I20260709 13:19:55.160171 14376 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.160892 14538 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.122188  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:55.162986  6513 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:55.163355 14538 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.115238  6662 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:55.167092  6411 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 19 to 20, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.172458 14506 consensus_queue.cc:1048] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:19:55.173656  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.160243  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 97994.2us
I20260709 13:19:55.175534  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.178767 14255 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:55.114603 14516 consensus_meta.cc:287] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.004s
I20260709 13:19:55.121351 14474 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.189409  6488 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:55.207264  6591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1688) took 1097 ms. Trace:
I20260709 13:19:55.207901  6591 rpcz_store.cc:276] 0709 13:19:54.110267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.110467 (+   200us) service_pool.cc:224] Handling call
0709 13:19:55.206973 (+1096506us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54001b6f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:55.207170 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289498368}
W20260709 13:19:55.220960  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 47836.8us
I20260709 13:19:55.222514  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:55.222843  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.226127  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:55.228797  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53916.4us
I20260709 13:19:55.197127 14235 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 13:19:55.201206 14516 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.237140 14413 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:55.243188  6563 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.243518 14413 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 24 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:55.243978  6597 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:55.244515  6563 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:55.244653 14413 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.245354  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.245916 14237 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:55.241076  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.246726 14237 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:55.247196 14237 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.247224  7233 raft_consensus.cc:1275] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:55.247665 14237 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:55.246471  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.249667  6514 raft_consensus.cc:1240] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:55.236994 14019 consensus_queue.cc:1059] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:19:55.256285  6563 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.255347  6514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:55.258002  6514 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:55.259454  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:55.268955 14476 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.263741 14474 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:55.271934  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.272815  6594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34536 (request call id 1690) took 1129 ms. Trace:
I20260709 13:19:55.273180  6594 rpcz_store.cc:276] 0709 13:19:54.143494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.143635 (+   141us) service_pool.cc:224] Handling call
0709 13:19:55.272600 (+1128965us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400080f54. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:55.272760 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2318723840}
I20260709 13:19:55.278102 14474 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:55.279436  6665 raft_consensus.cc:2468] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:55.279965  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.264219  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:55.278867  6669 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:55.282094 14472 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:55.282478  6667 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:55.284528 14459 raft_consensus.cc:3055] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:55.285724  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:55.292805  7236 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:55.305356 14580 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.305567 14554 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:55.306082 14554 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:55.306684 14554 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:55.306013 14580 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.310045  7229 raft_consensus.cc:1275] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 24 index: 19. (term mismatch)
I20260709 13:19:55.310405 14019 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:55.310549  7229 pending_rounds.cc:85] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 18
W20260709 13:19:55.312460  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 69852.8us
I20260709 13:19:55.290491 14459 raft_consensus.cc:740] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:55.319832  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.319766  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.320516 14568 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:55.320976 14595 log.cc:927] Time spent T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:19:55.311772  7229 pending_rounds.cc:107] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 22.19
I20260709 13:19:55.324450  7229 raft_consensus.cc:2889] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:55.329169 14459 consensus_queue.cc:260] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.332374 14568 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.368831 14568 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.371059 14551 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:55.341418 14459 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:55.371860 14551 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.343189  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.344722  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.347012  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.375402  6667 raft_consensus.cc:3060] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 24
W20260709 13:19:55.347811  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 62251.2us
I20260709 13:19:55.348088  6669 raft_consensus.cc:1240] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:55.346366  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.379421  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:55.379763  6670 raft_consensus.cc:2376] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19. Re-sending same reply.
I20260709 13:19:55.348477 14472 consensus_queue.cc:1048] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:55.380461 14019 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:55.380961 14019 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:55.381384 14019 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:55.350569 14580 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.350850  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.331952  6517 raft_consensus.cc:1275] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:55.315739 14334 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.385141  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:55.389472 14237 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:55.358775  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.291110  6411 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 13 to 17, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 17 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.314644 14401 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.401464 14472 consensus_queue.cc:1048] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:55.389865 14334 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.374289 14551 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.391161 14019 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.392117  6594 raft_consensus.cc:1240] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:55.392235  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:55.393337  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.394006  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:55.432878  7233 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:55.394992  7236 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:19:55.395013  6517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f"
I20260709 13:19:55.396488 14506 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:19:55.357162 14590 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:55.399595  6514 raft_consensus.cc:2468] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:55.401592 14436 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:55.439646  7232 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:55.439988  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:55.448187  7229 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:55.451613  6667 raft_consensus.cc:2468] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 24.
I20260709 13:19:55.455521  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.459371  6521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:55.460170  6518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "921c423406da4f289884afca51ca198f" is_pre_election: true
I20260709 13:19:55.470765  7233 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:55.474957  7200 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:55.468008  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:55.487248  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44218.3us
I20260709 13:19:55.473726 14568 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.489508  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.502192  6667 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:55.490713 14401 raft_consensus.cc:2804] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:55.492573 14235 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.500028 14580 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.499563 14533 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:55.526835 14485 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:55.527009 14586 consensus_queue.cc:1059] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:19:55.527669 14485 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.529475 14255 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:55.530128 14485 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.530135 14255 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.535463 14237 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.539409  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.540086 14235 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.541442 14436 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.542353 14536 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:55.555152 14506 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:55.537830 14554 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.588783 14485 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:55.593685  6636 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:55.594470 14235 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.598939  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:55.521037 14401 raft_consensus.cc:697] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:55.506955 14568 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.596292 14536 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:55.609005  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:55.610205  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 70936.5us
I20260709 13:19:55.609858 14536 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.611366 14536 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:55.612080 14586 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:55.613521 14461 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:55.613870  7229 raft_consensus.cc:1240] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:55.614125 14461 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.614727  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.608793 14268 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.621258  6593 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:55.624337  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.624941  6670 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:55.625906  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.626466  6667 raft_consensus.cc:3060] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:55.601877 14376 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.614444  7200 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:55.598356 14436 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.631788 14531 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:55.632380 14436 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.589036 14533 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.628371  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.616565 14334 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:55.667358  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 153957us
W20260709 13:19:55.669643  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 44356.3us
W20260709 13:19:55.674007  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 59275.4us
I20260709 13:19:55.509773  6411 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 9 to 21. New cstate: current_term: 21 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.684721  6597 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:19:55.684952  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:55.616753 14580 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 4
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:19:55.688350  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
I20260709 13:19:55.687707  6411 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.689682 14459 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:19:55.691322 14459 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:55.691840 14459 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:55.617983 14553 consensus_queue.cc:1048] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:19:55.695647 14459 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:55.697520 14598 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:19:55.624980 14485 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.700103 14568 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.700443 14485 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.617995 14506 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:55.704459 14601 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:19:55.710733 14609 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:55.711344 14609 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.713438 14609 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.659590 14461 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.715888  6667 raft_consensus.cc:2468] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:19:55.680835 14551 raft_consensus.cc:2804] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:55.701871 14599 log.cc:927] Time spent T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:19:55.669270 14268 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.661899 14376 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.689800  6411 catalog_manager.cc:5697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.727608 14551 raft_consensus.cc:697] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:55.719797 14334 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.731127 14235 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:55.751479 14268 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.752283 14613 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.752835 14613 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.753862 14334 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.754611 14255 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.730847 14553 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:19:55.771778  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.776913  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.781308  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 187683us
W20260709 13:19:55.781311  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 75630.6us
I20260709 13:19:55.782364  6636 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.782958 14376 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.731796 14401 consensus_queue.cc:237] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.783533  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.784595 14539 raft_consensus.cc:3055] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:55.785063 14539 raft_consensus.cc:740] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:55.787868 14586 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:55.788146 14620 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:55.792026  6590 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
W20260709 13:19:55.795027 14600 log.cc:927] Time spent T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:19:55.774103 14579 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:55.816922  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.796404 14235 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.798924 14551 consensus_queue.cc:237] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.800720 14516 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.802392  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.787981 14554 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.805387  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.807440  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:55.832113 14586 consensus_queue.cc:1048] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:19:55.808740 14580 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:55.833926 14579 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.808645  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:19:55.811213  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.811273  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.812849  6597 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:55.813252  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.840514  6593 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:55.813535 14533 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:55.813887 14613 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.815780  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:55.846448  6667 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:19:55.804711 14506 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.832140  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:55.835376  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.847944  6594 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 20: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:55.848047  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65802.1us
I20260709 13:19:55.832849  6662 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
W20260709 13:19:55.870725  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 62192.6us
W20260709 13:19:55.879369  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44185.1us
I20260709 13:19:55.867614  6589 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:55.882875  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:55.883915  6591 raft_consensus.cc:2368] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 23. Current term is 24.
I20260709 13:19:55.882468  7230 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
W20260709 13:19:55.904740  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 280427us
I20260709 13:19:55.905624  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:19:55.905997  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.888161 14268 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:55.909106  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.912174  6591 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:55.817081 14539 consensus_queue.cc:260] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.880481 14609 consensus_queue.cc:1059] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:19:55.915664  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:55.882965 14568 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:19:55.922257  6561 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:55.917990 14613 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:55.928645  6561 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:19:55.972229  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 66686.6us
W20260709 13:19:55.974035  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 80855.1us
I20260709 13:19:55.816892  6411 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 13 to 14, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 14 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.932138 14476 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:55.957079  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:55.973461  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:55.931988 14539 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:55.928833 14474 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:55.960639 14613 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.979033  6596 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:19:55.986135 14237 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:55.978718 14476 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.986687 14237 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.987011 14476 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:55.987176 14237 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:55.989182 14235 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:55.990734 14622 raft_consensus.cc:2804] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:55.991353 14622 raft_consensus.cc:697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:55.991379 14586 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.992151 14586 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:55.992388 14622 consensus_queue.cc:237] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:55.927074 14568 consensus_queue.cc:1243] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer 3ceee54e49d041be9f7f48a2052709c5 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 13:19:55.997982  6637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.003671  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:56.026190  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 98623.3us
I20260709 13:19:56.000959  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:55.998644  7198 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:55.982473 14474 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.998356 14613 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.044678  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:55.930449 14538 consensus_queue.cc:1059] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 13:19:55.992151 14019 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.039407 14609 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:56.050524 14609 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:56.041976  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62373us
I20260709 13:19:55.984980 14617 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.994503 14586 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:56.056022  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 76275.5us
W20260709 13:19:56.056710 14624 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:56.056833  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:56.057827  6592 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:56.058204  7230 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:56.059971 14617 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.063087 14617 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.065006  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.065203 14476 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.050949 14609 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:56.066711  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 217924us
I20260709 13:19:56.067801 14472 consensus_queue.cc:1048] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:19:56.042878 14580 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:56.065773  6588 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:56.072059  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.086122  6594 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:56.052685  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.074177  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.072609 14268 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:56.086906  6593 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:56.045723 14474 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:56.048221 14621 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:56.093958 14617 consensus_queue.cc:1059] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:56.095258 14617 raft_consensus.cc:3055] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:56.095727 14617 raft_consensus.cc:740] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.101481  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:56.106807 14019 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:19:56.108243 14019 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:56.108628 14019 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.109486 14019 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.088055 14580 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:56.111203 14580 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.112370 14461 raft_consensus.cc:3055] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:56.105001  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:56.115371  6636 leader_election.cc:400] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 23. Current term is 24.
I20260709 13:19:56.115703  6636 leader_election.cc:403] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:56.114521  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:56.080899 14476 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.083473  6562 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:56.084589  7198 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:56.065702  6591 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:56.086800  7236 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:56.117133 14619 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:56.118103 14619 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.118296 14633 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:56.118515 14619 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:56.119004 14633 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:56.120070 14633 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:56.120398 14633 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:56.121276 14633 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.124358 14633 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:56.126355  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:56.127241 14413 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:56.127655 14413 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:56.128055 14413 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:56.129127  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 97117.4us
I20260709 13:19:56.130805 14268 raft_consensus.cc:2764] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:19:56.131886  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47409us
I20260709 13:19:56.133263  7198 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:56.139169 14606 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:56.113029 14538 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:56.140443 14538 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.140791 14538 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.142429  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:56.142889 14629 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:56.144207 14627 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:19:56.088210 14579 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.146962  6411 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 13 to 14, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 14 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:56.148940 14568 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:56.149458  6411 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 16 to 20, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:56.151423  7198 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:56.150972 14459 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:56.112797 14461 raft_consensus.cc:740] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:19:56.152825  7200 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.166567  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.150799 14619 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.168236  6591 pending_rounds.cc:85] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 14
I20260709 13:19:56.179512  6591 pending_rounds.cc:107] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:19:56.179847  6591 raft_consensus.cc:2889] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:56.185999  6588 raft_consensus.cc:1240] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:56.187994  6594 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:56.192854 14413 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.116214 14609 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:56.184149  7200 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.206990  7197 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.178496 14019 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:56.208113  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65769.7us
I20260709 13:19:56.212873  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.214195  6637 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:56.180789 14459 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:56.211233  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:56.218286  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.158564 14506 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:56.186447 14235 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.205533 14568 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:56.226927  6637 leader_election.cc:400] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 23. Current term is 24.
I20260709 13:19:56.227322  6637 leader_election.cc:403] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:19:56.205204 14485 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:56.231282  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.235553  6670 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:56.212076 14620 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.216403 14334 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 23. Current term is 24.
I20260709 13:19:56.219496 14459 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.223057 14516 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:56.245492  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:56.249332  6593 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:56.224992 14568 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:19:56.251271  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 99987us
W20260709 13:19:56.256300  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 168696us
I20260709 13:19:56.184830 14617 consensus_queue.cc:260] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.257925 14334 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977) with higher term. Message: Invalid argument: T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 23. Current term is 24.
I20260709 13:19:56.258476 14617 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:56.232059 14506 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.244678 14485 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.266884 14461 consensus_queue.cc:260] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.268232 14461 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:56.268527 14485 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.245839 14620 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.272300 14629 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.272550 14620 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:56.272794 14629 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:56.264292 14586 raft_consensus.cc:3055] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:56.279750 14586 raft_consensus.cc:740] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:56.245971 14235 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.280725 14586 consensus_queue.cc:260] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.282158 14586 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:56.283371 14639 log.cc:927] Time spent T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:19:56.284466 14506 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:56.286672  6563 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.195501  6411 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:56.289583  6589 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:56.294338  6670 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:19:56.255873 14568 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:56.306363  6588 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:19:56.319551  6597 raft_consensus.cc:1240] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:56.243975 14610 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:19:56.299463 14413 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.337126  6593 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:19:56.325654  7199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.302661 14474 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.384864  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.385648  7229 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:56.385840  6562 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:56.386976  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:56.387707 14622 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:56.388228  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:56.395099  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.400957 14619 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.412289 14237 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.414927  6594 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:56.415654  6636 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.417361 14506 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:56.417547  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.418414 14533 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:56.431104  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:56.434748  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
W20260709 13:19:56.435110  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.435424  6589 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:56.435513  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 130942us
W20260709 13:19:56.436131  7200 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.438017 14579 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:56.438397 14579 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.439266 14579 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.440039 14619 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:56.440438 14579 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.441422 14619 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:56.442163 14436 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:56.443572 14619 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.444537 14619 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.445494 14609 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.449944 14617 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.450510 14617 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.452936 14235 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.455513 14617 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.456719  6589 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
W20260709 13:19:56.459357  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.461001 14538 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.462296  7233 raft_consensus.cc:2368] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21.
I20260709 13:19:56.467084 14459 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.469432 14459 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.482291  6592 raft_consensus.cc:1240] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:56.486474 14474 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.476500 14613 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.493268 14613 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.479871  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:56.498544 14617 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:56.499182  6588 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.479581  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.478605  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:56.504451  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 67791.1us
I20260709 13:19:56.507871  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:56.508692  6597 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.511165  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:56.513495  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 93701.7us
I20260709 13:19:56.517135 14436 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:56.518311 14413 consensus_queue.cc:1059] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:56.519271 14413 raft_consensus.cc:3055] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:56.519629 14413 raft_consensus.cc:740] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.520061  7230 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:19:56.520433  7230 pending_rounds.cc:85] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Aborting all ops after (but not including) 16
I20260709 13:19:56.520675  7230 pending_rounds.cc:107] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:19:56.520953  7230 raft_consensus.cc:2889] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:56.520553 14413 consensus_queue.cc:260] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.521798 14413 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:56.527019  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.529567 14436 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.530664 14436 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:56.534803  6563 leader_election.cc:336] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.537976  7233 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:56.538340  6563 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.538851  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:56.539217  6563 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.539537  7230 raft_consensus.cc:2393] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:56.540035  6563 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:56.540714  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.541258  6563 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.541903  6563 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.546149 14580 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.547302  6596 raft_consensus.cc:1240] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:56.548250 14235 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:19:56.549666  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.550331  6563 leader_election.cc:336] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:56.555007  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.558601  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.559418 14646 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.560020 14646 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.562242 14646 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.534281 14237 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.465163 14538 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.583088  6597 raft_consensus.cc:2468] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:56.592000  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:56.595045  6588 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:56.598668  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.603214  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:56.603972  7230 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:56.607105  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:56.608522  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44360.7us
W20260709 13:19:56.610944  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.612841  6563 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.607760  7237 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
W20260709 13:19:56.613488  6563 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.356030 14516 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:56.609761  6488 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:56.614099  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.620170  6563 leader_election.cc:336] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.596935 14646 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.608567 14019 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.612500 14609 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.622769  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:56.627907  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.612775 14268 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:56.641824 14268 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.644279 14268 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.622062 14459 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:56.619297 14617 consensus_queue.cc:1059] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:19:56.600973 14568 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.605005 14629 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.671731  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:56.645841 14609 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.628288 14646 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.676565 14609 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.677508 14646 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:56.668531  6636 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:56.680892 14579 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:56.682251 14648 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:56.681584 14579 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.640355 14019 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.686826 14019 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.686968 14579 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:56.689345  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.696483  7229 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
W20260709 13:19:56.701741  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.708595  7237 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:56.663879 14459 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.709491  7234 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:56.712574  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.713172  6588 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:19:56.714572  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.715478  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:56.723510  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.725976  6596 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:56.724525  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.710172 14613 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.734941 14613 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:56.736562  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 509736us
I20260709 13:19:56.737332  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:56.722801 14610 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:56.739441 14413 raft_consensus.cc:3055] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:56.739297 14610 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.739925 14413 raft_consensus.cc:740] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:56.740914 14413 consensus_queue.cc:260] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.741561 14610 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:56.742115 14610 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:56.742870 14459 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.743261  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.743713  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.743947  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.744141  6664 raft_consensus.cc:2410] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:19:56.744272  6668 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:56.723582 14580 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.743171  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.746161 14606 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:56.746452  6666 raft_consensus.cc:1240] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:56.746906 14413 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.749046  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.750219  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.716012 14516 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.751602 14568 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:56.756170  7200 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:56.758595  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:56.765255  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.765551  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.766436  6662 raft_consensus.cc:2393] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 20: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:56.766528  6663 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:56.765496  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.767982 14586 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:56.769186  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:56.769717  7231 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.770924  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:56.746590 14606 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.771247  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:56.777405  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.766988  6668 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:19:56.779420  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.766250 14648 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:56.778905 14568 raft_consensus.cc:2804] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:56.774508 14629 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:56.755591 14610 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.782356 14470 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:56.795101  6665 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:56.801945  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:56.784538 14606 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:56.795802  6666 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:56.803098  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:56.806692  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.784200 14533 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:56.808362 14554 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:56.780592 14646 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.832562  6666 raft_consensus.cc:1275] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 13:19:56.818727  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.828528  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:56.833889  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.842031  6669 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:19:56.783797 14506 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:19:56.843052 14646 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:56.847455  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45877.4us
I20260709 13:19:56.854303  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.847965 14401 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:56.848071  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.848542  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.857393  6664 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.857825  7228 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:56.859026  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.860107 14646 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:56.846166  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.860532 14647 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:56.861061 14647 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:56.861433  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.848873  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.862517  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.862911  6671 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:56.863569  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.863703  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.815809 14470 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.842535  6669 pending_rounds.cc:85] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 14
I20260709 13:19:56.868774  6669 pending_rounds.cc:107] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:19:56.869115  6669 raft_consensus.cc:2889] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:56.848824  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.829802 14568 raft_consensus.cc:697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:56.860060  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.864621  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.833271 14436 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:56.821506 14533 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.867554  7237 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:19:56.852877  6665 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:19:56.872474  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.895720  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.854112 14613 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:56.899019  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.853300 14485 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:56.902151  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.903100  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.870357 14470 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:56.907466  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.909108  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:56.895200 14401 consensus_queue.cc:1059] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 13:19:56.880890 14533 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.911707  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.873881 14268 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:56.918735  6664 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:19:56.921356  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46288us
W20260709 13:19:56.909276  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:56.941135  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:56.942277  6663 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:56.944434  7235 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:56.940936 14619 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:56.896492 14609 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
W20260709 13:19:56.941179  6637 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.950716  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 171394us
I20260709 13:19:56.951200  7228 raft_consensus.cc:2393] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:19:56.957509  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.959679  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:56.961161  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 178397us
I20260709 13:19:56.947469 14647 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:56.968502  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:56.969163  7237 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 18.
W20260709 13:19:56.962042  6637 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.911096 14485 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.950599 14617 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:56.961712  6562 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.970065  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.945564 14619 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.970357  7230 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:19:56.972476  7235 raft_consensus.cc:1240] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:56.955756 14579 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:56.975756  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.976651  6588 raft_consensus.cc:2393] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 24: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:56.981518  6596 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:19:56.982985  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.957443 14642 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.185s
W20260709 13:19:56.983340  6637 leader_election.cc:336] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.980629  6562 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:56.983341  6671 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:56.983819  6593 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
W20260709 13:19:56.987762  6637 leader_election.cc:336] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.984381  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.988162  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.984228  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.990268  6562 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:56.991214  6637 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.991503  7231 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:56.991672  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:56.992177  7234 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:56.993708  6562 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:56.994035  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:56.995690  6637 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:56.996609  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:56.997397  6637 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:56.980394 14622 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:57.001997  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:56.989267 14516 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:57.000968  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:56.962393 14568 consensus_queue.cc:237] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.981623 14579 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.000520 14643 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:56.981436 14617 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:56.993202 14647 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:57.001472 14506 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:57.065019  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:57.031008 14610 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.036538 14422 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:56.999161 14235 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:57.075377  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.077498  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:57.077989 14652 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:19:57.079425 14660 consensus_queue.cc:1048] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:57.083070  6592 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 23: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:57.083103  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.043924 14580 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:57.112207 14617 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.112522 14619 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.114249 14648 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:57.114746 14648 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:57.115272 14485 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.099828 14579 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:57.100775 14658 log.cc:927] Time spent T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:19:57.102797  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:57.104564 14654 log.cc:927] Time spent T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.257s	user 0.002s	sys 0.000s
I20260709 13:19:57.074682 14019 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:57.111238  6591 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:19:57.118780  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.156728  6664 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 18.
I20260709 13:19:57.157899  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:57.135006  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.158921 14554 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:57.159647  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:57.160213  6671 raft_consensus.cc:2376] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 24. Re-sending same reply.
I20260709 13:19:57.160983  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:57.162039  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.135512 14607 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:57.155710 14579 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:57.162886 14607 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.163553 14579 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.165138 14607 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.135967 14586 raft_consensus.cc:3055] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:57.165494 14579 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.165707 14586 raft_consensus.cc:740] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:57.166699 14586 consensus_queue.cc:260] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.167228 14019 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.167912 14586 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:57.169245 14019 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:57.162902  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.171265  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.172423  6664 raft_consensus.cc:2368] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 23. Current term is 24.
I20260709 13:19:57.137540 14436 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:57.068559 14643 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.139101 14470 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.140632 14533 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.134398  6663 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:57.142555  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.143457  6669 raft_consensus.cc:1240] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Rejecting Update request from peer 64b95375974b48d18c596a23c9cac75c for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:57.144120  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:57.151026  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:57.155505  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 78076.9us
W20260709 13:19:57.176775  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:57.092111 14422 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.142458 14580 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:57.181236  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.212021  6594 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:57.209623  6561 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:57.213462  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.214211  7237 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:57.217993 14506 consensus_queue.cc:1059] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:57.218412  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:57.218816  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.219878  6636 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.215608  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:57.155192 14235 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:57.225901  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.226917  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:57.228519  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:57.229048  6487 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.231060  6667 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:57.241245  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.208709 14610 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.245023  6588 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:57.242271  6671 raft_consensus.cc:1275] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:57.242471  6668 raft_consensus.cc:1275] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:19:57.247478  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.209033 14647 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.249517  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.227643 14606 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.254266 14019 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:57.254863 14019 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:57.255573 14664 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:19:57.255748 14436 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.218526 14580 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.256443 14436 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.244223 14607 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:57.258754 14661 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:57.259326 14661 raft_consensus.cc:697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:19:57.252417  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101481us
I20260709 13:19:57.261091  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.218842 14506 consensus_queue.cc:1243] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer 64b95375974b48d18c596a23c9cac75c log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:19:57.249251  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.264776  7198 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:57.262252  7200 leader_election.cc:336] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:57.268064  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 274396us
I20260709 13:19:57.225210 14617 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:57.272595  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:57.280637  7200 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:57.281453  6486 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.241852 14646 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:57.276657  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.293844  7230 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:57.273346  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.221796 14412 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:19:57.275791  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.264194 14606 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:57.288241  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:57.287091  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.297307 14422 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.293973 14613 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.301970  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.266881 14647 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.305059 14235 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.283475 14268 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:57.287956 14516 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:57.261015 14610 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.260226 14661 consensus_queue.cc:237] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.258795 14607 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.320113  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.326004  6665 raft_consensus.cc:2376] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20. Re-sending same reply.
W20260709 13:19:57.309186  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44577.3us
I20260709 13:19:57.324107 14670 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:57.338104  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.325548 14237 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:57.286391 14617 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.299522 14579 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:19:57.325421 14516 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.326817 14646 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:57.333083 14485 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:57.328709 14606 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:57.358008 14606 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.328608 14647 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.355625 14485 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:57.301857 14613 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.338969 14670 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:57.342902 14436 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:57.349351 14019 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.350739 14568 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:57.354673 14516 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:57.352867 14629 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:57.382532  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 166980us
I20260709 13:19:57.367795 14670 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:57.412762  6637 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.418375  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.424750  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:57.421798  6637 leader_election.cc:336] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:57.396936  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 78341.4us
I20260709 13:19:57.412055 14533 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:57.430045  6487 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:19:57.437135  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 227649us
W20260709 13:19:57.432683  6637 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.437904  6636 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:57.438645  6637 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.414685 14470 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.439271  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.442226  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.422886 14474 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:57.426434 14538 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:57.445901  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.432761 14643 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.441037 14436 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:57.451550 14579 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:57.479880  6485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.474423 14533 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:57.453081 14474 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.475831 14436 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:57.441759 14485 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.497063 14485 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:57.497285 14643 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:57.499294  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 53474.8us
W20260709 13:19:57.501214 14674 log.cc:927] Time spent T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:57.506201  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.506774 14610 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:57.514918  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62551.1us
W20260709 13:19:57.516120  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 49981.9us
I20260709 13:19:57.485466 14678 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:57.490348 14647 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:57.522003  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.523710  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:57.519914  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 81372us
I20260709 13:19:57.527297 14622 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.529670 14622 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.528059 14607 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.533320 14607 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:57.534380 14474 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:57.537024 14436 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.544770  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:57.545461  6595 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:57.546046  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.547315 14619 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.551913  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:19:57.531642  6637 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:57.555131  6637 leader_election.cc:336] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.555039  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.555729  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.556421 14694 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:57.563548  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 57435.9us
I20260709 13:19:57.564857  6486 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:57.567754  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 247724us
I20260709 13:19:57.519657 14643 raft_consensus.cc:3060] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:57.571926  6486 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.572511  6595 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:57.575750 14622 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:57.570228  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.571357 14613 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:57.567826 14506 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:57.573288 14672 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:57.573472 14673 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:19:57.581185 14648 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.547966 14619 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.588755 14619 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.591019 14617 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.599959 14586 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:57.610252  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 172400us
I20260709 13:19:57.613700  6636 leader_election.cc:400] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21.
I20260709 13:19:57.618054  6595 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:57.621636 14516 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.623008  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.647835  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.649834 14696 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:57.650774 14643 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:57.652132 14606 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:19:57.626296  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 292116us
I20260709 13:19:57.629220  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.633581  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:57.662743  6591 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:57.648084  6663 raft_consensus.cc:3055] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:57.663293  6663 raft_consensus.cc:740] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:57.627586 14648 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:57.636574  6488 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:57.637017 14642 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:57.637145 14019 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:57.676352 14606 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:57.676659  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.676908 14606 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:57.677263  6591 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:19:57.677740 14606 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:57.674422 14670 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.679972 14670 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:57.680521  6663 consensus_queue.cc:260] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:57.681313 14663 log.cc:927] Time spent T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.570s	user 0.000s	sys 0.002s
W20260709 13:19:57.636134  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 66026.7us
I20260709 13:19:57.600600 14586 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.682426  6487 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.683538 14586 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:57.683722 14643 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:57.639590 14647 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:57.642145 14237 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:57.686192 14237 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:57.686408 14690 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:57.687002 14690 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.689117 14690 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:57.646745  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:57.649863 14694 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:57.651072  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.651686 14516 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:19:57.635438  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64564.6us
I20260709 13:19:57.674641  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:57.704069  7230 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:57.681654 14642 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 24 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:57.681756  6663 raft_consensus.cc:3060] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:57.697654  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:57.722538  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.697877  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:57.713011  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:57.714826  6597 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:57.721357  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:57.721891  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 70916.8us
I20260709 13:19:57.725183  6670 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:57.723301 14670 consensus_queue.cc:1059] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 13:19:57.731595 14461 raft_consensus.cc:3055] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:57.732062 14461 raft_consensus.cc:740] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:57.733006 14461 consensus_queue.cc:260] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:57.734510 14461 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:57.741590  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.744196  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.747223  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:57.748083  6588 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:19:57.753474  6590 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 24: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:57.753607  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:19:57.754165  6637 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.754369  7237 raft_consensus.cc:2393] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 15: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:19:57.754598  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.763401  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.764212  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.764792  7232 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:57.766571  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:57.767910  6637 leader_election.cc:336] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.765805  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.770232  7230 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:19:57.770521  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:19:57.768554  6637 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.739691 14647 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:57.767787  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:57.775862  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:57.762737 14656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.023s	sys 0.003s
I20260709 13:19:57.777181  7236 raft_consensus.cc:1240] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:57.737787 14643 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:57.770359  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.789882  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:57.763753 14516 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:57.794878  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.771361 14459 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:57.784246 14656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.023s	sys 0.003s
I20260709 13:19:57.780493 14579 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:57.808640  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:57.813153  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:57.784876 14643 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.815974  6591 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 15.
I20260709 13:19:57.817798  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:57.818544  6592 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 921c423406da4f289884afca51ca198f in this term.
I20260709 13:19:57.771709 14485 consensus_queue.cc:1059] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 13:19:57.823657  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 69787.4us
I20260709 13:19:57.813076  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:19:57.837754  6563 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:57.832770  6663 raft_consensus.cc:1275] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:19:57.838250  6563 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.842983  6663 pending_rounds.cc:85] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 15
I20260709 13:19:57.843462  6663 pending_rounds.cc:107] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:19:57.848179  6663 raft_consensus.cc:2889] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:57.808420 14470 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:57.862644  6562 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:57.832238 14610 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:57.833442 14643 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:57.817968 14696 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:57.859135 14536 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 13:19:57.849498 14401 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:57.897430  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 59655.7us
I20260709 13:19:57.814050 14268 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:19:57.912714  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:57.860771 14646 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:57.919112  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 106027us
I20260709 13:19:57.926846  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:57.855554 14680 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:57.878962 14470 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.907609 14660 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:57.974546  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62022.1us
I20260709 13:19:57.913952 14610 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:57.973488  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 197705us
I20260709 13:19:57.979525 14691 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:57.979949 14617 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:57.917799 14413 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.813757 14642 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:57.993896 14687 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:19:57.928591 14538 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.007247 14698 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
W20260709 13:19:58.019045  6634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.899458 14580 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:58.023366 14470 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:58.023464 14690 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:57.901031 14579 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.035751  6486 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:57.921243 14646 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.111281 14646 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:58.003523 14697 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:19:58.042841 14710 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:58.043526 14413 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.117175 14413 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.045301 14237 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:57.979377 14664 log.cc:927] Time spent T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.007s
I20260709 13:19:58.119089  6596 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:58.038062 14610 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:58.073143 14692 log.cc:927] Time spent T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:19:58.074285 14580 raft_consensus.cc:697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:58.054533 14670 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:19:58.081383 14579 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.082490 14691 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:19:58.082708  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 47047.9us
I20260709 13:19:58.082864 14711 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:19:58.088383  7200 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 95 similar messages]
W20260709 13:19:58.152530  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 64183.7us
W20260709 13:19:58.095333 14705 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:19:58.098093  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 171364us
I20260709 13:19:58.122747  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:58.150985 14686 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.170871 14672 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:58.171458 14672 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:58.172300 14672 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.173066 14610 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.170120 14686 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.162388  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:58.166761  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 334915us
I20260709 13:19:58.179373  7198 leader_election.cc:400] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 23. Current term is 24.
W20260709 13:19:58.169479  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46820.4us
I20260709 13:19:58.179731  7198 leader_election.cc:403] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:19:58.180774  7198 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:58.064035 14538 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:58.184358  6637 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:58.189399  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.189954  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.186640  6662 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:19:58.184224  7200 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.191129  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.193729  6663 raft_consensus.cc:1275] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:19:58.155328 14691 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.196380  6668 pending_rounds.cc:85] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 13
I20260709 13:19:58.197320  6668 pending_rounds.cc:107] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:19:58.197636  6668 raft_consensus.cc:2889] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:58.198307  7198 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:58.202041 14533 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.202258 14683 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:58.203004 14717 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.202812 14533 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.203787 14715 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:58.204098 14716 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) with higher term. Message: Invalid argument: T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c for earlier term 23. Current term is 24.
I20260709 13:19:58.204471  6665 raft_consensus.cc:3055] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:58.204329 14715 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.204826  6665 raft_consensus.cc:740] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:58.200855 14690 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:58.205717  6665 consensus_queue.cc:260] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:58.206302 14701 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:19:58.206802  6665 raft_consensus.cc:3060] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:58.207325 14580 consensus_queue.cc:237] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.209630  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:58.210140  6637 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.211118  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.213048  6590 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:58.214587  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.215248  6670 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:19:58.215749  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.190871 14710 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.218278  7198 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:58.194659 14606 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:58.221688  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.228722  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.239082  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.241266  6671 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:58.247138  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:58.252530 14684 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:19:58.252990 14579 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.254550  7235 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:19:58.255014 14643 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.211689 14646 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.257280 14643 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.257902 14568 raft_consensus.cc:2804] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:58.258291 14646 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.258472 14568 raft_consensus.cc:697] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:58.258805 14470 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.203547 14717 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.217602 14681 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:58.228588 14694 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:58.259617 14622 consensus_queue.cc:1048] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:58.253032 14538 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:58.260346 14647 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:58.261305 14678 raft_consensus.cc:3055] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:58.305835 14678 raft_consensus.cc:740] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:58.240823 14710 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.262332 14714 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:58.263447  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.310223  6591 raft_consensus.cc:2393] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:58.307529 14714 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:58.264112  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 45905.8us
I20260709 13:19:58.312224 14714 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:58.306841 14670 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:19:58.214694 14610 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:58.315176  6667 raft_consensus.cc:2393] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:58.316807 14470 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.319190  7235 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:19:58.317914  7232 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:19:58.322705 14717 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.266345  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.269088 14568 consensus_queue.cc:237] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.326229 14710 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.202143 14689 consensus_queue.cc:1059] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:19:58.327260  6595 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:58.327709 14689 raft_consensus.cc:3055] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:58.217875 14660 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:58.274278 14536 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:58.275184 14516 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:58.275316 14686 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.275874 14237 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.359602  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.239960 14586 consensus_queue.cc:1048] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:19:58.234853 14715 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.250878 14485 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:58.233302 14716 consensus_queue.cc:1048] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:19:58.277812 14019 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.281315  6665 raft_consensus.cc:1275] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:19:58.283394  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.283684  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:58.371731  6665 pending_rounds.cc:85] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Aborting all ops after (but not including) 13
I20260709 13:19:58.286216  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.372087  6665 pending_rounds.cc:107] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:19:58.288511  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:58.372450  6665 raft_consensus.cc:2889] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:58.372655  6597 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:58.290777  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.293928  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.301100  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.270431 14533 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.341944  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.324298 14678 consensus_queue.cc:260] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.361515  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.364364  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.381695  6666 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:19:58.366293  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:58.382638  7235 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.346940 14470 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.383713 14470 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.368172  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:58.360674 14536 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.328269 14689 raft_consensus.cc:740] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:19:58.377262  6589 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:58.387423  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.387943  7236 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:58.388114  6670 raft_consensus.cc:2468] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:58.373854  6593 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 15.
I20260709 13:19:58.387159  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.377287  6590 raft_consensus.cc:2393] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 19: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:58.391952  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.393088  7231 raft_consensus.cc:1217] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Deduplicated request from leader. Original: 17.14->[18.15-20.16]   Dedup: 18.15->[20.16-20.16]
I20260709 13:19:58.393620  7231 raft_consensus.cc:1240] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: [18.15-20.16]
I20260709 13:19:58.393671  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:58.394482  6590 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:58.405083  6597 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
W20260709 13:19:58.408391  7200 leader_election.cc:336] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.409864  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.410480  7230 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
W20260709 13:19:58.411109  6637 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.370926 14691 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.375837 14019 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.421073  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.421629  6664 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:19:58.422794  6486 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:19:58.417248  6637 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.424726  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.427614  7233 raft_consensus.cc:2368] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23.
I20260709 13:19:58.429176  6487 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:58.430459  6562 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:58.431367  6636 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.433750  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.432107  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.407047 14586 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.363054 14660 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.435168  7237 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 24.
I20260709 13:19:58.437376  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.440312  6636 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.438720  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.376250 14485 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.440196  7231 raft_consensus.cc:1240] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:58.443293  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:58.444152  6669 raft_consensus.cc:1240] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:58.469597  6664 raft_consensus.cc:3060] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:19:58.465704  6486 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:58.471655  6589 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:19:58.429711 14685 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 13:19:58.436216 14474 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:58.435912 14672 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:58.445412 14401 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:58.482568  6590 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:19:58.432950 14642 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:19:58.471864 14718 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.441636 14647 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:58.470090 14711 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:58.442106 14422 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:19:58.490262 14538 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.495661  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 61806.5us
I20260709 13:19:58.441192 14681 consensus_queue.cc:1048] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:19:58.495201  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:58.506559  6636 leader_election.cc:400] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23.
I20260709 13:19:58.510741  6636 leader_election.cc:403] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:58.511413  6486 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:58.482585 14610 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.490094 14697 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.054s	user 0.001s	sys 0.006s
I20260709 13:19:58.468675 14568 consensus_queue.cc:1048] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:19:58.484776 14686 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:58.487147 14670 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:58.521149  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:58.528537  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.497370 14711 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.465667 14586 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.480348 14668 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.084s
W20260709 13:19:58.528460  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 52500.5us
I20260709 13:19:58.521129 14617 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:58.517637 14680 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:58.544090  6664 raft_consensus.cc:2468] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
W20260709 13:19:58.540869 14726 log.cc:927] Time spent T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:19:58.547169 14516 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:58.591099 14728 log.cc:927] Time spent T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:19:58.595273 14691 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.595942  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:19:58.597410  7200 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:58.599509 14516 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.600379 14681 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:58.600816 14681 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.533463 14694 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:58.600103 14536 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:58.601322 14694 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.601756 14694 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:58.609028 14019 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.610006 14646 raft_consensus.cc:2804] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:58.614115 14470 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.618026  7235 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 19.
I20260709 13:19:58.525555 14610 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.560369 14586 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.563454 14485 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.547173 14579 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:19:58.570889 14606 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:58.479351 14689 consensus_queue.cc:260] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.548906 14714 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for earlier term 22. Current term is 23.
I20260709 13:19:58.520016 14538 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.509600 14718 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.535054 14686 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.600095 14516 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.537619 14711 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:58.554282  6486 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:58.621240  7236 raft_consensus.cc:1240] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 53fee9894731461a821a131be24dbbca for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:19:58.595391  6637 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:58.646485  6562 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:19:58.646934  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 51633.9us
I20260709 13:19:58.640717  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.601166 14681 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:58.648619  6589 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:19:58.648644  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.623165 14646 raft_consensus.cc:697] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:19:58.655966  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 101794us
I20260709 13:19:58.658130  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.658798  7237 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:58.659013  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:58.663873  6486 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:19:58.666185  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:58.669819 14715 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:58.672254 14516 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:58.674999 14680 raft_consensus.cc:2804] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:58.675534 14680 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.675933 14680 raft_consensus.cc:3060] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:58.678764 14607 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:58.679351 14607 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.679783 14607 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:58.685436 14610 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:19:58.698709  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 253949us
I20260709 13:19:58.698352  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:58.701048  6663 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:58.701370  6563 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:58.701444  7200 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:58.702423  6563 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.702811  6563 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:19:58.704430  7200 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.656006 14485 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:58.657688 14536 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.659235 14470 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:58.710206  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 181740us
I20260709 13:19:58.656313 14690 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:58.706763  6563 leader_election.cc:336] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.720167  6563 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:58.714280  6670 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:19:58.715899  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.657721 14019 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:58.721304  6666 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:19:58.721731  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.722442  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.722486  6670 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:19:58.724321  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:19:58.725520  6563 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.692816 14235 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.750866 14235 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.753512 14235 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.755075  6667 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:19:58.728827  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.729343  7234 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:58.730755 14668 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:19:58.732008 14470 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:58.726199 14554 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.735340 14718 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.739776 14646 consensus_queue.cc:237] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.718890 14613 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:58.749958 14538 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.742008 14694 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.822475  6486 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:58.837303  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:58.854879  6663 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:19:58.746352 14681 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.859390  6665 raft_consensus.cc:1240] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:19:58.836615 14642 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:58.819638 14643 consensus_queue.cc:1059] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:19:58.823566 14554 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.881053  7231 raft_consensus.cc:1240] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:58.854592 14715 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:58.841336 14670 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:19:58.873646 14642 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.831689 14718 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:19:58.896095  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 168789us
I20260709 13:19:58.898201 14680 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.899503 14734 consensus_queue.cc:1059] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:19:58.900393 14607 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:58.890199 14642 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:58.887125 14470 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.918478  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:58.907568 14681 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:58.958551  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:58.914292 14689 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:58.959250  6589 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:19:58.900447 14734 raft_consensus.cc:3055] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:58.900442 14680 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.902585 14607 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.912225 14554 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:58.909409 14606 raft_consensus.cc:3055] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:58.912242 14694 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:58.904742 14536 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:58.986265  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:58.889071 14715 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:58.998311  6668 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:58.998678  6670 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
W20260709 13:19:59.006361  7200 leader_election.cc:336] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.968634 14734 raft_consensus.cc:740] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:58.985764 14606 raft_consensus.cc:740] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:59.026002 14607 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:59.048177 14715 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.051520  6592 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:19:59.051914 14746 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:19:59.055210  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:59.059295  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:59.109628  7234 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:19:59.108886  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:59.111377  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:59.112520  6595 raft_consensus.cc:2393] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:59.111390  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 52144.1us
I20260709 13:19:59.115530  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.115793  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.112242  6592 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:19:59.124739  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:59.125488  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:59.105847 14643 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.128545  7198 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:19:59.111366 14711 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.136914 14711 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.140285  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:59.141394  6561 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:59.036530 14607 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.118187 14019 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:59.105769 14461 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:19:59.144883 14646 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:59.142642 14629 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:59.036692 14734 consensus_queue.cc:260] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.147534 14723 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:59.149425 14661 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:59.155719 14019 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:59.145035 14643 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.123476 14606 consensus_queue.cc:260] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:59.162740 14723 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.164269 14734 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:59.172652 14019 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:59.189220  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:59.202575 14607 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.184794 14723 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:59.214076  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 85589.3us
W20260709 13:19:59.221412 14752 log.cc:927] Time spent T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.159s	user 0.000s	sys 0.002s
W20260709 13:19:59.224139 14755 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:19:59.214284 14606 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.226640  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:59.228629 14516 raft_consensus.cc:3055] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:59.229056 14516 raft_consensus.cc:740] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:19:59.231161 14643 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.232170 14660 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.234985  7232 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:19:59.243037  7237 raft_consensus.cc:1275] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:59.229945 14516 consensus_queue.cc:260] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.316069 14660 consensus_queue.cc:1048] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:59.319268 14723 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.322546 14723 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.323714 14723 raft_consensus.cc:493] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:59.324298 14723 raft_consensus.cc:515] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.326920 14723 leader_election.cc:290] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.332718  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:59.247851 14470 raft_consensus.cc:493] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
W20260709 13:19:59.339622  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 197392us
I20260709 13:19:59.348057  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:19:59.242802 14642 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:59.370925  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:59.372532  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:59.294997 14713 raft_consensus.cc:493] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:19:59.339031 14461 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.381421  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:59.347205 14413 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.383002 14413 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:59.387238  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 133565us
I20260709 13:19:59.387545  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:59.388134  6588 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 19.
I20260709 13:19:59.388731  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:59.389302  7237 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:59.389612  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:59.390328  6588 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:59.382197 14723 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:59.393070  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.369123 14470 raft_consensus.cc:515] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.383605 14461 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:59.418480  5410 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 6547)
I20260709 13:19:59.379763 14713 raft_consensus.cc:515] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.432287 14642 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:19:59.456290 14413 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.456930 14678 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.461084 14461 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.465134  6671 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:59.467996  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:19:59.468662  6597 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:59.475600  7234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46684 (request call id 1959) took 1112 ms. Trace:
I20260709 13:19:59.476178  6597 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:19:59.476073  7234 rpcz_store.cc:276] 0709 13:19:58.362909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.363686 (+   777us) service_pool.cc:224] Handling call
0709 13:19:59.475393 (+1111707us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54002e4ad4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:19:59.475505 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789602048}
I20260709 13:19:59.482923  6588 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 16.
I20260709 13:19:59.483589 14696 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:19:59.483843 14750 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.438s	user 0.002s	sys 0.000s
I20260709 13:19:59.484151 14696 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:19:59.485242  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 438373us
I20260709 13:19:59.486218 14696 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:59.487547  6666 raft_consensus.cc:1275] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:19:59.492807  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.495223  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:59.495800  6666 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:59.445639  7237 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:19:59.509881  7233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46684 (request call id 1965) took 1013 ms. Trace:
W20260709 13:19:59.515355  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.515434  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:19:59.516175  7237 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:59.520390  6563 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:59.491144  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:59.527385  6665 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:59.522871  6669 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:19:59.533521  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:59.445791 14723 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:19:59.535780  7235 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:59.529626  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:59.543357  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:19:59.544472  6597 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 24: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:19:59.548338  6488 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:19:59.552416  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.512533  7233 rpcz_store.cc:276] 0709 13:19:58.496544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.496698 (+   154us) service_pool.cc:224] Handling call
0709 13:19:58.497104 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.622292 (+125188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.622339 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:58.623445 (+  1106us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.879720 (+256275us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:58.879750 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.879809 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:59.483272 (+603463us) spinlock_profiling.cc:243] Waited 71.1 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:59.490701 (+  7429us) raft_consensus.cc:1685] finished
0709 13:19:59.490722 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:59.509804 (+ 19082us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11916,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2813,"spinlock_wait_cycles":163625216,"thread_start_us":298661,"threads_started":2,"wal-append.queue_time_us":341566}
I20260709 13:19:59.557914  7237 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:59.560562  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:59.561168  6665 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:59.562651  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:59.563287  6663 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:59.514730 14680 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.568734  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.564812  6488 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:59.571687  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.546032 14436 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:59.572496  7228 raft_consensus.cc:2468] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:19:59.558910 14617 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:59.577113  6488 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:59.585925  6662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3461) took 1173 ms. Trace:
W20260709 13:19:59.588866  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 108783us
I20260709 13:19:59.567662 14734 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:59.590996  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.545748 14413 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:19:59.577081 14459 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:59.581341 14533 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:59.600859  7228 raft_consensus.cc:2393] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 24: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:19:59.603369  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 98406.5us
I20260709 13:19:59.612381  6665 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:19:59.617915  6671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60486 (request call id 3454) took 1326 ms. Trace:
I20260709 13:19:59.618486  6671 rpcz_store.cc:276] 0709 13:19:58.291817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.291941 (+   124us) service_pool.cc:224] Handling call
0709 13:19:58.299423 (+  7482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.465537 (+1166114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.617603 (+152066us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000e51d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b9081e01 00007f04b908083e 00007f04be045419 00007f04b75bcbd4 00007f04b75bcb62 00007f04b75bca91 00007f04b75bca0c 00007f04b75bbcc2 00007f04b73bc48c 00007f04b73bb8c6
0709 13:19:59.617838 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2680078720}
I20260709 13:19:59.583541 14436 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.597919 14619 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:59.622488 14619 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:59.623474 14619 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:59.586418  6662 rpcz_store.cc:276] 0709 13:19:58.407991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.408110 (+   119us) service_pool.cc:224] Handling call
0709 13:19:58.408420 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.428133 (+ 19713us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:58.571687 (+143554us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b18002d8320. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f76e07 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bca793c7 00007f04b908b9a7 00007f04b909b0bf 00007f04b909b038
0709 13:19:58.843532 (+271845us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.952994 (+109462us) log.cc:844] Serialized 14 byte log entry
0709 13:19:59.439059 (+486065us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b18002d8320. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b5f0ec0a 00007f04bf64e219 00007f04b5f96a40 00007f04b5f914e4 00007f04b5f91455 00007f04b9079060 00007f04b908b716 00007f04b909c287 00007f04b909c208
0709 13:19:59.439228 (+   169us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.441588 (+  2360us) log.cc:844] Serialized 26 byte log entry
0709 13:19:59.441760 (+   172us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:59.441998 (+   238us) log.cc:844] Serialized 14 byte log entry
0709 13:19:59.580879 (+138881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.580962 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:59.581007 (+    45us) raft_consensus.cc:1685] finished
0709 13:19:59.581034 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:59.581179 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139239,"prepare.queue_time_us":212112,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":130,"raft.queue_time_us":681937,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":6097,"spinlock_wait_cycles":36503680,"thread_start_us":349030,"threads_started":3,"wal-append.queue_time_us":271056}
I20260709 13:19:59.629259 14723 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:59.599815 14459 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.631016 14459 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.574195 14680 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.621603  7200 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:19:59.640412 14436 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.576879 14756 consensus_queue.cc:1059] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:59.643625 14686 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.648466 14617 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:19:59.649031 14617 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.658145 14681 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:59.665072  6666 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:19:59.620882  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:19:59.658651 14681 raft_consensus.cc:493] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.666568 14681 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.668567 14696 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:19:59.671253  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 50444.9us
W20260709 13:19:59.678962  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 101946us
I20260709 13:19:59.681063  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:19:59.682345  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.703462  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:19:59.683347  6486 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:19:59.683887 14757 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.707950 14642 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.717720 14536 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:59.714192 14268 consensus_queue.cc:1048] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
W20260709 13:19:59.777241  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 118417us
I20260709 13:19:59.716151 14733 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:19:59.755007 14683 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.776228 14459 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:59.771173 14436 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:59.725104 14757 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:19:59.773974 14536 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.825112 14536 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.827111 14757 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:59.829249 14680 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.834223  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:19:59.836383  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:59.836990  6666 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:19:59.836907  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc61b2fa4ee43b7a7a91b2eb2ecd874" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:59.755291 14642 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.792495 14617 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.851181  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:59.851946  6668 raft_consensus.cc:2393] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:19:59.851895  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:19:59.772738 14711 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:19:59.840569 14757 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.849052 14436 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:19:59.866672 14757 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.867586 14436 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.869124 14757 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:19:59.854197  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:19:59.869832  6671 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:59.875790  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:19:59.881070 14758 log.cc:927] Time spent T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.776s	user 0.001s	sys 0.009s
I20260709 13:19:59.882545  6667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51542 (request call id 1659) took 1164 ms. Trace:
I20260709 13:19:59.863346 14716 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:59.886284  6671 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:19:59.882917  6667 rpcz_store.cc:276] 0709 13:19:58.717752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.756114 (+ 38362us) service_pool.cc:224] Handling call
0709 13:19:58.756468 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.756810 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.756836 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:58.757274 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.955443 (+198169us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:58.955507 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.955625 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:59.871651 (+916026us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:19:59.882258 (+ 10607us) raft_consensus.cc:1685] finished
0709 13:19:59.882279 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:59.882490 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":316,"raft.run_cpu_time_us":43,"raft.run_wall_time_us":44,"spinlock_wait_cycles":26802944,"thread_start_us":197812,"threads_started":1,"wal-append.queue_time_us":274511}
I20260709 13:19:59.796705 14683 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.863245 14401 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:59.896482 14779 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.920090 14568 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:19:59.921854 14268 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:19:59.914810 14716 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.955011 14268 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:19:59.955930  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 248741us
I20260709 13:19:59.931007 14401 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:00.013376 14789 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:20:00.014635 14401 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:19:59.952824 14568 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:19:59.941414 14779 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:00.016178 14716 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.017164 14568 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.017304 14779 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.017740 14683 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.020623 14536 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.958382 14268 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.023937 14642 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.030021  6670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60486 (request call id 3468) took 1197 ms. Trace:
I20260709 13:20:00.035039 14681 raft_consensus.cc:515] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.021164 14536 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:00.040565 14536 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:00.044754 14681 leader_election.cc:290] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.049909  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.050654  6669 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:20:00.054920  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.030566  6670 rpcz_store.cc:276] 0709 13:19:58.832612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.836988 (+  4376us) service_pool.cc:224] Handling call
0709 13:20:00.029799 (+1192811us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540011e9d4. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b90852fa 00007f04be046b17 00007f04b75c16d4 00007f04b75c1662 00007f04b75c1591 00007f04b75c150c 00007f04b75c07c2 00007f04b73bc48c 00007f04b73bb8c6 00007f04b73bb26f
0709 13:20:00.029917 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368630528}
W20260709 13:20:00.087349  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.087745  6635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.088497  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.090806  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:00.091928 14691 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:00.092298 14691 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:00.092648 14691 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:00.118974  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.118938 14713 leader_election.cc:290] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.128352 14770 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:00.129020 14770 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.135087  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.136715  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.139082  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.139741  7234 raft_consensus.cc:2368] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:20:00.141096  7235 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:00.148717 14470 leader_election.cc:290] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.134420 14770 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.161026  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.162293 14646 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.165343 14646 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.167277 14459 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.175554  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.178984  6591 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:20:00.179446  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.182025  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:00.182902  6591 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:00.188472 14784 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.189024 14784 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.186365 14757 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.191109 14784 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:00.187079  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 121765us
I20260709 13:20:00.191272 14757 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.194181  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.177891 14696 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.195063 14696 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.196707 14516 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:00.197479 14696 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.195780 14610 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.198314 14610 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:00.200604 14610 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:00.202366 14757 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.186595 14747 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.203923 14747 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.190970 14646 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.204983 14019 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.205607 14019 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.207756 14019 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.181401 14617 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:00.208662 14617 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.211150  6562 leader_election.cc:400] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:20:00.211601  6562 leader_election.cc:403] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:00.216862  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.218784  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.223393  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:00.224951  6668 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:00.204865 14785 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.249172  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.250195  6663 raft_consensus.cc:2410] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:20:00.217798 14617 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.255914  6592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.206328 14747 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.214102 14685 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:20:00.270817 14784 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.258759 14785 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.279577 14784 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:00.271366  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 79336.1us
I20260709 13:20:00.210812 14413 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.281177 14413 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.274459  6592 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 13:20:00.261153  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48795.1us
I20260709 13:20:00.204258  6591 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:20:00.308110  6668 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:20:00.308573  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.310385  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.312070  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.312915  6588 raft_consensus.cc:2468] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
W20260709 13:20:00.316893  6637 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.317610  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:00.319260 14536 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.323493  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.326283  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.327960  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:00.330524  6637 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.331077  6637 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:00.333811 14459 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.334252  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "627436404a154d80a882f4900dff2948" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.335279  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a758d1835d6148668ab495504660d72a" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.333969  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.345978  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5
W20260709 13:20:00.336812  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 151098us
I20260709 13:20:00.313190 14661 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:20:00.350656  6487 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.354393  7200 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:00.365829  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:00.366307  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.366815  6670 raft_consensus.cc:3060] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:00.366987  6669 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:20:00.372344  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.373150  6669 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
W20260709 13:20:00.380067 14804 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:20:00.382105  6595 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:20:00.357085 14617 raft_consensus.cc:2804] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:00.382987 14617 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.383483 14617 raft_consensus.cc:3060] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:00.390710  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.392014  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.393020  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.393661  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:20:00.393965  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.394776  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.395413  6594 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:20:00.395754  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.396064 14691 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.402148 14413 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.337129 14696 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:00.335916 14734 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:20:00.356781 14777 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:00.404090 14777 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.405400  6668 raft_consensus.cc:1275] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:00.342236 14785 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:00.361017  6637 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.409945  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:00.411311  6561 leader_election.cc:304] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.417115 14660 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:00.417534 14689 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:00.417697 14660 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.418217 14660 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:00.419006  5410 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 6547)
I20260709 13:20:00.417639 14691 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.350091 14422 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.431974  6666 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:00.422832  7198 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:00.423686  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.424685  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:00.434727  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
W20260709 13:20:00.432602  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71639.4us
W20260709 13:20:00.436779  6637 leader_election.cc:336] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.437366  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:00.437668  6561 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:00.439442  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.446317  6486 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:00.449702 14436 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:00.450224 14436 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:00.451220 14643 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:00.451149 14436 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:00.454465  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 103930us
I20260709 13:20:00.435046 14516 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.457243 14516 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:00.458997  6637 leader_election.cc:336] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.459620  6637 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:00.462064 14610 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:00.454494 14717 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:00.463852 14784 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.422464 14768 consensus_queue.cc:1059] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 13:20:00.466454  6415 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 19 to 22, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 22 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:00.467442 14768 raft_consensus.cc:3055] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:20:00.468052 14768 raft_consensus.cc:740] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:00.468305  6415 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:00.471469  6670 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:20:00.473899  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.406603 14777 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.418241 14689 raft_consensus.cc:697] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:20:00.469887  6415 catalog_manager.cc:5697] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:00.474831  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.476183 14607 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:00.477082 14731 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.477644 14731 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.478343  6595 raft_consensus.cc:1275] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:00.479732 14731 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.473297 14606 raft_consensus.cc:3060] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:00.432143 14696 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.438655 14646 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:00.444896 14538 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:00.444398 14757 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:00.482986  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:00.488323  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 114878us
I20260709 13:20:00.440342 14797 raft_consensus.cc:2749] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:00.493505  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.494549  7233 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:20:00.496054  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.496855  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.470403 14770 raft_consensus.cc:2804] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:00.504055  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.505524  7237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.505771  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.509895  7234 raft_consensus.cc:1240] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:00.506402  7233 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 19.
I20260709 13:20:00.513322  7198 leader_election.cc:304] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:00.514775  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.515200  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.515329  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.487161 14731 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:00.521157  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.487968 14538 raft_consensus.cc:493] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.518798  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.486536 14646 raft_consensus.cc:493] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.526070  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.532411  6667 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 18.
W20260709 13:20:00.539880  6563 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:00.541615  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:00.542388  6563 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:00.543602  6563 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.543973  6563 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:00.544701 14797 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:00.547822  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.547749 14619 raft_consensus.cc:3055] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:00.548528 14619 raft_consensus.cc:740] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:20:00.549607  6563 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:00.551371  6563 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.555248 14797 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:00.555909 14797 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:00.507962 14422 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.563500  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.569993  6488 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:00.571771  6487 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.517107 14770 raft_consensus.cc:697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:00.468986 14768 consensus_queue.cc:260] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.581573  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:00.536190 14538 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:00.592332  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.535457 14630 raft_consensus.cc:2804] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:00.487804 14696 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:00.595656  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:00.536075 14646 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:00.508524 14784 consensus_queue.cc:1059] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:20:00.598317  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:00.607570  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.608186  6488 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.617028 14516 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:00.619038 14797 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.623065 14642 raft_consensus.cc:2804] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:00.623557 14642 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:00.623919 14642 raft_consensus.cc:3060] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:00.594168 14643 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:00.630198 14643 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:00.631027 14807 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:20:00.631196 14770 consensus_queue.cc:237] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:00.633296 14809 log.cc:927] Time spent T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:00.639416  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a72647203b143708b26072192a9c102" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:00.596961 14779 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:00.597324 14422 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.612201 14019 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:00.591583 14413 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.053s
I20260709 13:20:00.644273 14461 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:00.613184 14630 raft_consensus.cc:697] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:20:00.661906  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 54417.8us
I20260709 13:20:00.681581  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.684080  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:00.666273  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 113904us
I20260709 13:20:00.630586 14643 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:00.579093 14689 consensus_queue.cc:237] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.653213 14718 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:00.639575 14783 consensus_queue.cc:1059] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 13:20:00.677630 14685 raft_consensus.cc:2749] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575) with higher term. Message: Invalid argument: T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 for earlier term 20. Current term is 21.
I20260709 13:20:00.709093  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.711501  6637 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.696671 14422 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:00.675594 14680 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:20:00.732267 14436 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:00.735704 14808 log.cc:927] Time spent T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:20:00.736568 14630 consensus_queue.cc:237] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.736914 14436 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:00.680641 14617 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.705049 14710 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.035s
I20260709 13:20:00.761070  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:00.809070 14436 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.796679 14747 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.815192 14696 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:00.772059 14691 consensus_queue.cc:1048] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:20:00.818969 14660 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:00.821312 14646 raft_consensus.cc:515] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.827459 14660 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:00.841836  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:00.841821  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:00.843152  6662 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
I20260709 13:20:00.846316  6486 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:00.849200  6595 raft_consensus.cc:2468] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
I20260709 13:20:00.884537  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.857563 14436 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:00.884429  6561 leader_election.cc:304] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.859387 14685 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.858206 14538 raft_consensus.cc:515] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.895901  6561 leader_election.cc:304] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:00.891274 14771 raft_consensus.cc:493] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:00.906822  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 145872us
W20260709 13:20:00.924676  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 213350us
I20260709 13:20:00.927507 14610 raft_consensus.cc:2749] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:00.903579 14554 raft_consensus.cc:2749] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:00.928108 14771 raft_consensus.cc:515] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.929644 14685 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.893042 14436 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.907056 14770 raft_consensus.cc:2749] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:00.948161 14819 log.cc:927] Time spent T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:20:00.946823 14660 consensus_queue.cc:1048] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:20:00.952224  6637 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:00.967969  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48649.7us
I20260709 13:20:00.973310  7230 raft_consensus.cc:1240] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:00.974505  6671 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:20:00.974968  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:20:00.977985  6415 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:00.957319 14436 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:00.953933 14824 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:00.983904  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:00.995509  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.019133  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.019835  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:01.024446 14606 consensus_meta.cc:287] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.003s
I20260709 13:20:01.025765 14643 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:00.972790 14731 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:01.051469 14757 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.027583 14771 leader_election.cc:290] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:01.028491  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 48586.5us
I20260709 13:20:01.093462 14696 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:01.093892 14696 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.094213 14619 consensus_queue.cc:260] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.095624 14619 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:01.099789 14830 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.100306 14830 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.105233 14757 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.107062 14768 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:01.107594 14757 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:01.108801 14436 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.102449 14830 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.036319  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:00.964174 14617 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:01.056947 14821 log.cc:927] Time spent T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:20:00.977012 14717 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:01.062817 14820 log.cc:927] Time spent T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:20:01.064011 14685 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:01.112394  6415 catalog_manager.cc:5697] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 23, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 23 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:01.068640 14747 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.113669 14747 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.094238 14696 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:01.115845 14747 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:01.052094 14554 consensus_queue.cc:1059] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:20:01.116925 14554 raft_consensus.cc:3055] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:01.117286 14554 raft_consensus.cc:740] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:01.050164 14777 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:20:01.118659  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.076704  6588 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:20:01.027132  7231 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:01.081092 14824 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.121675 14824 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.082737 14606 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:01.086635 14766 raft_consensus.cc:493] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:01.123111 14766 raft_consensus.cc:515] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.124663 14643 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:01.125357 14766 leader_election.cc:290] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:01.086750 14731 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.118083 14554 consensus_queue.cc:260] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.127292 14554 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:01.127488 14731 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:01.087807 14642 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.054852 14683 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.128301 14683 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.129505 14642 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:01.032775  6671 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:01.015727 14694 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.113943  6415 catalog_manager.cc:5697] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 18 to 21, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 21 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:01.134636 14694 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.136793 14694 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:01.130331 14683 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:01.138921 14642 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.139588 14642 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.071575 14538 leader_election.cc:290] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.142551 14642 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.151263 14696 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.138499  6415 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 21, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 21 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:01.153401  6415 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 22 to 23, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 23 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:01.156112  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.155964  6415 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 23, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 23 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:01.157744  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:01.158330  6662 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:20:01.161142  6563 leader_election.cc:336] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.164562  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.165278  6669 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:20:01.165771  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.167240  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:01.168408 14642 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:01.185734  7200 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:01.186789 14691 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:01.169505  6662 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:20:01.173220 14734 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:01.189646 14734 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.190150 14734 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:01.161626  6563 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:01.193528 14766 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.194198 14646 leader_election.cc:290] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.194044 14766 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.195250 14797 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:01.196462 14766 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:01.180903 14718 raft_consensus.cc:3055] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:01.197180 14718 raft_consensus.cc:740] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:01.198105 14718 consensus_queue.cc:260] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.206321 14718 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:01.213006  6411 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 331) took 29790 ms (client timeout 15000 ms). Trace:
I20260709 13:20:01.215466  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:01.216779  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:01.220554  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.224747  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.224900 14436 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:01.225694  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.275758  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:20:01.239192  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 53538.1us
I20260709 13:20:01.157418  6415 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:01.241350  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:20:01.250355  6563 consensus_peers.cc:597] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:01.278028  6669 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:01.278383  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 303456us
W20260709 13:20:01.279115  6563 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.279191  6637 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.242882  6486 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:01.282941  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9bbf06685e047fdaa1b6bc881dc7a93" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:01.284940  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8885ae897ec84eca940e2aa65f7311b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:01.285681  6662 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:01.292987  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.293823  6593 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:20:01.294306  6415 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:01.295815  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.296469  6593 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:01.297523  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:01.298146  6597 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:20:01.299410  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:01.299758  6664 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:20:01.301367  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:01.300117  7229 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 25.
I20260709 13:20:01.305097  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.311677 14838 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.312239 14838 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.314220  6589 raft_consensus.cc:2468] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:20:01.316241  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:01.319252  6415 catalog_manager.cc:5697] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 22, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 22 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:01.301877  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:01.260735 14630 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.322471 14630 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.326457  7200 leader_election.cc:304] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:01.332226  6662 raft_consensus.cc:2468] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:20:01.294642 14019 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:01.334182  6415 catalog_manager.cc:5697] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 22, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 22 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:01.229890 14734 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.316042 14689 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.297353 14768 consensus_queue.cc:1059] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 13:20:01.338812  6591 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:01.344576  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.336164 14610 raft_consensus.cc:2749] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:01.347527  6415 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 19 to 22, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 22 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:01.352623  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.351856  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:01.359359  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:01.362779  6415 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:01.364182  6663 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:20:01.367525  6662 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:20:01.339442 14646 raft_consensus.cc:2804] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:01.370532  6664 raft_consensus.cc:1240] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:01.369336  6415 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:01.371181  7231 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:20:01.368677  6597 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:20:01.369097 14838 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.356237 14586 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.373764 14586 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.376282 14586 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:01.369274 14019 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:01.378151 14019 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.355257 14689 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.381127 14689 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:01.359707 14827 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.382081 14827 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.385232 14827 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.355921 14747 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:01.386397 14747 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.389458 14747 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.355588 14762 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.390821 14762 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.391811 14734 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.393113 14696 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:01.393532  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:01.395022  6488 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.395838 14646 raft_consensus.cc:697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:20:01.396186  6488 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.397121 14436 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:01.396883 14646 consensus_queue.cc:237] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.396932  6415 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 19, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 19 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:01.398499  6415 catalog_manager.cc:5697] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 19, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 19 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:01.400130  6415 catalog_manager.cc:5697] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 13 to 22, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 22 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:01.393944 14762 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.405062  6488 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:20:01.419435  5410 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 6547)
I20260709 13:20:01.441277 14619 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:01.440950  6415 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 14, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 14 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:01.442770  6415 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:01.396966 14461 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:20:01.263314  6411 rpcz_store.cc:269] 0709 13:19:31.422495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.422809 (+   314us) service_pool.cc:224] Handling call
0709 13:19:33.262234 (+1839425us) spinlock_profiling.cc:243] Waited 57.1 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8feb090 00007f04b9089f3a 00007f04bee401cc 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc 00007f04bd3fe392 00007f04b73bc48c 00007f04b73bb8c6
0709 13:20:01.212904 (+27950670us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:56.675249 (+     0us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f91455 00007f04bcab616c
0709 13:19:56.999257 (+324008us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:57.113412 (+114155us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:57.113483 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:19:57.113527 (+    44us) tablet.cc:662] Decoding operations
0709 13:19:57.115532 (+  2005us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:57.115654 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:57.115677 (+    23us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:19:57.160102 (+ 44425us) tablet.cc:701] Row locks acquired
0709 13:19:57.160164 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:19:57.160393 (+   229us) write_op.cc:270] Start()
0709 13:19:57.160723 (+   330us) write_op.cc:276] Timestamp: P: 1783603197160313 usec, L: 0
0709 13:19:57.160777 (+    54us) op_driver.cc:348] REPLICATION: starting
0709 13:19:57.161844 (+  1067us) log.cc:844] Serialized 8237 byte log entry
0709 13:19:57.557737 (+395893us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b5f770ac 00007f04bedd1ab9 00007f04b5f9369b 00007f04b5f9193f 00007f04b5f93c02 00007f04b8c1b20b
0709 13:19:58.447375 (+889638us) op_driver.cc:464] REPLICATION: finished
0709 13:19:58.734993 (+287618us) write_op.cc:301] APPLY: starting
0709 13:19:58.735087 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:59.298808 (+563721us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:59.335464 (+ 36656us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b54001ac554. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b8fea571 00007f04b907e32a 00007f04b9003f14 00007f04b9003ec3 00007f04b9003e38 00007f04b9003df3 00007f04b90030dc 00007f04b902ce16 00007f04b9014cbb 00007f04b8fff39d 00007f04b9002d75
0709 13:19:59.338662 (+  3198us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:59.636490 (+297828us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:59.636897 (+   407us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 13:19:59.636955 (+    58us) write_op.cc:312] APPLY: finished
0709 13:19:59.637416 (+   461us) log.cc:844] Serialized 188 byte log entry
0709 13:19:59.921916 (+284500us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:59.941326 (+ 19410us) write_op.cc:454] Released schema lock
0709 13:19:59.950290 (+  8964us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:00.014336 (+ 64046us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:00.014369 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1097161,"reported_tablets":44,"spinlock_wait_cycles":131399040,"child_traces":[["op",{"apply.queue_time_us":255246,"apply.run_cpu_time_us":63165,"apply.run_wall_time_us":1279309,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91865,"num_ops":21,"prepare.queue_time_us":319638,"prepare.run_cpu_time_us":31582,"prepare.run_wall_time_us":862796,"raft.queue_time_us":179340,"raft.run_cpu_time_us":41678,"raft.run_wall_time_us":946745,"replication_time_us":1259296,"spinlock_wait_cycles":66836480,"thread_start_us":1265573,"threads_started":5,"wal-append.queue_time_us":1191902,"wal-append.run_cpu_time_us":25283,"wal-append.run_wall_time_us":2352760}]]}
I20260709 13:20:01.397006 14459 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:01.444330  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fb01da2cc5490ebbbca7ec57b64a20" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.459556  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.460229  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.424283  6663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35df31b6c7b2475cac55c1e114ba24dc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.445290  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:01.460846  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.466536  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:01.468778  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 115540us
I20260709 13:20:01.450703  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:01.451094  6669 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:20:01.455384  6662 raft_consensus.cc:1275] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:20:01.459336  6415 catalog_manager.cc:5697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 19, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 19 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:01.479956  6415 catalog_manager.cc:5697] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:01.477234  7198 leader_election.cc:304] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:20:01.481808  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.400893 14630 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.490296  6666 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 25: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:20:01.484159  7200 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:01.493417  7200 leader_election.cc:336] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.493920  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:01.486533  6563 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:01.497900  7200 leader_election.cc:336] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.498445  7200 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:01.499282 14838 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:01.503240  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:01.503639  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:01.552387  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 75256.1us
I20260709 13:20:01.489521 14685 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:20:01.564808  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 60603.8us
W20260709 13:20:01.565680  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 62740.9us
I20260709 13:20:01.482200 14785 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:01.574498  6415 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 19, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 19 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:01.575928  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 52319.5us
I20260709 13:20:01.576781  6488 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:01.577780 14766 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:01.578446 14422 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:20:01.490836 14617 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 13:20:01.582788 14832 raft_consensus.cc:3055] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:20:01.591092  6637 leader_election.cc:336] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.592634 14019 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:01.600924 14832 raft_consensus.cc:740] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:01.601257 14019 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:01.602082 14019 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.603029  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:01.604768  6637 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.595562  6415 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 19, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 19 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:01.598569 14422 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:01.606565  6637 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.607307 14696 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:01.607944 14422 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.601912 14832 consensus_queue.cc:260] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:01.593041  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:01.616109 14619 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:01.620174 14777 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.621227  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:01.609479  6415 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:01.625871  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.571990 14630 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:01.576402 14681 raft_consensus.cc:2804] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:01.621598 14617 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:01.637527  6488 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.662825  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 86108.2us
I20260709 13:20:01.663729  6667 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:01.664727  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 68983.4us
I20260709 13:20:01.637086  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:01.663722  6415 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 18 to 19, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 19 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:01.663781  7200 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.667946  7200 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:01.666221  6415 catalog_manager.cc:5697] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:01.571568 14685 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.636736  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:20:01.650245 14851 log.cc:927] Time spent T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.073s	user 0.002s	sys 0.000s
W20260709 13:20:01.678223  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 71698.3us
W20260709 13:20:01.679564  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.679971  7231 raft_consensus.cc:1240] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:20:01.680183  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:01.682569  6637 leader_election.cc:336] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.686655  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:01.689282  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:01.690608  7228 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:01.695653  7235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:01.684845  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:01.701290  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:01.673187 14685 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:01.709371  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:01.711882  6667 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:20:01.678395 14681 raft_consensus.cc:697] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:20:01.679662 14661 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:01.713845  6561 leader_election.cc:304] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.691546 14840 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:01.685552 14696 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:01.677799 14747 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:01.670650 14617 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.712604 14832 raft_consensus.cc:3060] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:01.675675 14777 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.715168 14694 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:01.719781 14554 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:01.713043 14829 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.733631  7231 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:01.710464 14766 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:20:01.746987  6486 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:01.755309  6487 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.759816  6487 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.760926  6487 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.734131 14661 raft_consensus.cc:2749] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:01.765857 14834 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:01.766539 14834 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:01.767221 14834 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:20:01.774892 14681 consensus_queue.cc:237] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.775228  7228 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 20.
I20260709 13:20:01.776149 14617 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.780073 14777 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:01.782699 14785 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:01.783296 14785 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:01.784205 14785 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.796275  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.796387  6666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:01.746412 14829 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.872296  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:20:01.875571  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 161809us
I20260709 13:20:01.876542  6561 leader_election.cc:304] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.866590 14718 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:01.875114  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 160849us
I20260709 13:20:01.889463  6666 raft_consensus.cc:3060] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:01.879534 14696 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.895339  6637 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:01.875068 14691 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:20:01.918205 14771 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:20:01.921238  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.921403  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:01.922703  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:01.945138  6671 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:01.949286  6588 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
W20260709 13:20:01.950114  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54842.7us
I20260709 13:20:01.955897  6667 raft_consensus.cc:1240] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Rejecting Update request from peer 3ceee54e49d041be9f7f48a2052709c5 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:20:01.895174 14538 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:01.892287 14768 raft_consensus.cc:2749] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:01.887863 14661 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:01.972409 14834 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:01.950728 14840 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:01.983155 14661 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.988623  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:01.903398 14554 consensus_queue.cc:1048] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:20:01.891674 14718 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:01.997694  6666 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:01.950204 14771 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:01.989114 14661 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.021159 14829 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.021791 14829 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:20:02.022259 14829 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.024778  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:20:02.040022  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 168140us
I20260709 13:20:02.051000 14771 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:02.053006 14829 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.053735 14784 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:02.060863  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.075018  7229 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:01.954108 14696 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:02.077209  6637 leader_election.cc:336] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.080687  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.091491 14685 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.095243 14685 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.095597  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:02.120483  6666 raft_consensus.cc:1275] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:20:02.123404  6637 leader_election.cc:336] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.125625  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.113390 14718 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.068305 14837 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
I20260709 13:20:02.127390  6591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.083297 14840 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:02.128551  6597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:02.130470  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.133620  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.133760  6590 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 25.
I20260709 13:20:02.126287  6595 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
W20260709 13:20:02.126538  6637 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.125962 14832 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:02.159114  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.159138  6595 raft_consensus.cc:1275] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:20:02.136471  7200 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.163177  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:02.168804  7200 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.169260  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.171386  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4112f607ad043a3889d117a4fb6ca59" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:02.171144  7200 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.129712 14733 consensus_queue.cc:1048] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
W20260709 13:20:02.173596  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 202740us
I20260709 13:20:02.167232 14832 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:02.193615  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.195904  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:02.196520  7200 leader_election.cc:336] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.196991  7200 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.204077  6590 raft_consensus.cc:1240] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 25. Current term is 26. Ops: []
I20260709 13:20:02.206390  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d598d50f158e42a6b8d5e297f271951d" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:02.171710 14554 raft_consensus.cc:493] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:02.172858 14630 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:02.174852 14865 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:02.209103  6666 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:20:02.214344  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:02.214761  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.215477  6637 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:02.215621 14459 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:02.215992 14646 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:02.240048  7200 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:02.241580  7200 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.243834  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.245425  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.242236  7200 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.248741 14694 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:02.249243 14717 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:02.249717 14536 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:20:02.253800  7200 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.258759  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.222819 14554 raft_consensus.cc:515] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.273270  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.259681 14837 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:02.284941  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
W20260709 13:20:02.299625  6563 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.285887  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:02.287411 14862 log.cc:927] Time spent T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:20:02.288160 14832 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:02.302547  6588 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:20:02.302348  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:02.271149 14838 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:02.305558  6593 raft_consensus.cc:1275] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:02.301708  6596 raft_consensus.cc:2468] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 16.
I20260709 13:20:02.281243 14646 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:02.271083 14422 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:20:02.327062  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:02.335232  6488 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:02.344166  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:02.315989 14837 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:02.348011  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.285473 14829 consensus_queue.cc:1059] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
W20260709 13:20:02.354466  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 109117us
I20260709 13:20:02.356580  6561 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:02.355209 14461 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:02.358723  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:02.358651 14680 consensus_queue.cc:1059] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:20:02.359793 14680 raft_consensus.cc:3055] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:20:02.360044 14866 log.cc:927] Time spent T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:20:02.360154 14680 raft_consensus.cc:740] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:02.361027 14680 consensus_queue.cc:260] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.366813 14554 leader_election.cc:290] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.370728 14829 raft_consensus.cc:3055] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:02.371250 14829 raft_consensus.cc:740] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:02.372242 14829 consensus_queue.cc:260] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:02.373744 14829 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:02.374428 14865 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:02.374871 14865 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:02.375252 14865 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:02.381374  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 79130us
I20260709 13:20:02.386065  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.387070  6595 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:20:02.390157  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:02.390726  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17e76729a2fc4273abedd8be9331a882" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:20:02.395393  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 94581.8us
I20260709 13:20:02.347136 14835 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:02.396984  6563 leader_election.cc:336] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.397403  6563 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.343858 14607 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:02.348410 14837 raft_consensus.cc:493] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:02.399425 14837 raft_consensus.cc:3060] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:02.399776 14868 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:02.400525 14868 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:02.403846 14696 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.404997 14413 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.405505 14413 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:02.409314  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 54744.4us
I20260709 13:20:02.362463 14680 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:20:02.415836  6563 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:02.426157  6592 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:02.437335  6562 leader_election.cc:304] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:02.441164  5410 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 6547)
I20260709 13:20:02.446169  6668 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:20:02.419836 14691 consensus_queue.cc:1048] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:20:02.456471 14718 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:02.429672 14784 raft_consensus.cc:493] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:02.459643 14784 raft_consensus.cc:515] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.407461 14838 consensus_queue.cc:1048] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
W20260709 13:20:02.509403  6560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:02.518699  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:02.520562  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 102510us
I20260709 13:20:02.462524 14784 leader_election.cc:290] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:02.523743  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 49259.1us
I20260709 13:20:02.477349 14867 raft_consensus.cc:2749] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:02.529963  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.539214 14019 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:02.539784 14019 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:02.538233  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.550854  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:02.553457  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542a321aae654f82803540bffcf7968f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.567636 14019 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.579440  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.562665 14617 raft_consensus.cc:2804] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:02.584800 14617 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.585352 14617 raft_consensus.cc:3060] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:02.585071  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.588872  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:02.562952 14459 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:20:02.597360 14413 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:02.601702  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:02.613099 14876 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:20:02.618649  6563 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.622037  6563 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.625914 14837 raft_consensus.cc:515] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.627875 14617 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.629099 14797 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:02.584134 14689 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
W20260709 13:20:02.636888  6563 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.645234  6563 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:20:02.638119  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 69122.6us
I20260709 13:20:02.647323 14554 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:02.649569 14617 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.669628  7232 raft_consensus.cc:2410] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:20:02.670480 14837 leader_election.cc:290] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:02.677198 14413 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:02.679162  6487 leader_election.cc:304] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:02.657922  6563 leader_election.cc:336] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.680265  6563 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.680734  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
W20260709 13:20:02.681522  6563 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.681962  6563 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.679224  7228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eb5773be2294301bdb1bbedf5765666" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.679507  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.683423  7230 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 21.
W20260709 13:20:02.644345 14884 log.cc:927] Time spent T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.069s	user 0.001s	sys 0.003s
W20260709 13:20:02.652812 14881 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:20:02.692373  6563 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.692998  6563 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.694478  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:02.702562  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:02.677794 14413 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.707041 14619 raft_consensus.cc:2749] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:02.707609 14413 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.698758 14867 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:02.708352 14660 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:02.708887 14859 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:20:02.715361 14536 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:02.716143 14536 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.716815  6487 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:02.718077  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:20:02.720359  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.720984 14617 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:02.719597  6589 raft_consensus.cc:2468] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 25.
I20260709 13:20:02.721935 14617 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:02.722595 14617 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.723213 14617 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:02.734514  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.748059 14617 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.662307 14865 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.750685  7229 raft_consensus.cc:1240] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Rejecting Update request from peer 921c423406da4f289884afca51ca198f for earlier term 23. Current term is 24. Ops: []
I20260709 13:20:02.753845  6593 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 13:20:02.754559 14865 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:02.754580  6593 pending_rounds.cc:85] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Aborting all ops after (but not including) 13
I20260709 13:20:02.755132  6593 pending_rounds.cc:107] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:20:02.755434  6593 raft_consensus.cc:2889] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:02.757515 14617 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Requested vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.759313 14536 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.784097  6636 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:02.786732  6671 raft_consensus.cc:1275] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:02.787245  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:02.788311 14835 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:02.788969 14835 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:02.790581 14875 raft_consensus.cc:2749] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:02.791940 14459 consensus_queue.cc:1059] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:02.793222 14413 raft_consensus.cc:3055] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:20:02.793620 14413 raft_consensus.cc:740] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:02.793995  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.794780  7233 raft_consensus.cc:2393] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:20:02.794464 14413 consensus_queue.cc:260] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.796000 14413 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:02.798962  6637 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.801743  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c0a546cb4d749bc83ffa9cbd0f40297" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:02.802649  6668 raft_consensus.cc:2393] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 25: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
W20260709 13:20:02.805918  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.807734  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:02.807853  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53f3e5f2d81c4952b6ae972bdb3a18c1" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.808454  7232 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:02.813097  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.813910  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:02.814519  6662 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:20:02.821527  6487 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:02.829988  7232 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:20:02.830629  6662 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 22.
I20260709 13:20:02.832504  6487 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
W20260709 13:20:02.837869  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.838714  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:02.846045  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:02.850862 14784 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.851666 14784 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:02.853212 14889 consensus_queue.cc:1048] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
I20260709 13:20:02.854571 14784 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:02.861770  7200 leader_election.cc:304] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca
I20260709 13:20:02.859895  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:02.864279 14586 raft_consensus.cc:2804] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:02.860198 14461 raft_consensus.cc:493] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.864818 14586 raft_consensus.cc:697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 25 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:20:02.865510  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.865324 14461 raft_consensus.cc:515] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.865875 14586 consensus_queue.cc:237] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.827596 14718 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:02.867219 14459 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:02.867399 14536 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:02.867645 14461 leader_election.cc:290] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:02.882598  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.896247 14459 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
W20260709 13:20:02.897459  6486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 64356.3us
I20260709 13:20:02.897244 14459 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.888693  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:02.911561 14784 raft_consensus.cc:493] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:02.919091 14784 raft_consensus.cc:515] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.921165 14784 leader_election.cc:290] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:02.925549  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c84605b85a2f4c4b839274e0724ef52b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:02.940407  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:02.945771  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:02.949084 14835 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.972980  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:02.976203  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4f0dad199224b8182ae036d97281b72" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:02.950755 14630 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:02.970794 14835 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:02.993408  6486 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:03.003283 14413 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:03.029359 14835 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:03.039954  6486 leader_election.cc:304] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:03.041137 14617 raft_consensus.cc:2749] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:03.052078  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:03.055871  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:03.063997  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c870451bc2945e8b0a88e8fedf73ce6" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:03.068413  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:20:03.104224  7200 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 138 similar messages]
I20260709 13:20:03.105687  6487 leader_election.cc:304] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:03.107195  6488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 47204.7us
I20260709 13:20:03.107823  6488 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:03.108361 14680 raft_consensus.cc:2749] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:03.139302 14901 log.cc:927] Time spent T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: Append to log took a long time: real 0.152s	user 0.003s	sys 0.000s
I20260709 13:20:03.132376 14785 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:03.147595  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:03.179785 14834 raft_consensus.cc:493] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.180841 14834 raft_consensus.cc:515] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:03.195050  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:03.195596  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce959bc5e0b4481abad1a3bef3085967" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:03.197512  6488 leader_election.cc:304] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:03.198139 14834 leader_election.cc:290] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:03.199218 14777 raft_consensus.cc:2749] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:20:03.205183 14834 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:03.205904 14834 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:03.208868 14834 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:03.209371  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:03.215978  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389b5012cd8548a099f7dec661e47e4f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:03.218454  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:03.231184  6488 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:03.232988 14904 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:03.235961  6413 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.276904  6413 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 14 to 15, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 15 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:03.306317  6415 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35466 (request call id 336) took 9091 ms. Trace:
I20260709 13:20:03.321561 14777 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.322081  6671 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:20:03.322985 14777 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:03.323958 14691 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:20:03.326284  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:03.326648  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:03.328558  6486 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:03.329433 14834 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:03.332712 14777 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:03.325729  6415 rpcz_store.cc:276] 0709 13:19:54.214857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.215150 (+   293us) service_pool.cc:224] Handling call
0709 13:20:03.306233 (+9091083us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:02.198755 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:02.248273 (+ 49518us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:02.248314 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:20:02.248339 (+    25us) tablet.cc:662] Decoding operations
0709 13:20:02.250396 (+  2057us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:02.250464 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:02.250499 (+    35us) tablet.cc:685] Acquiring locks for 28 operations
0709 13:20:02.289594 (+ 39095us) tablet.cc:701] Row locks acquired
0709 13:20:02.289610 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:20:02.289787 (+   177us) write_op.cc:270] Start()
0709 13:20:02.289956 (+   169us) write_op.cc:276] Timestamp: P: 1783603202289716 usec, L: 0
0709 13:20:02.289977 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:20:02.395817 (+105840us) log.cc:844] Serialized 10949 byte log entry
0709 13:20:02.650919 (+255102us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b4400022b4c. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5ebac85 00007f04b5ebb6a2 00007f04b8c2d1db 00007f04b8c28aaf 00007f04b8c1c86e 00007f04b8c1bbb6 00007f04b8c1b7d8 00007f04b8c25221 00007f04b8c251d9 00007f04b8c25169
0709 13:20:02.763598 (+112679us) op_driver.cc:464] REPLICATION: finished
0709 13:20:02.782762 (+ 19164us) write_op.cc:301] APPLY: starting
0709 13:20:02.782846 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:02.797086 (+ 14240us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:02.797115 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:03.061394 (+264279us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18003e1880. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04bee5cde1 00007f04bee695b1 00007f04b8dc00ec 00007f04b8e0cc0a 00007f04b8e0c4f7 00007f04b8e0c490 00007f04b8dc3f48 00007f04bcad81bb 00007f04bcb1f5d4 00007f04bc9cb8da 00007f04bc9ce3cb
0709 13:20:03.147060 (+ 85666us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:03.147446 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=0,mrs_lookups=0
0709 13:20:03.147496 (+    50us) write_op.cc:312] APPLY: finished
0709 13:20:03.148122 (+   626us) log.cc:844] Serialized 244 byte log entry
0709 13:20:03.148300 (+   178us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:03.148539 (+   239us) write_op.cc:454] Released schema lock
0709 13:20:03.148611 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:03.149053 (+   442us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:03.149092 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":566108,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":18876,"apply.run_cpu_time_us":40864,"apply.run_wall_time_us":366247,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121111,"num_ops":28,"prepare.queue_time_us":311054,"prepare.run_cpu_time_us":13315,"prepare.run_wall_time_us":252378,"raft.queue_time_us":26984,"raft.run_cpu_time_us":17371,"raft.run_wall_time_us":64463,"replication_time_us":473397,"spinlock_wait_cycles":31160832,"thread_start_us":246941,"threads_started":4,"wal-append.queue_time_us":141833}]]}
I20260709 13:20:03.336097  6670 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:20:03.340231 14777 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:20:03.351013 14617 raft_consensus.cc:493] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 64b95375974b48d18c596a23c9cac75c)
I20260709 13:20:03.351624 14617 raft_consensus.cc:515] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:03.355350  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:03.356503 14617 leader_election.cc:290] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:03.358407  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b7cf612223418ea8fe2d636986bb3f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:03.363648  6486 leader_election.cc:304] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:03.367090 14834 raft_consensus.cc:2749] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:03.385408  6594 raft_consensus.cc:1275] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:20:03.400754 14630 consensus_queue.cc:1048] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.012s
I20260709 13:20:03.417989 14660 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.430280 14660 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:03.432503 14660 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:03.434188  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:03.434923  6670 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:20:03.437170  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:20:03.441800  5410 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 6547)
I20260709 13:20:03.443022 14859 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:03.443645 14859 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.444020 14859 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:03.448041  6563 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.490430  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:03.503608 14859 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:03.509867  6563 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.528801 14859 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:03.531131  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:03.531747  6671 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:03.533929 14785 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:03.534471 14785 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:03.539158  7232 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:20:03.539403  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:03.538522  6671 raft_consensus.cc:2468] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:20:03.540486  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:03.541272  6561 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:03.541565  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c3ba394f290437b90045c4a98fb75ca" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:03.542591 14859 raft_consensus.cc:2804] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:03.543220 14859 raft_consensus.cc:697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:03.543460 14617 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:03.544415 14859 consensus_queue.cc:237] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:03.546259  6487 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:03.547227 14777 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:03.538511 14785 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:03.580729  6412 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.591456 14617 raft_consensus.cc:493] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
W20260709 13:20:03.589850  6413 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 330) took 54036 ms (client timeout 15000 ms). Trace:
I20260709 13:20:03.592163 14617 raft_consensus.cc:515] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:03.595397  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:03.592199  6413 rpcz_store.cc:269] 0709 13:19:09.553708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.553996 (+   288us) service_pool.cc:224] Handling call
0709 13:20:03.354968 (+53800972us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:03.354999 (+    31us) write_op.cc:454] Released schema lock
0709 13:20:03.589746 (+234747us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:03.335793 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:03.336372 (+   579us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:03.336408 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:20:03.336426 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:03.336806 (+   380us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:03.336895 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:03.336920 (+    25us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:03.337239 (+   319us) tablet.cc:701] Row locks acquired
0709 13:20:03.337258 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:20:03.337389 (+   131us) write_op.cc:270] Start()
0709 13:20:03.337498 (+   109us) write_op.cc:276] Timestamp: P: 1783603203337342 usec, L: 0
0709 13:20:03.337523 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:20:03.338151 (+   628us) log.cc:844] Serialized 913 byte log entry
0709 13:20:03.340275 (+  2124us) op_driver.cc:464] REPLICATION: finished
0709 13:20:03.340721 (+   446us) write_op.cc:301] APPLY: starting
0709 13:20:03.340797 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:03.342044 (+  1247us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:03.342138 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:03.343420 (+  1282us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:03.343565 (+   145us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:20:03.343605 (+    40us) write_op.cc:312] APPLY: finished
0709 13:20:03.343778 (+   173us) log.cc:844] Serialized 33 byte log entry
0709 13:20:03.343950 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:03.344034 (+    84us) write_op.cc:454] Released schema lock
0709 13:20:03.344071 (+    37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":85897,"reported_tablets":44,"spinlock_wait_cycles":2816,"child_traces":[["op",{"apply.queue_time_us":223,"apply.run_cpu_time_us":3618,"apply.run_wall_time_us":3632,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8356,"num_ops":2,"prepare.queue_time_us":45665,"prepare.run_cpu_time_us":2670,"prepare.run_wall_time_us":2671,"replication_time_us":2620,"thread_start_us":64348,"threads_started":1}]]}
I20260709 13:20:03.598960 14617 leader_election.cc:290] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:03.599860  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbaf7daabfe40028228e412b8f9214b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:03.607780  6487 leader_election.cc:304] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:03.609279 14617 raft_consensus.cc:2749] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:03.670794  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:03.696822  6415 catalog_manager.cc:5697] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 22 to 24, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 24 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.702241  6415 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:03.717918  6415 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:03.719295  6415 catalog_manager.cc:5697] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 23 to 28, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 28 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:03.720963  6415 catalog_manager.cc:5697] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.722177  6415 catalog_manager.cc:5697] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 23, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 23 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.723639  6415 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.724814  6415 catalog_manager.cc:5697] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 12 to 20, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 20 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:03.738827  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:03.726082  6415 catalog_manager.cc:5697] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 17, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 17 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:03.752336  6412 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 254) took 33426 ms (client timeout 15000 ms). Trace:
I20260709 13:20:03.755092  6415 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:03.756697  6415 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 18 to 21, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 21 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
W20260709 13:20:03.752828  6412 rpcz_store.cc:269] 0709 13:19:30.326444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.331049 (+  4605us) service_pool.cc:224] Handling call
0709 13:20:03.752258 (+33421209us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:03.597898 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:03.598423 (+   525us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:03.598454 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:20:03.598471 (+    17us) tablet.cc:662] Decoding operations
0709 13:20:03.598808 (+   337us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:03.598860 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:03.598886 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:03.599075 (+   189us) tablet.cc:701] Row locks acquired
0709 13:20:03.599093 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:20:03.599239 (+   146us) write_op.cc:270] Start()
0709 13:20:03.599354 (+   115us) write_op.cc:276] Timestamp: P: 1783603203599190 usec, L: 0
0709 13:20:03.599376 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:20:03.599979 (+   603us) log.cc:844] Serialized 533 byte log entry
0709 13:20:03.601925 (+  1946us) op_driver.cc:464] REPLICATION: finished
0709 13:20:03.602367 (+   442us) write_op.cc:301] APPLY: starting
0709 13:20:03.602447 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:03.603314 (+   867us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:03.603416 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:03.604354 (+   938us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:03.604563 (+   209us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:20:03.604612 (+    49us) write_op.cc:312] APPLY: finished
0709 13:20:03.604816 (+   204us) log.cc:844] Serialized 25 byte log entry
0709 13:20:03.604974 (+   158us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:03.605056 (+    82us) write_op.cc:454] Released schema lock
0709 13:20:03.605095 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:03.605427 (+   332us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:03.605458 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":417480,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":233,"apply.run_cpu_time_us":3010,"apply.run_wall_time_us":3022,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":3397,"prepare.run_cpu_time_us":2375,"prepare.run_wall_time_us":2376,"replication_time_us":2433}]]}
I20260709 13:20:03.758185  6415 catalog_manager.cc:5697] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 22 to 23, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 23 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.776988 14586 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:03.777643 14586 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:03.779696 14586 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:03.781567  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:03.782395  6588 raft_consensus.cc:2393] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 26: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:20:03.783912  6415 catalog_manager.cc:5697] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 19 to 22, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 22 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.785998  6415 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 23, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 23 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:03.787633  7200 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.787722 14617 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:03.793920 14617 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:03.800571 14617 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:03.801553  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:03.802898  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:03.815832  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85490b85caca40d19a25af108b1da46a" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:03.813874 14630 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:03.807060 14617 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.835124  6487 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:03.842818  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:03.833767  6415 catalog_manager.cc:5697] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 19, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 19 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.846748 14422 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:03.847016  6415 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:03.846215 14617 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:03.848351  6415 catalog_manager.cc:5697] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 14 to 16, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 16 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.850313  6415 catalog_manager.cc:5697] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 17 to 24, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 24 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.851174  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:03.850587 14617 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:03.851945  6415 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 11 to 18, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 18 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.853920  6415 catalog_manager.cc:5697] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } }
I20260709 13:20:03.855229  6415 catalog_manager.cc:5697] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.857056  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc1d5a094e64eadb9c93f5e81f30fbc" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:03.856634  6415 catalog_manager.cc:5697] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 18 to 24, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 24 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.857770  6671 raft_consensus.cc:2393] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 921c423406da4f289884afca51ca198f in current term 23: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:20:03.858948  6486 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:03.858162  6415 catalog_manager.cc:5697] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 19 to 21, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 21 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.860354  6415 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 16 to 20, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.862109  6415 catalog_manager.cc:5697] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 19 to 22, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 22 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.864892  6415 catalog_manager.cc:5697] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:03.866808 14777 raft_consensus.cc:3060] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:03.873466 14630 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:03.866199  6415 catalog_manager.cc:5697] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 15 to 24, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 24 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.874589 14777 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:03.875594  6415 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.899884 14422 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.904115 14422 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:03.907558 14422 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:03.908550  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:03.908730  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23ba4e226865438d85ee0c061786f3db" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:03.924578  6488 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:03.931609 14422 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:03.952718  6563 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:03.967996  6662 raft_consensus.cc:1275] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:20:03.969964 14797 consensus_queue.cc:1048] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:04.126175  6416 catalog_manager.cc:5697] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 18 to 19, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 19 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:04.128592  6416 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 17 to 21, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 21 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.138147  6416 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 18 to 19, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 19 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.141141  6416 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 15 to 17, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 17 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.155095  6416 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.156710  6416 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 17 to 18, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 18 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.163311  6416 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 13 to 17, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 17 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.165994  6416 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 15 to 17, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 17 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:04.195688  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:04.227090 14422 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.227669 14422 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.230131 14422 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:04.233601  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:04.234279  7229 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 26.
I20260709 13:20:04.243806 14617 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.244417 14617 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.247484 14617 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:04.253755  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:04.254904  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:04.255154  6487 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:04.255501  6588 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 26.
I20260709 13:20:04.254400  6590 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 23.
I20260709 13:20:04.257995  6488 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:04.258566  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:04.259212  7232 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 24.
I20260709 13:20:04.259610 14461 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:20:04.260039 14461 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.260393 14461 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:04.262722 14777 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:04.263207 14777 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.263581 14777 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:04.269063 14461 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.278450  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:04.279058  6590 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:04.294140 14461 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:04.298085  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:04.291944  6590 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 27.
I20260709 13:20:04.299579  6488 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:04.300653 14461 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:20:04.301347 14461 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 27 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:04.302230 14461 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.344470  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:04.345106  7232 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:04.381666  7232 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 27.
I20260709 13:20:04.386559  6415 catalog_manager.cc:5697] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 22 to 25, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 25 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.388314  6415 catalog_manager.cc:5697] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 19 to 23, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 23 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.392818 14777 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.394953 14777 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:04.403678 14617 raft_consensus.cc:493] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.404551 14617 raft_consensus.cc:515] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.414227 14918 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:04.418676 14918 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.420547 14617 leader_election.cc:290] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:04.428550 14918 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:04.442389  5410 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 6547)
W20260709 13:20:04.455881  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:04.467017 14881 log.cc:927] Time spent T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:20:04.498189  6487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 113666us
I20260709 13:20:04.500551  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:04.501449  7231 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:04.511662  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:04.514441  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:20:04.519922  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:04.524870 14785 raft_consensus.cc:493] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.525563 14785 raft_consensus.cc:515] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.530203  7231 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:20:04.533064  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:04.533756  6594 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:20:04.538762 14785 leader_election.cc:290] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:04.566040  6411 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.568296  6411 catalog_manager.cc:5697] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 18 to 20, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 20 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.578974  6411 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.582926  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ba1578b97b4faa966168899769e067" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:04.583242  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6be297bb363c43c99d04cccfc55bec8f" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:04.551551  6487 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f; no voters: 
I20260709 13:20:04.585561  6411 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 17 to 20, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 20 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:04.590062  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:04.592403  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7365731c56504a678dc1a5ceac3f8da4" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:04.594257 14924 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:04.595108 14924 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Becoming Leader. State: Replica: 921c423406da4f289884afca51ca198f, State: Running, Role: LEADER
I20260709 13:20:04.596123 14924 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.606297  6594 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 921c423406da4f289884afca51ca198f in term 25.
I20260709 13:20:04.611433  6488 leader_election.cc:304] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:04.613164  6488 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:04.613343 14777 raft_consensus.cc:2749] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:04.614315  6488 leader_election.cc:304] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c
W20260709 13:20:04.614476  6416 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 334) took 24629 ms (client timeout 15000 ms). Trace:
I20260709 13:20:04.615609 14918 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:04.614924  6416 rpcz_store.cc:269] 0709 13:19:40.044979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.048346 (+  3367us) service_pool.cc:224] Handling call
0709 13:20:04.614405 (+24566059us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:04.179153 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:04.179652 (+   499us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:04.179688 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:20:04.179705 (+    17us) tablet.cc:662] Decoding operations
0709 13:20:04.180507 (+   802us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:04.180563 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:04.180590 (+    27us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:20:04.181238 (+   648us) tablet.cc:701] Row locks acquired
0709 13:20:04.181253 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:20:04.181389 (+   136us) write_op.cc:270] Start()
0709 13:20:04.181499 (+   110us) write_op.cc:276] Timestamp: P: 1783603204181344 usec, L: 0
0709 13:20:04.181517 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:20:04.182168 (+   651us) log.cc:844] Serialized 3190 byte log entry
0709 13:20:04.184399 (+  2231us) op_driver.cc:464] REPLICATION: finished
0709 13:20:04.184910 (+   511us) write_op.cc:301] APPLY: starting
0709 13:20:04.185002 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:04.189188 (+  4186us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:04.189285 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:04.211250 (+ 21965us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:04.211452 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:20:04.211498 (+    46us) write_op.cc:312] APPLY: finished
0709 13:20:04.352740 (+141242us) log.cc:844] Serialized 81 byte log entry
0709 13:20:04.354415 (+  1675us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:04.354532 (+   117us) write_op.cc:454] Released schema lock
0709 13:20:04.354575 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:04.354950 (+   375us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:04.354982 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":677985,"reported_tablets":45,"spinlock_wait_cycles":11136,"child_traces":[["op",{"apply.queue_time_us":315,"apply.run_cpu_time_us":10766,"apply.run_wall_time_us":169970,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33424,"num_ops":8,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":3381,"prepare.run_wall_time_us":3561,"replication_time_us":2776,"spinlock_wait_cycles":30592}]]}
I20260709 13:20:04.627641 14918 raft_consensus.cc:2749] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:04.630923  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 87050.2us
W20260709 13:20:04.665544  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:04.688983  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 46726.4us
I20260709 13:20:04.704022  6594 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:20:04.706234 14916 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:20:04.717325  7229 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:20:04.719374 14925 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 13:20:04.731372  6563 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:04.817605  6411 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 335) took 8573 ms. Trace:
I20260709 13:20:04.817423  6417 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 18 to 19, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 19 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.818318  6411 rpcz_store.cc:276] 0709 13:19:56.243337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.600178 (+5356841us) service_pool.cc:224] Handling call
0709 13:20:04.816604 (+3216426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:04.596102 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:04.596613 (+   511us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:04.596648 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:20:04.596665 (+    17us) tablet.cc:662] Decoding operations
0709 13:20:04.597248 (+   583us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:04.597303 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:04.597332 (+    29us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:20:04.597707 (+   375us) tablet.cc:701] Row locks acquired
0709 13:20:04.597726 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:20:04.597864 (+   138us) write_op.cc:270] Start()
0709 13:20:04.597977 (+   113us) write_op.cc:276] Timestamp: P: 1783603204597815 usec, L: 0
0709 13:20:04.598002 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:20:04.607472 (+  9470us) log.cc:844] Serialized 1674 byte log entry
0709 13:20:04.618451 (+ 10979us) op_driver.cc:464] REPLICATION: finished
0709 13:20:04.618884 (+   433us) write_op.cc:301] APPLY: starting
0709 13:20:04.618970 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:04.621315 (+  2345us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:04.621423 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:04.623972 (+  2549us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:04.624157 (+   185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:20:04.624208 (+    51us) write_op.cc:312] APPLY: finished
0709 13:20:04.624488 (+   280us) log.cc:844] Serialized 49 byte log entry
0709 13:20:04.625541 (+  1053us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:04.625658 (+   117us) write_op.cc:454] Released schema lock
0709 13:20:04.625704 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:04.666781 (+ 41077us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:04.666812 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5441,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":251,"apply.run_cpu_time_us":6399,"apply.run_wall_time_us":21512,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16712,"num_ops":4,"prepare.queue_time_us":971,"prepare.run_cpu_time_us":3136,"prepare.run_wall_time_us":11720,"replication_time_us":20313,"spinlock_wait_cycles":12032}]]}
I20260709 13:20:04.860893 14916 raft_consensus.cc:493] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:04.866834  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:04.861480 14916 raft_consensus.cc:515] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:04.870497  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:04.873953 14916 leader_election.cc:290] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:04.927063  6416 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 23 to 27. New cstate: current_term: 27 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:04.962080  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "004d58f566fd4b6d954bc425675471fa" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:04.980289  6487 leader_election.cc:304] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:04.987537 14777 raft_consensus.cc:2749] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:05.006925 14916 raft_consensus.cc:493] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.013660 14916 raft_consensus.cc:515] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:05.018277 14916 leader_election.cc:290] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:05.035023  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:05.036928  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffa871a3f5334bd49b9c75539f218332" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:05.047040  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 92138.6us
I20260709 13:20:05.051234  6487 leader_election.cc:304] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c
I20260709 13:20:05.053512 14777 raft_consensus.cc:2749] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:05.059695  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:05.066669  6589 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:20:05.081885 14777 raft_consensus.cc:493] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:20:05.082743 14777 raft_consensus.cc:515] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:05.081506 14918 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 13:20:05.092232 14777 leader_election.cc:290] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:05.095175  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:05.106911  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0e818d5c3014869b85a1d5b725c58cf" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:05.113000  7231 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Refusing update from remote peer 921c423406da4f289884afca51ca198f: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:20:05.133703 14617 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
W20260709 13:20:05.134193  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:05.144176  6488 leader_election.cc:304] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:05.145690 14617 raft_consensus.cc:2749] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:05.190564  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:05.213629  6417 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 255) took 15150 ms (client timeout 15000 ms). Trace:
W20260709 13:20:05.214160  6417 rpcz_store.cc:269] 0709 13:19:50.063190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.063437 (+   247us) service_pool.cc:224] Handling call
0709 13:20:04.912280 (+14848843us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:04.912382 (+   102us) write_op.cc:454] Released schema lock
0709 13:20:05.213413 (+301031us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:04.850223 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:04.850835 (+   612us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:04.850868 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:20:04.850881 (+    13us) tablet.cc:662] Decoding operations
0709 13:20:04.851218 (+   337us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:04.851279 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:04.851306 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:04.851483 (+   177us) tablet.cc:701] Row locks acquired
0709 13:20:04.851499 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:20:04.851766 (+   267us) write_op.cc:270] Start()
0709 13:20:04.851894 (+   128us) write_op.cc:276] Timestamp: P: 1783603204851687 usec, L: 0
0709 13:20:04.851913 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:20:04.852618 (+   705us) log.cc:844] Serialized 529 byte log entry
0709 13:20:04.854999 (+  2381us) op_driver.cc:464] REPLICATION: finished
0709 13:20:04.856803 (+  1804us) write_op.cc:301] APPLY: starting
0709 13:20:04.856880 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:04.857667 (+   787us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:04.857746 (+    79us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:04.858437 (+   691us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:04.858592 (+   155us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:20:04.858657 (+    65us) write_op.cc:312] APPLY: finished
0709 13:20:04.858844 (+   187us) log.cc:844] Serialized 25 byte log entry
0709 13:20:04.859012 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:04.859098 (+    86us) write_op.cc:454] Released schema lock
0709 13:20:04.859136 (+    38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":389612,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":1575,"apply.run_cpu_time_us":2601,"apply.run_wall_time_us":2601,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":208,"prepare.run_cpu_time_us":2715,"prepare.run_wall_time_us":2716,"replication_time_us":2946}]]}
W20260709 13:20:05.242650  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:05.269230  6419 catalog_manager.cc:5697] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:05.294797  6418 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35476 (request call id 333) took 47403 ms (client timeout 15000 ms). Trace:
W20260709 13:20:05.295187  6418 rpcz_store.cc:269] 0709 13:19:17.891222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.914355 (+ 23133us) service_pool.cc:224] Handling call
0709 13:20:05.294557 (+47380202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95301,"reported_tablets":45}
I20260709 13:20:05.391353  6419 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 332) took 9639 ms. Trace:
I20260709 13:20:05.393620  6419 rpcz_store.cc:276] 0709 13:19:55.751969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.752262 (+   293us) service_pool.cc:224] Handling call
0709 13:19:55.812209 (+ 59947us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b5400002490. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5cb71 00007f04b5f5d0b2 00007f04beed6a89 00007f04bee6eda6 00007f04bee40088 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc
0709 13:19:55.885878 (+ 73669us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b5400002490. stack: 00007f04b5f62343 00007f04b5f62240 00007f04b58f11d4 00007f04bf62f304 00007f04bf62f299 00007f04b5e1cdb1 00007f04b5f5ce9c 00007f04b5f5d0f6 00007f04beed6a89 00007f04bee6eda6 00007f04bee40088 00007f04bef3e37e 00007f04bd3ff2a4 00007f04bd3ff232 00007f04bd3ff161 00007f04bd3ff0dc
0709 13:20:05.305434 (+9419556us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:05.305462 (+    28us) write_op.cc:454] Released schema lock
0709 13:20:05.391285 (+ 85823us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:05.290430 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:05.290984 (+   554us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:05.291016 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:20:05.291034 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:05.291393 (+   359us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:05.291458 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:05.291486 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:05.291687 (+   201us) tablet.cc:701] Row locks acquired
0709 13:20:05.291701 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:20:05.291855 (+   154us) write_op.cc:270] Start()
0709 13:20:05.291958 (+   103us) write_op.cc:276] Timestamp: P: 1783603205291807 usec, L: 0
0709 13:20:05.291982 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:20:05.292579 (+   597us) log.cc:844] Serialized 533 byte log entry
0709 13:20:05.297522 (+  4943us) op_driver.cc:464] REPLICATION: finished
0709 13:20:05.298170 (+   648us) write_op.cc:301] APPLY: starting
0709 13:20:05.298256 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:05.299152 (+   896us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:05.299253 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:05.299923 (+   670us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:05.300063 (+   140us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:20:05.300134 (+    71us) write_op.cc:312] APPLY: finished
0709 13:20:05.300318 (+   184us) log.cc:844] Serialized 25 byte log entry
0709 13:20:05.300490 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:05.300580 (+    90us) write_op.cc:454] Released schema lock
0709 13:20:05.300621 (+    41us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":31495,"reported_tablets":44,"spinlock_wait_cycles":86758528,"child_traces":[["op",{"apply.queue_time_us":449,"apply.run_cpu_time_us":2724,"apply.run_wall_time_us":2724,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":156,"prepare.run_cpu_time_us":2440,"prepare.run_wall_time_us":2441,"replication_time_us":5427}]]}
W20260709 13:20:05.440589  6420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56974 (request call id 252) took 82940 ms (client timeout 15000 ms). Trace:
W20260709 13:20:05.440999  6420 rpcz_store.cc:269] 0709 13:18:42.500072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.500932 (+   860us) service_pool.cc:224] Handling call
0709 13:20:05.440366 (+82939434us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44527,"reported_tablets":44}
W20260709 13:20:05.460229  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 46140.1us
W20260709 13:20:05.472611  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:05.482684  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:05.485303  5410 tablet_replica.cc:333] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.487118  5410 raft_consensus.cc:2243] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.488286  5410 raft_consensus.cc:2272] T c7d55fddd552409bbf822686f45eda69 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.514469  5410 tablet_replica.cc:333] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f: stopping tablet replica
W20260709 13:20:05.530368  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:05.531376  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:05.530761  5410 raft_consensus.cc:2243] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.537443  5410 raft_consensus.cc:2272] T e15a9820f1e1414f829e7fecece15dc0 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.545560  6419 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:05.568795  6407 catalog_manager.cc:4755] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): ChangeConfig:REMOVE_PEER RPC failed for tablet 938e1826b2ce4babab3a70e5219411a4: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.575273  5410 tablet_replica.cc:333] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.588411  5410 raft_consensus.cc:2243] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.593557  6411 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 23 to 24, leader changed from 64b95375974b48d18c596a23c9cac75c (127.5.72.132) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 24 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:05.599908  6411 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 19 to 20, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:05.602290  6411 catalog_manager.cc:5697] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 19 to 20, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 20 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:05.618558  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:05.597790  6414 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35464 (request call id 329) took 75847 ms (client timeout 15000 ms). Trace:
W20260709 13:20:05.623848  6414 rpcz_store.cc:269] 0709 13:18:49.750667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.751852 (+  1185us) service_pool.cc:224] Handling call
0709 13:20:05.597576 (+75845724us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1737719,"reported_tablets":44}
I20260709 13:20:05.604251  6411 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:05.630339  5410 raft_consensus.cc:2272] T 520b1aa490ee4869a2814c313fac4a0e P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:05.631157  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:05.633635  5410 tablet_replica.cc:333] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.641947 14924 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.642832 14924 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:05.645411 14924 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:05.628229  6411 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 17 to 22, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 921c423406da4f289884afca51ca198f (127.5.72.129). New cstate: current_term: 22 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:05.646422  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:05.647111  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98ecd701e924e049261801a6e8b1e3b" candidate_uuid: "921c423406da4f289884afca51ca198f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:05.648686  5410 raft_consensus.cc:2243] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.651629  5410 raft_consensus.cc:2272] T 542a321aae654f82803540bffcf7968f P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.652343  6488 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 921c423406da4f289884afca51ca198f; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:05.653321 14924 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:05.650400  6411 catalog_manager.cc:5697] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 20 to 21, leader changed from 53fee9894731461a821a131be24dbbca (127.5.72.131) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 21 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:05.655779  5410 tablet_replica.cc:333] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.656989  5410 raft_consensus.cc:2243] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.658429  5410 raft_consensus.cc:2272] T d83f8b945cd641db90647da0461be4a3 P 921c423406da4f289884afca51ca198f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.659819  6411 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "921c423406da4f289884afca51ca198f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:05.676841  5410 tablet_replica.cc:333] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.687188  5410 raft_consensus.cc:2243] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.688467  5410 raft_consensus.cc:2272] T f67b820295ac47a4b54e21400f8b4201 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.691710  5410 tablet_replica.cc:333] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.692875  5410 raft_consensus.cc:2243] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.693913  5410 pending_rounds.cc:70] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Trying to abort 1 pending ops.
I20260709 13:20:05.694160  5410 pending_rounds.cc:77] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f: Aborting op as it isn't in flight: id { term: 23 index: 17 } timestamp: 7305638692646432768 op_type: NO_OP noop_request { }
I20260709 13:20:05.694564  5410 raft_consensus.cc:2889] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:05.723577  5410 raft_consensus.cc:2272] T 23ba4e226865438d85ee0c061786f3db P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.727054  5410 tablet_replica.cc:333] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.727986  5410 raft_consensus.cc:2243] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.729290  5410 raft_consensus.cc:2272] T 34cabfbde16f4e10affcaec358aa0dd4 P 921c423406da4f289884afca51ca198f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.732357  5410 tablet_replica.cc:333] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.733450  5410 raft_consensus.cc:2243] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.734690  5410 raft_consensus.cc:2272] T b2c67ec2db9643088bc65fe9d83aa732 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.737933  5410 tablet_replica.cc:333] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.739068  5410 raft_consensus.cc:2243] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.741109  5410 raft_consensus.cc:2272] T 938e1826b2ce4babab3a70e5219411a4 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.744500  5410 tablet_replica.cc:333] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.745446  5410 raft_consensus.cc:2243] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.746665  5410 raft_consensus.cc:2272] T 9fc18602dfbd4fc9897b6b696b497f73 P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.752681  5410 tablet_replica.cc:333] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.755127  5410 raft_consensus.cc:2243] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.756227  5410 raft_consensus.cc:2272] T b75fb632ebbc415092b08b38d0835144 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.776578  5410 tablet_replica.cc:333] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.777578  5410 raft_consensus.cc:2243] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.780299  5410 raft_consensus.cc:2272] T 91d990297ed0410d894aa69d60271fe7 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.798949  5410 tablet_replica.cc:333] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.799937  5410 raft_consensus.cc:2243] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.801138  5410 raft_consensus.cc:2272] T e1a87665f0ba4296b0ba819fcb712842 P 921c423406da4f289884afca51ca198f [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:05.802999  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:05.804395  5410 tablet_replica.cc:333] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.805538  5410 raft_consensus.cc:2243] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.806181  5410 pending_rounds.cc:70] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Trying to abort 1 pending ops.
I20260709 13:20:05.806411  5410 pending_rounds.cc:77] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f: Aborting op as it isn't in flight: id { term: 24 index: 19 } timestamp: 7305638682149965824 op_type: NO_OP noop_request { }
I20260709 13:20:05.843047  5410 raft_consensus.cc:2889] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:05.843600  5410 raft_consensus.cc:2272] T 6eb5773be2294301bdb1bbedf5765666 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.846798  5410 tablet_replica.cc:333] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.847630  5410 raft_consensus.cc:2243] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.848394  5410 raft_consensus.cc:2272] T 2cbaf7daabfe40028228e412b8f9214b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.851419  5410 tablet_replica.cc:333] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.863649  5410 raft_consensus.cc:2243] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.864557  5410 raft_consensus.cc:2272] T c84605b85a2f4c4b839274e0724ef52b P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.867944  5410 tablet_replica.cc:333] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.868887  5410 raft_consensus.cc:2243] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.869706  5410 raft_consensus.cc:2272] T f4112f607ad043a3889d117a4fb6ca59 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.873860  5410 tablet_replica.cc:333] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f: stopping tablet replica
W20260709 13:20:05.879840  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:05.880409  5410 raft_consensus.cc:2243] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.881774  5410 raft_consensus.cc:2272] T f8a48eeccdac45838ef8456e5e9332b5 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.884877  5410 tablet_replica.cc:333] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.895371  5410 raft_consensus.cc:2243] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.896126  5410 pending_rounds.cc:70] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Trying to abort 1 pending ops.
I20260709 13:20:05.896406  5410 pending_rounds.cc:77] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f: Aborting op as it isn't in flight: id { term: 23 index: 14 } timestamp: 7305638682959290368 op_type: NO_OP noop_request { }
I20260709 13:20:05.896845  5410 raft_consensus.cc:2889] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:05.897279  5410 raft_consensus.cc:2272] T 389b5012cd8548a099f7dec661e47e4f P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.900247  5410 tablet_replica.cc:333] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.911715  5410 raft_consensus.cc:2243] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.913198  5410 raft_consensus.cc:2272] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.925369  5410 tablet_replica.cc:333] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.926314  5410 raft_consensus.cc:2243] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.927062  5410 raft_consensus.cc:2272] T c0e818d5c3014869b85a1d5b725c58cf P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.930189  5410 tablet_replica.cc:333] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.936916  5410 raft_consensus.cc:2243] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:05.938501  5410 pending_rounds.cc:70] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Trying to abort 1 pending ops.
I20260709 13:20:05.938889  5410 pending_rounds.cc:77] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f: Aborting op as it isn't in flight: id { term: 15 index: 13 } timestamp: 7305638699723042816 op_type: NO_OP noop_request { }
I20260709 13:20:05.939343  5410 raft_consensus.cc:2889] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:05.939798  5410 raft_consensus.cc:2272] T 56ba1578b97b4faa966168899769e067 P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.944650  5410 tablet_replica.cc:333] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.949756  5410 raft_consensus.cc:2243] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.951083  5410 raft_consensus.cc:2272] T bab5f01bd4774a168d3c01940a615cce P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:05.951807  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:05.954293  5410 tablet_replica.cc:333] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.964998  5410 raft_consensus.cc:2243] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.966352  5410 raft_consensus.cc:2272] T ff27e205e84e4edf946bc02c573a2c25 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.969511  5410 tablet_replica.cc:333] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.972641  5410 raft_consensus.cc:2243] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:05.973748  5410 raft_consensus.cc:2272] T 0d4a8b90b128485fa69bc10112de24e9 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:05.980309  5410 tablet_replica.cc:333] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:05.999513  5410 raft_consensus.cc:2243] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.003365  5410 raft_consensus.cc:2272] T 85490b85caca40d19a25af108b1da46a P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.008800  5410 tablet_replica.cc:333] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.009742  5410 raft_consensus.cc:2243] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.010501  5410 raft_consensus.cc:2272] T 1c3ba394f290437b90045c4a98fb75ca P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.013635  5410 tablet_replica.cc:333] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.014529  5410 raft_consensus.cc:2243] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.016209  5410 raft_consensus.cc:2272] T 7365731c56504a678dc1a5ceac3f8da4 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.052870  5410 tablet_replica.cc:333] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.053691  5410 raft_consensus.cc:2243] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.054809  5410 raft_consensus.cc:2272] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 921c423406da4f289884afca51ca198f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.074193  5410 tablet_replica.cc:333] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f: stopping tablet replica
W20260709 13:20:06.077755  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:06.075320  5410 raft_consensus.cc:2243] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.081425  5410 raft_consensus.cc:2272] T 835ac063488a40878ccb02b662d3898b P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.094558  5410 tablet_replica.cc:333] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.097766  5410 raft_consensus.cc:2243] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.098476  5410 pending_rounds.cc:70] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Trying to abort 1 pending ops.
I20260709 13:20:06.098790  5410 pending_rounds.cc:77] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f: Aborting op as it isn't in flight: id { term: 27 index: 15 } timestamp: 7305638668102615040 op_type: NO_OP noop_request { }
I20260709 13:20:06.099218  5410 raft_consensus.cc:2889] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:06.099659  5410 raft_consensus.cc:2272] T ce959bc5e0b4481abad1a3bef3085967 P 921c423406da4f289884afca51ca198f [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.109855  5410 tablet_replica.cc:333] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.110807  5410 raft_consensus.cc:2243] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.111634  5410 raft_consensus.cc:2272] T 004d58f566fd4b6d954bc425675471fa P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.126369  5410 tablet_replica.cc:333] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.129562  5410 raft_consensus.cc:2243] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.130898  5410 raft_consensus.cc:2272] T 943ce377e70a422a8376b294da2840fa P 921c423406da4f289884afca51ca198f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.144687  5410 tablet_replica.cc:333] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.145633  5410 raft_consensus.cc:2243] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.147995  5410 raft_consensus.cc:2272] T 4c870451bc2945e8b0a88e8fedf73ce6 P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.172932  5410 tablet_replica.cc:333] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.173875  5410 raft_consensus.cc:2243] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.174726  5410 raft_consensus.cc:2272] T f4f0dad199224b8182ae036d97281b72 P 921c423406da4f289884afca51ca198f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.180969  5410 tablet_replica.cc:333] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.190428  5410 raft_consensus.cc:2243] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.191327  5410 raft_consensus.cc:2272] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.194334  5410 tablet_replica.cc:333] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.195183  5410 raft_consensus.cc:2243] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.195901  5410 raft_consensus.cc:2272] T 6be297bb363c43c99d04cccfc55bec8f P 921c423406da4f289884afca51ca198f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.198652  5410 tablet_replica.cc:333] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.202874  5410 raft_consensus.cc:2243] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.205354  5410 raft_consensus.cc:2272] T 40b7cf612223418ea8fe2d636986bb3f P 921c423406da4f289884afca51ca198f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.218909  5410 tablet_replica.cc:333] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.219766  5410 raft_consensus.cc:2243] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.220449  5410 pending_rounds.cc:70] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Trying to abort 1 pending ops.
I20260709 13:20:06.220737  5410 pending_rounds.cc:77] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305638682196189184 op_type: NO_OP noop_request { }
I20260709 13:20:06.221189  5410 raft_consensus.cc:2889] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:06.221585  5410 raft_consensus.cc:2272] T c98ecd701e924e049261801a6e8b1e3b P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.225062  5410 tablet_replica.cc:333] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.235613  5410 raft_consensus.cc:2243] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.237303  5410 raft_consensus.cc:2272] T 17b598f21a79476cabdf3f5191b4f9a8 P 921c423406da4f289884afca51ca198f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.240450  5410 tablet_replica.cc:333] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.241288  5410 raft_consensus.cc:2243] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.242108  5410 raft_consensus.cc:2272] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.248085  5410 tablet_replica.cc:333] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.250869  5410 raft_consensus.cc:2243] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:06.263006  5410 pending_rounds.cc:70] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Trying to abort 1 pending ops.
I20260709 13:20:06.263446  5410 pending_rounds.cc:77] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f: Aborting op as it isn't in flight: id { term: 18 index: 15 } timestamp: 7305638649060691968 op_type: NO_OP noop_request { }
I20260709 13:20:06.263886  5410 raft_consensus.cc:2889] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:06.264286  5410 raft_consensus.cc:2272] T ffa871a3f5334bd49b9c75539f218332 P 921c423406da4f289884afca51ca198f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:06.268746  5410 tablet_replica.cc:333] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f: stopping tablet replica
I20260709 13:20:06.270208  5410 raft_consensus.cc:2243] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:06.273844  5410 raft_consensus.cc:2272] T 1a2845fc76124deab8a20512c6112df7 P 921c423406da4f289884afca51ca198f [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:06.297353  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:06.451314  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:06.477772  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:06.531287  6563 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:06.540570 14941 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.541249 14941 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.543607 14941 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:06.545982  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:06.555599  6637 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:06.556094  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:06.559664 14941 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:06.680184 14935 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.680904 14935 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.722781 14935 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:06.736186  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:06.736706  6671 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:20:06.763629  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:20:06.766593 14917 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:06.767179 14917 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.767562 14917 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:06.769610  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:06.770351  6563 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.766708 14941 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.775146 14941 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.777721 14941 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:06.788951 14797 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.789664 14797 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.792093 14797 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:06.798025  6563 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.804203 14917 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.805289  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:06.812734  6563 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:06.812861  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:06.812814  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:06.821565  6561 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:06.822492 14797 raft_consensus.cc:2749] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:06.830945 14941 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.831619 14941 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.833786 14941 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:06.841028  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
W20260709 13:20:06.850776  6637 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.853500  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:06.853601  6407 catalog_manager.cc:4755] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): ChangeConfig:REMOVE_PEER RPC failed for tablet 938e1826b2ce4babab3a70e5219411a4: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.854022 14917 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:06.868007 14941 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:06.875648 14941 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.876256 14941 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.878396 14941 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:06.882817  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7d55fddd552409bbf822686f45eda69" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:06.883440  6671 raft_consensus.cc:3060] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:06.897517  6671 raft_consensus.cc:2468] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:20:06.907440 14917 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.908113 14917 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:06.908504  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.910990  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:06.911234 14797 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.911844 14797 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:06.916208 14917 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:06.916888  6563 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.917435 14947 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.916393  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:06.917994 14947 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:06.919857  6562 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:06.920883  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:06.911846 14941 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:06.922025  6563 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.922482 14797 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:06.922228 14947 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:06.920970 14917 raft_consensus.cc:2749] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:20:06.922777  6562 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:06.924487 14917 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:06.924743  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:06.926791  7200 leader_election.cc:336] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.927910  7198 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:06.928673 14942 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.929210 14942 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:06.931474 14942 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:06.931991 14942 raft_consensus.cc:2749] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:06.934978  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:06.935667  6588 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 27.
I20260709 13:20:06.936813  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:06.937579 14942 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:20:06.938041 14942 raft_consensus.cc:493] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.938476 14942 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:20:06.940508  7200 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:06.949848  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:06.952818  6561 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:06.954905 14797 raft_consensus.cc:2804] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:06.955451 14797 raft_consensus.cc:697] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:06.956411 14797 consensus_queue.cc:237] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:06.961879 14942 raft_consensus.cc:515] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:06.973146  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:06.982154 14941 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:06.989825 14941 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:06.990332  6417 catalog_manager.cc:5697] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 22 to 23, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 23 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:06.992271 14941 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:07.001631  7200 leader_election.cc:336] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.004808  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.015914  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d83f8b945cd641db90647da0461be4a3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:07.016564  6588 raft_consensus.cc:3060] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:20:06.982370  6637 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.018832  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.019594 14941 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:07.026041  6588 raft_consensus.cc:2468] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 28.
I20260709 13:20:07.026571 14942 leader_election.cc:290] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.025671 14797 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.049393 14797 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.051371 14797 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.028584 14917 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.051968 14917 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.053963 14917 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.029088 14935 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.054809 14935 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:07.055259  6563 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.056795 14935 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.036537 14947 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.057505 14947 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.044730 14941 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
W20260709 13:20:07.059046  6563 leader_election.cc:336] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.058894 14941 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.060964 14941 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.062726 14947 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.064678  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:07.068534  6563 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.074671  7200 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.075448  7200 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.076813 14586 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:07.076893 14954 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.077579 14954 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.078291 14950 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.078851 14950 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.079998 14954 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.081023 14950 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.082291  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.082901  7231 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:20:07.084712 14954 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.085244 14954 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.085875  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.086503 14941 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.087627 14954 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.085206 14950 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.089627 14941 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.089867 14950 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.091946 14941 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.092284 14950 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.092545 14941 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:07.092885 14950 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.093364 14941 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.093516 14950 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.094820  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.096056 14950 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.094039 14941 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:07.124221  7200 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.125279  7200 leader_election.cc:336] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.126287 14954 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.127123 14917 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.126933 14954 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.127727 14917 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.128679  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.129374 14954 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.130036 14917 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.131345  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.131934  7231 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:20:07.133608  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.134202  6670 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 24.
I20260709 13:20:07.134389  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.135071  6668 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
W20260709 13:20:07.135874  6563 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.136111  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.136652 14917 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.137230 14917 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.138008  6636 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.138688 14961 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.139433 14961 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.141654 14961 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.142222 14917 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:07.144383  6563 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.160642 14941 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.161940  6671 raft_consensus.cc:2393] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 20: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:20:07.168565 14941 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:07.180886  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 44780.3us
I20260709 13:20:07.182605 14942 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:07.183632 14942 raft_consensus.cc:493] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.184289 14942 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:07.187996  7231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.188738  7231 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:20:07.195149  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.197908 14947 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.198343 14947 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.198876 14947 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:07.203409  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 65505.5us
I20260709 13:20:07.203621 14960 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.204125 14960 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.204443 14960 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.206897 14941 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.207484 14941 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.208442  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.209089  6588 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 14.
I20260709 13:20:07.209924 14961 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.210490 14961 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.211012  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.211633  6595 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:20:07.213162 14942 raft_consensus.cc:515] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.214565 14961 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.223865 14960 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:07.196291  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 49068us
I20260709 13:20:07.224777 14941 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.225816  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.238682 14954 raft_consensus.cc:3060] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:07.239640  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:20:07.240310  6637 leader_election.cc:336] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.240828  6637 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.243670 14941 raft_consensus.cc:2749] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:07.245891 14586 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.246838 14954 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:07.246403 14586 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.248064  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:20:07.248838 14961 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:07.249151 14586 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.249696 14961 raft_consensus.cc:493] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.251194 14961 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:07.256604  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab5f01bd4774a168d3c01940a615cce" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.262311 14959 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.262817 14959 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.263212 14959 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.265321  7236 raft_consensus.cc:2468] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:20:07.266541  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.195339  7200 leader_election.cc:304] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.267256  7232 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:20:07.267676  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.267261 14961 raft_consensus.cc:515] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.268256 14960 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.268277  7233 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:20:07.237073 14947 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.270061 14961 leader_election.cc:290] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.270835 14947 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.271839 14586 raft_consensus.cc:2804] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 28 FOLLOWER]: Leader election won for term 28
W20260709 13:20:07.271940  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 76684.1us
I20260709 13:20:07.272331 14586 raft_consensus.cc:697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 28 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
W20260709 13:20:07.272939  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:07.273213 14586 consensus_queue.cc:237] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.276718  6636 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.273391 14959 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.279222  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.280324 14960 raft_consensus.cc:2804] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:07.280448 14961 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:07.280925 14961 raft_consensus.cc:493] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.280849 14960 raft_consensus.cc:697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:07.281502 14961 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:07.282528  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:07.283560 14917 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.284374 14917 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.285921 14959 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.286568 14917 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.287531  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.288640  7232 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
W20260709 13:20:07.289826  7200 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.296754  6563 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.297394  6563 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.302517 14966 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.303239 14966 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:07.304574  7200 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.306003 14966 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:07.316394  7200 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.317231  7200 leader_election.cc:336] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.287137 14960 consensus_queue.cc:237] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.250324 14942 leader_election.cc:290] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.329758 14917 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.330587 14917 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.331096 14961 raft_consensus.cc:515] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.332283 14941 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.332862 14941 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.333352 14961 leader_election.cc:290] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.334781 14917 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.334995 14941 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:07.341935  6563 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.248944  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
W20260709 13:20:07.360041  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 111155us
I20260709 13:20:07.360383 14941 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.360929 14941 raft_consensus.cc:493] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.361325 14941 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.361653  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.362517  6671 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:20:07.367334  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f67b820295ac47a4b54e21400f8b4201" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.368057  6671 raft_consensus.cc:3060] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:07.369431 14941 raft_consensus.cc:515] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:07.374868  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.375142 14941 leader_election.cc:290] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.376837  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.377627  6662 raft_consensus.cc:2393] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 21: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:20:07.380049  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34cabfbde16f4e10affcaec358aa0dd4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.380649  7236 raft_consensus.cc:3060] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:07.376974 14966 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.383069  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.383039 14966 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.384048  6594 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:20:07.385439 14966 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:07.390532  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 128935us
I20260709 13:20:07.393795  7236 raft_consensus.cc:2468] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 16.
I20260709 13:20:07.394927  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d9d0fb5a0864567ac4cc6ee615911b6" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.396991  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.399361  7233 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 25.
I20260709 13:20:07.400245  7236 raft_consensus.cc:3060] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.403901  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.404570  6670 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 22.
I20260709 13:20:07.404385  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.405633  6668 raft_consensus.cc:2393] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 15: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:20:07.406653  6667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.407323  6667 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 25.
I20260709 13:20:07.407408  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.408044  6670 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:20:07.408260  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.409094  6594 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 25.
I20260709 13:20:07.416056  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.416666  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.417301  6589 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:20:07.416666  6594 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:20:07.420593  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.423621  7236 raft_consensus.cc:2468] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:20:07.425289  6671 raft_consensus.cc:2468] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 25.
I20260709 13:20:07.426424  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.426584  6417 catalog_manager.cc:5697] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 27 to 28, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 28 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:07.427595  7233 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 23.
I20260709 13:20:07.429134 14968 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:07.430779 14968 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.431440 14968 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:07.432747 14966 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.436784 14586 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.437304 14586 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.439287 14947 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.439947 14947 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.449847  6636 leader_election.cc:304] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.452559 14961 raft_consensus.cc:2804] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:07.452673  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.453788 14941 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:07.454281 14941 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.454794 14941 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:07.467697  7200 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.468863 14942 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:07.469565 14942 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.470152 14942 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.471971 14961 raft_consensus.cc:697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:07.472172  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.473282 14954 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:07.473179 14961 consensus_queue.cc:237] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.476850  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.477548  6664 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
W20260709 13:20:07.482654  6562 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 62149us
I20260709 13:20:07.484241  6562 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.484709 14947 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.485780 14958 raft_consensus.cc:2804] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:07.486382 14958 raft_consensus.cc:697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:07.486958 14586 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.487313 14958 consensus_queue.cc:237] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.433287 14966 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.491514 14966 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:07.492096  6637 leader_election.cc:336] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.496133  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.497906 14947 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:07.499096 14947 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.499480 14947 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:07.501467  7198 leader_election.cc:304] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.503660  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.504267  6671 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 20.
I20260709 13:20:07.506146 14961 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.506335  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:07.506489  7200 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.506721 14961 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:07.507789  6637 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.508543  6637 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.509086 14586 raft_consensus.cc:2804] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:07.509547 14586 raft_consensus.cc:697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 25 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:20:07.511706  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.508925  6637 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:20:07.517938  6637 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.518968  6637 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.520026  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.520095  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.522285 14954 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.527311 14954 raft_consensus.cc:493] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.527717 14954 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.527601 14968 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.529909 14968 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.523572 14941 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.531715 14961 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.517894 14950 raft_consensus.cc:2749] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:07.532858 14964 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.533524 14964 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.532919 14941 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.536769 14964 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:07.537334  6563 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.537586 14586 consensus_queue.cc:237] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:07.542974  6563 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:07.537746 14947 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.544443 14942 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.545109 14947 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:07.545478  6563 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.546015  6563 consensus_peers.cc:597] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:07.554008  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:07.554883  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "520b1aa490ee4869a2814c313fac4a0e" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:07.555508  6594 raft_consensus.cc:3060] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.557960 14942 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.558259  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a87665f0ba4296b0ba819fcb712842" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:07.561851  6595 raft_consensus.cc:3060] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:07.562152  7200 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.562640  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:07.563648  7200 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.564082  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.560660 14954 raft_consensus.cc:515] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.566411 14954 leader_election.cc:290] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.567900 14947 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:07.568733 14947 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:07.569789  7200 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.570986  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.568051  6417 catalog_manager.cc:5697] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 19 to 20, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 20 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:07.576666  6595 raft_consensus.cc:2468] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 15.
I20260709 13:20:07.577504  6594 raft_consensus.cc:2468] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:20:07.581679 14941 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.582306 14941 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.582944 14970 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:07.583467 14970 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.583854 14970 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.584709 14941 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:07.586606  7200 leader_election.cc:336] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.587225  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.592579 14964 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.593187 14964 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.595343  6561 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:20:07.595230  7200 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.596632  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 84990.6us
I20260709 13:20:07.581147  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:07.602110  7198 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.604000 14964 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.605137 14941 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:07.605731 14941 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.604667 14968 raft_consensus.cc:2749] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:07.606129 14941 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.604902  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 
I20260709 13:20:07.607437 14950 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:07.607801 14950 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.608146 14950 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:07.609447 14970 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.605149 14954 raft_consensus.cc:2749] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:07.613291  6637 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.614136  6637 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.614862  6637 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.616861  6637 leader_election.cc:336] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.623440 14950 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.627473  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8a48eeccdac45838ef8456e5e9332b5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.628108  6664 raft_consensus.cc:3060] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.628415  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:07.629050  6669 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 20.
I20260709 13:20:07.629619 14970 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.630721  6671 raft_consensus.cc:1275] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:20:07.631335  6637 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.631814  6637 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.632551 14961 raft_consensus.cc:2749] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:20:07.633498  6637 leader_election.cc:336] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.634088  6637 leader_election.cc:336] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.634855  6637 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.635504  6637 leader_election.cc:336] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.635612  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
W20260709 13:20:07.635942  6637 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.636439  6637 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.636868  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:07.638799  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff27e205e84e4edf946bc02c573a2c25" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.639370  7236 raft_consensus.cc:3060] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:07.641439  6664 raft_consensus.cc:2468] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
W20260709 13:20:07.641801  6637 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.643013  7232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.643683  7232 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
I20260709 13:20:07.644550  7229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.645308  7229 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 26: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:20:07.646721  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:07.647389  7233 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
W20260709 13:20:07.647874  5410 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 6543)
I20260709 13:20:07.649236  6411 catalog_manager.cc:5697] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 24 to 25, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 25 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:07.657887 14950 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:07.664240  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:20:07.664475  6637 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.665706 14950 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:07.666435  7198 leader_election.cc:304] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.666486 14950 raft_consensus.cc:493] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.667392 14975 raft_consensus.cc:2804] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:07.668023 14975 raft_consensus.cc:697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:20:07.668913 14975 consensus_queue.cc:237] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.674150  7236 raft_consensus.cc:2468] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 24.
I20260709 13:20:07.642786  6417 catalog_manager.cc:5697] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 15 to 16, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 16 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:07.681248  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
W20260709 13:20:07.684470  6563 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.700865  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.667382 14950 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.700953 14958 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:20:07.702198 14970 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:20:07.702581  6563 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.702852 14970 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.703316 14970 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:07.682610  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 77116.9us
I20260709 13:20:07.703951  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.704653 14958 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:07.705001 14958 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.705334 14958 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:07.713552 14958 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.714504 14970 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.717628  6636 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.718993 14961 raft_consensus.cc:2804] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:07.719528 14961 raft_consensus.cc:697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 24 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:07.720440 14961 consensus_queue.cc:237] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.721293 14970 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.715214  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.726867 14970 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:07.727698 14970 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.727999  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.728168 14970 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:07.729343 14917 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:07.729382  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.729866 14917 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.730266 14917 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:07.724936  6563 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.725781 14958 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.738040 14917 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.740140 14917 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:07.742051  6563 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.696573 14941 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.744166  6668 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 23: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:20:07.744148 14970 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.724251  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b598f21a79476cabdf3f5191b4f9a8" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.749315 14970 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.750202  7198 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.749375  7236 raft_consensus.cc:3060] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:07.754897 14797 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.799343 14797 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.799784 14797 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.756620 14964 consensus_queue.cc:1048] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:20:07.757846  6637 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:07.788822  7232 raft_consensus.cc:1275] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:20:07.806057  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 48251.3us
I20260709 13:20:07.767566 14954 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:07.807451  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fc18602dfbd4fc9897b6b696b497f73" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.806313  6414 catalog_manager.cc:5697] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 20 to 21, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 21 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:07.808449  6637 leader_election.cc:304] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.809198 14960 raft_consensus.cc:2804] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:07.809573  6637 leader_election.cc:304] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:07.811069  6671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d990297ed0410d894aa69d60271fe7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.811817  6671 raft_consensus.cc:3060] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:07.808879 14961 consensus_queue.cc:1048] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 13:20:07.771551  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:07.808059  6668 raft_consensus.cc:3060] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.815900  6662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.816939  6662 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ceee54e49d041be9f7f48a2052709c5 in current term 22: Already voted for candidate 53fee9894731461a821a131be24dbbca in this term.
I20260709 13:20:07.818378  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15a9820f1e1414f829e7fecece15dc0" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.819186  6665 raft_consensus.cc:3060] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:07.822005  7233 raft_consensus.cc:2393] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 23: Already voted for candidate 64b95375974b48d18c596a23c9cac75c in this term.
I20260709 13:20:07.826313  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 64b95375974b48d18c596a23c9cac75c, 921c423406da4f289884afca51ca198f
I20260709 13:20:07.834450 14961 raft_consensus.cc:2804] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:07.835072 14961 raft_consensus.cc:697] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:07.835976 14961 consensus_queue.cc:237] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:07.837255  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:07.838444 14978 raft_consensus.cc:2749] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:20:07.881799  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 44598us
I20260709 13:20:07.884658  6588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:07.885383  6588 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.838987 14960 raft_consensus.cc:697] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:07.887961 14960 consensus_queue.cc:237] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.840183  6671 raft_consensus.cc:2468] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 26.
I20260709 13:20:07.841135  6411 catalog_manager.cc:5697] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 21, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 21 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:07.842532 14797 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:07.895259 14797 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:07.841728  6417 catalog_manager.cc:5697] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 23 to 24, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 24 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:07.769953  6563 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.903550  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 133713us
I20260709 13:20:07.905210  7236 raft_consensus.cc:2468] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:20:07.752602 14941 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.859154 14917 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:07.860585  6589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:07.861829  6590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:07.907976  6590 raft_consensus.cc:2393] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 21: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:20:07.868145  6594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:07.878896  6595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:07.908751  6595 raft_consensus.cc:2393] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 64b95375974b48d18c596a23c9cac75c in current term 26: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:20:07.909262  6589 raft_consensus.cc:2393] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 53fee9894731461a821a131be24dbbca in current term 22: Already voted for candidate 3ceee54e49d041be9f7f48a2052709c5 in this term.
I20260709 13:20:07.838631  6668 raft_consensus.cc:2468] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 23.
I20260709 13:20:07.897071  6596 raft_consensus.cc:1275] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 13:20:07.900769  6588 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 23.
I20260709 13:20:07.844138  6665 raft_consensus.cc:2468] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:20:07.796518 14950 raft_consensus.cc:515] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:07.944867  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 135355us
W20260709 13:20:07.952847  6563 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.956092  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 129849us
I20260709 13:20:07.939940 14917 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:07.969729 14950 leader_election.cc:290] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:07.976296  6561 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
W20260709 13:20:07.981216  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:07.981402  6664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835ac063488a40878ccb02b662d3898b" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:07.981949  6664 raft_consensus.cc:3060] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.986089 14964 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:07.987500  6561 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:20:07.990484  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:07.991309  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:07.992045  6563 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:07.992703  6563 consensus_peers.cc:597] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:07.995452  6561 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:08.000268  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943ce377e70a422a8376b294da2840fa" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:08.000897  7233 raft_consensus.cc:3060] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.987967  6562 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:08.001773 14797 raft_consensus.cc:2804] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:08.002185  6561 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:08.002267 14797 raft_consensus.cc:697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:20:08.006224  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 64474.7us
I20260709 13:20:08.007457 14797 consensus_queue.cc:237] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:08.003917 14970 raft_consensus.cc:2804] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:20:08.020219  6664 raft_consensus.cc:2468] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 21.
I20260709 13:20:08.021158  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:08.027040  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 44574.6us
I20260709 13:20:08.028520 14586 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:20:08.029970  7233 raft_consensus.cc:2468] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 23.
I20260709 13:20:08.032061  6636 leader_election.cc:304] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:08.032315  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:08.032939 14960 raft_consensus.cc:2804] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:08.033983 14950 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:08.034159  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:08.035808 14960 raft_consensus.cc:697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 23 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:08.037997 14979 raft_consensus.cc:2749] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:08.038224  7230 raft_consensus.cc:1275] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 13:20:08.051872 14586 consensus_queue.cc:1048] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 13:20:08.062799  6664 raft_consensus.cc:1275] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:20:08.063524 14971 raft_consensus.cc:2804] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:08.028327 14964 raft_consensus.cc:2804] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:08.086102 14960 consensus_queue.cc:237] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:08.099046 14982 consensus_queue.cc:1048] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:20:08.113613 14987 consensus_queue.cc:1048] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:20:08.126798 14971 raft_consensus.cc:697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:08.128296 14971 consensus_queue.cc:237] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:08.138965  6407 catalog_manager.cc:4755] TS 921c423406da4f289884afca51ca198f (127.5.72.129:34121): ChangeConfig:REMOVE_PEER RPC failed for tablet 938e1826b2ce4babab3a70e5219411a4: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:08.084132 14964 raft_consensus.cc:697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
W20260709 13:20:08.105731  7200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111) [suppressed 301 similar messages]
W20260709 13:20:08.145699  7200 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:08.146183  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:08.153313 14969 log.cc:927] Time spent T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:20:08.175905  6671 raft_consensus.cc:1275] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:08.181211 14586 consensus_queue.cc:1048] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:08.147604 14917 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:08.186116 14964 consensus_queue.cc:237] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:08.199585  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:20:08.200243  6637 consensus_peers.cc:597] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.200249  7233 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 24.
I20260709 13:20:08.211376  6561 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
W20260709 13:20:08.200764  6637 leader_election.cc:336] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:08.214118  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:08.214995  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:08.147969 14970 raft_consensus.cc:697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:08.218834 14797 raft_consensus.cc:2804] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:08.219537 14797 raft_consensus.cc:697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:08.225167  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:08.227448 14996 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:08.227880 14996 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 921c423406da4f289884afca51ca198f)
I20260709 13:20:08.228240 14996 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:20:08.242033  6563 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:08.241561 14970 consensus_queue.cc:237] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:08.237396 14797 consensus_queue.cc:237] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:08.280859  7200 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.283421  6417 catalog_manager.cc:5697] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 23 to 24, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 24 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.308851 14996 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:08.310999  7233 raft_consensus.cc:1275] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:20:08.319128  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:08.327864  7233 raft_consensus.cc:1275] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:20:08.362686  7233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a2845fc76124deab8a20512c6112df7" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:08.364195  7233 raft_consensus.cc:3060] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:08.378837  6417 catalog_manager.cc:5697] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 14 to 15, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 15 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:08.385057  6417 catalog_manager.cc:5697] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 20 to 21, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 21 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:08.390746 14993 consensus_queue.cc:1048] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
W20260709 13:20:08.393635  6637 consensus_peers.cc:597] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.359151 14961 consensus_queue.cc:1048] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 13:20:08.405252  6417 catalog_manager.cc:5697] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 22 to 23, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 23 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.406555  7233 raft_consensus.cc:2468] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 25.
W20260709 13:20:08.423254  6637 consensus_peers.cc:597] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.427618  6562 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:08.429241 14970 raft_consensus.cc:2804] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:08.429863 14970 raft_consensus.cc:697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:08.430823 14970 consensus_queue.cc:237] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:08.427678  6637 consensus_peers.cc:597] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.371680 14996 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:08.462963  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 52686.3us
W20260709 13:20:08.483471  6563 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:08.500182  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:08.537108  6411 catalog_manager.cc:5697] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 20 to 21, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 21 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.538872  6411 catalog_manager.cc:5697] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 25 to 26, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 26 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.540355  6411 catalog_manager.cc:5697] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 22 to 23, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 23 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:08.543282  6637 consensus_peers.cc:597] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.541736  6411 catalog_manager.cc:5697] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 23 to 24, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 24 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.572803  6593 raft_consensus.cc:1275] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:20:08.598742  6563 consensus_peers.cc:597] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:08.600188  6636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:36575, user_credentials={real_user=slave}} blocked reactor thread for 52413.5us
W20260709 13:20:08.638881  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 94885.7us
W20260709 13:20:08.648306  5410 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 6543)
I20260709 13:20:08.650590  6593 raft_consensus.cc:1275] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:20:08.657810  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:08.664798  7230 raft_consensus.cc:1275] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:08.671350 14961 consensus_queue.cc:1048] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:20:08.672291  6662 raft_consensus.cc:1275] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:20:08.698407  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:08.718113 14959 consensus_queue.cc:1048] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:08.708994 14982 consensus_queue.cc:1048] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:20:08.724484  7236 raft_consensus.cc:1275] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:08.725425  6669 raft_consensus.cc:1275] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:08.726313 14958 consensus_queue.cc:1048] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:20:08.727500  6637 consensus_peers.cc:597] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.710009 15014 consensus_queue.cc:1048] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:20:08.732743 14970 consensus_queue.cc:1048] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:20:08.738950  6563 consensus_peers.cc:597] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.753854  6665 raft_consensus.cc:1275] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:20:08.792380 14971 consensus_queue.cc:1048] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:20:08.791682  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:08.829306  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:08.830384  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 65775.4us
W20260709 13:20:08.837168  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:08.837723  6563 consensus_peers.cc:597] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:08.838189  6563 consensus_peers.cc:597] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:08.887841  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:08.912468  6563 consensus_peers.cc:597] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.933211  6669 raft_consensus.cc:1275] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 13:20:08.937849  6563 consensus_peers.cc:597] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.920264  6417 catalog_manager.cc:5697] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 24 to 25, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 25 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.962491 14964 consensus_queue.cc:1048] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:09.030805  7236 raft_consensus.cc:1275] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:20:09.032195 14971 consensus_queue.cc:1048] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:20:09.234156  6563 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:09.243935  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:09.329255  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:09.438799 14979 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.442781 14947 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.443868 14947 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:09.439393 14979 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:09.446098 14947 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:09.455312  7200 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:09.461014  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:09.461588  6665 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 21.
W20260709 13:20:09.464916  6637 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:09.465633  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:09.466223 14979 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:09.466856 14966 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:09.467281 14966 raft_consensus.cc:493] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.467613 14966 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:09.471706  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:09.472443  6593 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 23.
I20260709 13:20:09.482030  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:09.482758  7230 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 23.
I20260709 13:20:09.485234  6636 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:09.486045 14979 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:09.486485 14979 raft_consensus.cc:493] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.486853 14979 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:09.494098 14979 raft_consensus.cc:515] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:09.499396 14966 raft_consensus.cc:515] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:09.507135  6593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:09.507802  6593 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:09.509030  6637 leader_election.cc:336] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:09.509941  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "938e1826b2ce4babab3a70e5219411a4" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:09.510499  7230 raft_consensus.cc:3060] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:09.516199  7200 leader_election.cc:336] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:09.529119  6593 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 24.
I20260709 13:20:09.530063  7230 raft_consensus.cc:2468] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 24.
I20260709 13:20:09.532330 14979 leader_election.cc:290] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:09.563469  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:09.547435 14966 leader_election.cc:290] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:09.548386  6665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2c67ec2db9643088bc65fe9d83aa732" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca"
I20260709 13:20:09.565111  6665 raft_consensus.cc:3060] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:09.565047  6637 leader_election.cc:304] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:09.566776 14979 raft_consensus.cc:2804] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:09.567255 14979 raft_consensus.cc:697] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 24 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:09.568149 14979 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:09.585115  6665 raft_consensus.cc:2468] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64b95375974b48d18c596a23c9cac75c in term 22.
I20260709 13:20:09.587255  7198 leader_election.cc:304] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:09.591323 14586 raft_consensus.cc:2804] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:09.591826 14586 raft_consensus.cc:697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Becoming Leader. State: Replica: 64b95375974b48d18c596a23c9cac75c, State: Running, Role: LEADER
I20260709 13:20:09.592640 14586 consensus_queue.cc:237] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:09.648782  5410 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:09.653360  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:09.670379  6414 catalog_manager.cc:5697] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c reported cstate change: term changed from 20 to 22, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 64b95375974b48d18c596a23c9cac75c (127.5.72.132). New cstate: current_term: 22 leader_uuid: "64b95375974b48d18c596a23c9cac75c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:09.677690  6411 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 22 to 24, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 24 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:09.851843  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:09.860513 15014 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.861070 15014 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:09.863083 15014 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:09.867175  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5" is_pre_election: true
I20260709 13:20:09.867827  6596 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 22.
W20260709 13:20:09.869397  6637 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:09.876266  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:09.890818 15014 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:09.891266 15014 raft_consensus.cc:493] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.891618 15014 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:09.900204  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:09.931347 14971 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.934682 14971 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:09.938138 14971 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:09.939160  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:09.939792  7230 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 26.
I20260709 13:20:09.940629  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:09.941793 14971 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:20:09.942214 14971 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.942641 14971 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:09.948436 15014 raft_consensus.cc:515] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:09.949676  6563 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:09.956112  6637 leader_election.cc:336] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:09.963747  6596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d4a8b90b128485fa69bc10112de24e9" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3ceee54e49d041be9f7f48a2052709c5"
I20260709 13:20:09.964325  6596 raft_consensus.cc:3060] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:09.970712 15014 leader_election.cc:290] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:09.977311  6596 raft_consensus.cc:2468] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 23.
I20260709 13:20:09.980489  6637 leader_election.cc:304] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:09.981575 15014 raft_consensus.cc:2804] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:09.986033 14971 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:09.988783 15014 raft_consensus.cc:697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 23 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:09.989825 15014 consensus_queue.cc:237] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
W20260709 13:20:09.995849  6563 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:10.008227  6563 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:10.013195 14971 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 27 election: Requested vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:10.014170  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd28ad0fdaad48d586176f4a8e3f1aa3" candidate_uuid: "3ceee54e49d041be9f7f48a2052709c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:10.015172  7230 raft_consensus.cc:3060] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:10.023077  6665 raft_consensus.cc:1275] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Refusing update from remote peer 64b95375974b48d18c596a23c9cac75c: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:20:10.024619  7230 raft_consensus.cc:2468] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ceee54e49d041be9f7f48a2052709c5 in term 27.
I20260709 13:20:10.027560 14586 consensus_queue.cc:1048] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [LEADER]: Connected to new peer: Peer: permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:20:10.028823  6562 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ceee54e49d041be9f7f48a2052709c5, 64b95375974b48d18c596a23c9cac75c; no voters: 921c423406da4f289884afca51ca198f
I20260709 13:20:10.031337 14964 raft_consensus.cc:2804] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:20:10.031821 14964 raft_consensus.cc:697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 LEADER]: Becoming Leader. State: Replica: 3ceee54e49d041be9f7f48a2052709c5, State: Running, Role: LEADER
I20260709 13:20:10.032721 14964 consensus_queue.cc:237] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:10.060163  6637 consensus_peers.cc:597] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:10.072661  7230 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:20:10.081226 14979 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:20:10.083033  6563 consensus_peers.cc:597] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:10.100265  6593 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:20:10.104494  6414 catalog_manager.cc:5697] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 reported cstate change: term changed from 25 to 27, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130). New cstate: current_term: 27 leader_uuid: "3ceee54e49d041be9f7f48a2052709c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:10.126977  6411 catalog_manager.cc:5697] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 21 to 23, leader changed from 921c423406da4f289884afca51ca198f (127.5.72.129) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 23 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:10.131480 14979 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:20:10.273458  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:10.278858  7198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 60601us
W20260709 13:20:10.291095  7200 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:10.342890  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 65558.7us
W20260709 13:20:10.375537  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:10.387327 15039 log.cc:927] Time spent T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:20:10.390738  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:10.442579  6637 consensus_peers.cc:597] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:10.452513  6563 consensus_peers.cc:597] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:10.456568  6405 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 938e1826b2ce4babab3a70e5219411a4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 53fee9894731461a821a131be24dbbca (127.5.72.131:38019) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:20:10.464385  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:10.470491  6563 consensus_peers.cc:597] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:10.473912  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:10.504585  6596 raft_consensus.cc:1275] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:20:10.507270  6637 consensus_peers.cc:597] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:10.511900  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:10.516793 15051 consensus_queue.cc:1048] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:20:10.596489  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:10.603072  7232 raft_consensus.cc:1275] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Refusing update from remote peer 3ceee54e49d041be9f7f48a2052709c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:20:10.606685 15054 consensus_queue.cc:1048] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:20:10.628895  7200 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:10.641477  6637 consensus_peers.cc:597] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:10.649437  5410 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:10.689031  7200 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:10.721199  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:10.756765  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:10.837924  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:11.004755  6637 consensus_peers.cc:597] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.013780  6637 consensus_peers.cc:597] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.109620  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:11.113595  6637 consensus_peers.cc:597] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.137073  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:11.231350  6563 consensus_peers.cc:597] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.293105  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:11.298470  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:11.328248  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:11.341213  6637 consensus_peers.cc:597] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.343607  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:11.352762  6563 consensus_peers.cc:597] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.361063  6563 consensus_peers.cc:597] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.413477  6563 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:11.478811  6563 consensus_peers.cc:597] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.503726  6563 consensus_peers.cc:597] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.504405  6563 consensus_peers.cc:597] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:11.649904  5410 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:11.846139  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:12.032990  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:12.110576  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:12.275628  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:12.285377  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 55835.4us
W20260709 13:20:12.397411  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:12.522734  6563 consensus_peers.cc:597] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:12.580334  6637 consensus_peers.cc:597] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:12.621642  7200 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:12.651240  5410 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:12.651979  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:12.744477  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:12.782496  6563 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:12.795441  6563 consensus_peers.cc:597] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:12.832047  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:12.839648  6637 consensus_peers.cc:597] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:12.912606  7200 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:12.930364  6563 consensus_peers.cc:597] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:12.941507  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:12.972633  6637 consensus_peers.cc:597] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:12.987609  7200 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:12.994309  6637 consensus_peers.cc:597] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:13.034268  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:13.063566  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:13.108836  6637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111) [suppressed 433 similar messages]
W20260709 13:20:13.122274  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.123584  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.179960  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.218533  6637 consensus_peers.cc:597] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.244493  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.265754  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:13.443130  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.479441  6563 consensus_peers.cc:597] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:13.520946  6637 consensus_peers.cc:597] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:13.527124  6637 consensus_peers.cc:597] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:13.527565  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.651863  5410 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:13.676985  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:13.697355  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.710131  6563 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:13.773051  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:13.958531  6637 consensus_peers.cc:597] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.023298  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:14.076922  6563 consensus_peers.cc:597] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.078522  6563 consensus_peers.cc:597] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.079276  6563 consensus_peers.cc:597] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.117057  6563 consensus_peers.cc:597] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.129210  6563 consensus_peers.cc:597] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.246133  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:14.569546  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:14.616909  6665 consensus_queue.cc:237] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:14.640164  7230 raft_consensus.cc:1275] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Refusing update from remote peer 53fee9894731461a821a131be24dbbca: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:20:14.651007 15064 consensus_queue.cc:1048] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:20:14.652688  5410 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:14.670399  6637 consensus_peers.cc:597] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:14.710893 15065 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index 5 to 16, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
I20260709 13:20:14.716310  7232 raft_consensus.cc:2955] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 5 to 16, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:14.721946  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:14.725680  6405 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 938e1826b2ce4babab3a70e5219411a4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:20:14.748401  6414 catalog_manager.cc:5697] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c reported cstate change: config changed from index 5 to 16, VOTER 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) evicted. New cstate: current_term: 24 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } } }
W20260709 13:20:14.757702  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:20:14.886303  6584 tablet_service.cc:1558] Processing DeleteTablet for tablet 938e1826b2ce4babab3a70e5219411a4 with delete_type TABLET_DATA_TOMBSTONED (TS 3ceee54e49d041be9f7f48a2052709c5 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:59160
I20260709 13:20:14.901006 15081 tablet_replica.cc:333] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
W20260709 13:20:14.902230  6563 consensus_peers.cc:597] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:14.925028 15081 raft_consensus.cc:2243] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:14.926241 15081 raft_consensus.cc:2272] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:14.929548 15081 ts_tablet_manager.cc:1916] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:20:14.959570  7200 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.972820  7258 debug-util.cc:398] Leaking SignalData structure 0x7b08003122e0 after lost signal to thread 5413
W20260709 13:20:14.974056  7258 debug-util.cc:398] Leaking SignalData structure 0x7b0800300560 after lost signal to thread 6454
W20260709 13:20:14.975534  7258 debug-util.cc:398] Leaking SignalData structure 0x7b0800300b60 after lost signal to thread 6621
W20260709 13:20:14.976735  7258 debug-util.cc:398] Leaking SignalData structure 0x7b0800300120 after lost signal to thread 6695
W20260709 13:20:14.978015  7258 debug-util.cc:398] Leaking SignalData structure 0x7b08003a32a0 after lost signal to thread 7261
I20260709 13:20:14.997849 15081 ts_tablet_manager.cc:1929] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 24.15
W20260709 13:20:15.001211  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:20:15.006772 15081 log.cc:1199] T 938e1826b2ce4babab3a70e5219411a4 P 3ceee54e49d041be9f7f48a2052709c5: Deleting WAL directory at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/wals/938e1826b2ce4babab3a70e5219411a4
I20260709 13:20:15.068223  6407 catalog_manager.cc:5028] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): tablet 938e1826b2ce4babab3a70e5219411a4 (table test-workload [id=b35ecb8e11bf4c77a827717b0226b074]) successfully deleted
W20260709 13:20:15.082818  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 48185us
W20260709 13:20:15.234400  6563 consensus_peers.cc:597] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.249140  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.254829  6637 consensus_peers.cc:597] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:15.263633  7200 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:15.278883  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.289335  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.396166  6563 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:15.438373  6637 consensus_peers.cc:597] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:15.558174  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:15.566924  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.571121  7200 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:15.578339  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.589938  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.601577  6637 consensus_peers.cc:597] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:15.620162  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.653342  5410 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:15.668195  6563 consensus_peers.cc:597] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:15.818637  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:15.825290  6637 consensus_peers.cc:597] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.864053  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:15.972391  6563 consensus_peers.cc:597] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:16.040370  6637 consensus_peers.cc:597] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:16.053370  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:16.059026  6563 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:16.071825  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:16.159770  6409 maintenance_manager.cc:419] P 10caaf50f65c441b844e3e6055117d18: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033353
W20260709 13:20:16.179136  6637 consensus_peers.cc:597] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:16.190644  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:16.207702  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:20:16.252086  6408 maintenance_manager.cc:643] P 10caaf50f65c441b844e3e6055117d18: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.091s	user 0.032s	sys 0.010s Metrics: {"bytes_written":6994,"delete_count":0,"lbm_write_time_us":1136,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":13189,"threads_started":1,"update_count":297,"wal-append.queue_time_us":12879}
I20260709 13:20:16.254292  6409 maintenance_manager.cc:419] P 10caaf50f65c441b844e3e6055117d18: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.307686
W20260709 13:20:16.294298  6637 consensus_peers.cc:597] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:16.313169  6563 consensus_peers.cc:597] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:16.346495  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:16.347143  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:16.982343  6408 thread.cc:641] dir /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.003s	sys 0.001s
W20260709 13:20:16.653927  5410 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:17.050338  6408 thread.cc:608] dir /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0/minicluster-data/master-0-root/data [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.005s	sys 0.002s
W20260709 13:20:17.009989  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 43501.1us
I20260709 13:20:17.051753  6408 maintenance_manager.cc:643] P 10caaf50f65c441b844e3e6055117d18: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.797s	user 0.095s	sys 0.011s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":27167,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":118696,"dirs.run_cpu_time_us":284,"dirs.run_wall_time_us":559,"lbm_read_time_us":1868,"lbm_reads_lt_1ms":23,"lbm_write_time_us":1548,"lbm_writes_lt_1ms":11,"peak_mem_usage":304878,"reinsert_count":0,"thread_start_us":582248,"threads_started":1,"update_count":297}
W20260709 13:20:16.562044  6563 consensus_peers.cc:597] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:17.071003  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 508979us
W20260709 13:20:17.075830  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:17.077536  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:17.163244  6563 consensus_peers.cc:597] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:17.164261  6563 consensus_peers.cc:597] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:17.175755  6563 consensus_peers.cc:597] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:17.178448  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:17.382750  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:17.526793  6637 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 87185.6us
W20260709 13:20:17.616546  6618 debug-util.cc:398] Leaking SignalData structure 0x7b0800266ee0 after lost signal to thread 5413
W20260709 13:20:17.680327  6618 debug-util.cc:398] Leaking SignalData structure 0x7b08000952c0 after lost signal to thread 6454
W20260709 13:20:17.681844  6618 debug-util.cc:398] Leaking SignalData structure 0x7b080033b420 after lost signal to thread 6621
W20260709 13:20:17.683352  6618 debug-util.cc:398] Leaking SignalData structure 0x7b08002cb340 after lost signal to thread 6695
W20260709 13:20:17.684814  6618 debug-util.cc:398] Leaking SignalData structure 0x7b08000abb60 after lost signal to thread 7261
W20260709 13:20:17.706496  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 96598.1us
W20260709 13:20:17.677373  6563 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34121, user_credentials={real_user=slave}} blocked reactor thread for 115957us
W20260709 13:20:17.774837  6563 consensus_peers.cc:597] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:17.797094  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:17.798501  6637 consensus_peers.cc:597] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:17.800114  6563 consensus_peers.cc:597] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:17.812875  7200 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:17.858131  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:17.920105  6637 consensus_peers.cc:597] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:17.936841  6563 consensus_peers.cc:597] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:17.952489  6563 consensus_peers.cc:597] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:18.027870  6563 consensus_peers.cc:597] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.027958  6637 consensus_peers.cc:597] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.051370  5410 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 6543)
W20260709 13:20:18.107722  7200 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.113677  7200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111) [suppressed 414 similar messages]
W20260709 13:20:18.120995  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:18.123184  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:18.274801  6563 consensus_peers.cc:597] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:18.330293  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:18.524528  6637 consensus_peers.cc:597] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.525132  6637 consensus_peers.cc:597] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.525352  7200 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.530843  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:18.558068  6563 consensus_peers.cc:597] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:18.576164  6563 consensus_peers.cc:597] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.632427  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:18.655148  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:18.679970  6563 consensus_peers.cc:597] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.703270  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:18.765538  6563 consensus_peers.cc:597] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:18.851838  6563 consensus_peers.cc:597] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:18.860436  6637 consensus_peers.cc:597] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.861433  6563 consensus_peers.cc:597] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.862497  6637 consensus_peers.cc:597] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.899181  6637 consensus_peers.cc:597] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.085783  6563 consensus_peers.cc:597] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:19.086454  6563 consensus_peers.cc:597] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:19.096127  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:19.149767  6563 consensus_peers.cc:597] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:19.233930  6563 consensus_peers.cc:597] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.341784  6563 consensus_peers.cc:597] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.346683  6563 consensus_peers.cc:597] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.531641  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:19.597654  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
W20260709 13:20:19.673651  6561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:38019, user_credentials={real_user=slave}} blocked reactor thread for 46757.5us
W20260709 13:20:19.809211  6563 consensus_peers.cc:597] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.820510  7200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:37977, user_credentials={real_user=slave}} blocked reactor thread for 50163.2us
I20260709 13:20:19.821971  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
W20260709 13:20:19.838847  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:19.877759  7200 consensus_peers.cc:597] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:19.882447  6637 consensus_peers.cc:597] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:19.888783  6563 consensus_peers.cc:597] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:19.914829  7200 consensus_peers.cc:597] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:19.936797  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:19.939456  5410 tablet_replica.cc:333] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.941550  5410 raft_consensus.cc:2243] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:19.942610  5410 raft_consensus.cc:2272] T 520b1aa490ee4869a2814c313fac4a0e P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.945760  5410 tablet_replica.cc:333] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.948889  5410 raft_consensus.cc:2243] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:19.949617  5410 raft_consensus.cc:2272] T 542a321aae654f82803540bffcf7968f P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.953220  5410 tablet_replica.cc:333] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.954372  5410 raft_consensus.cc:2243] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:19.955567  5410 raft_consensus.cc:2272] T c7d55fddd552409bbf822686f45eda69 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.958392  5410 tablet_replica.cc:333] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
W20260709 13:20:19.958722  6563 consensus_peers.cc:597] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:19.959388  5410 raft_consensus.cc:2243] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:19.960309  5410 raft_consensus.cc:2272] T 7a72647203b143708b26072192a9c102 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.963517  5410 tablet_replica.cc:333] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.966087  5410 raft_consensus.cc:2243] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:19.967775  5410 raft_consensus.cc:2272] T e15a9820f1e1414f829e7fecece15dc0 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.970728  5410 tablet_replica.cc:333] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.971577  5410 raft_consensus.cc:2243] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:19.973165  5410 raft_consensus.cc:2272] T 35df31b6c7b2475cac55c1e114ba24dc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.976455  5410 tablet_replica.cc:333] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.977375  5410 raft_consensus.cc:2243] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:19.978319  5410 raft_consensus.cc:2272] T d83f8b945cd641db90647da0461be4a3 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:19.979434  6692 debug-util.cc:398] Leaking SignalData structure 0x7b080015dd60 after lost signal to thread 5413
W20260709 13:20:19.980552  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08002af680 after lost signal to thread 6454
I20260709 13:20:19.981469  5410 tablet_replica.cc:333] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
W20260709 13:20:19.981840  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800300840 after lost signal to thread 6621
I20260709 13:20:19.982422  5410 raft_consensus.cc:2243] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:20:19.982555  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:19.982996  6692 debug-util.cc:398] Leaking SignalData structure 0x7b08001f0a20 after lost signal to thread 6695
I20260709 13:20:19.983862  5410 raft_consensus.cc:2272] T 23ba4e226865438d85ee0c061786f3db P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:19.984308  6692 debug-util.cc:398] Leaking SignalData structure 0x7b0800269080 after lost signal to thread 7261
W20260709 13:20:19.984572  6637 consensus_peers.cc:597] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:19.986745  5410 tablet_replica.cc:333] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.987708  5410 raft_consensus.cc:2243] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:19.989176  5410 raft_consensus.cc:2272] T b75fb632ebbc415092b08b38d0835144 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.994562  5410 tablet_replica.cc:333] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:19.996089  5410 raft_consensus.cc:2243] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.000247  5410 raft_consensus.cc:2272] T 9fc18602dfbd4fc9897b6b696b497f73 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:20.005479  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.006228  5410 tablet_replica.cc:333] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.007275  5410 raft_consensus.cc:2243] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:20.012068  6637 consensus_peers.cc:597] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.013454  5410 raft_consensus.cc:2272] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.016815  5410 tablet_replica.cc:333] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.017807  5410 raft_consensus.cc:2243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.018498  5410 raft_consensus.cc:2272] T c9bbf06685e047fdaa1b6bc881dc7a93 P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.023741  5410 tablet_replica.cc:333] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.026297  5410 raft_consensus.cc:2243] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.027655  5410 raft_consensus.cc:2272] T 91d990297ed0410d894aa69d60271fe7 P 3ceee54e49d041be9f7f48a2052709c5 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.030918  5410 tablet_replica.cc:333] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.031814  5410 raft_consensus.cc:2243] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:20.032217  7200 consensus_peers.cc:597] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.032788  5410 raft_consensus.cc:2272] T e1a87665f0ba4296b0ba819fcb712842 P 3ceee54e49d041be9f7f48a2052709c5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.039820  5410 tablet_replica.cc:333] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.040823  5410 raft_consensus.cc:2243] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:20:20.040891  7200 consensus_peers.cc:597] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.042761  5410 raft_consensus.cc:2272] T c98ecd701e924e049261801a6e8b1e3b P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.045747  5410 tablet_replica.cc:333] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.046674  5410 raft_consensus.cc:2243] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.048265  5410 raft_consensus.cc:2272] T 17b598f21a79476cabdf3f5191b4f9a8 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.051442  5410 tablet_replica.cc:333] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.052310  5410 raft_consensus.cc:2243] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.053498  5410 raft_consensus.cc:2272] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.056722  5410 tablet_replica.cc:333] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.057516  5410 raft_consensus.cc:2243] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.060232  5410 raft_consensus.cc:2272] T c84605b85a2f4c4b839274e0724ef52b P 3ceee54e49d041be9f7f48a2052709c5 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:20.066232  6637 consensus_peers.cc:597] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:20.069154  5410 tablet_replica.cc:333] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.070442  5410 raft_consensus.cc:2243] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.071759  5410 raft_consensus.cc:2272] T e5a761916a9242a99d2d5b2e54ffda61 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.075980  5410 tablet_replica.cc:333] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.076913  5410 raft_consensus.cc:2243] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.078364  5410 raft_consensus.cc:2272] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.081271  5410 tablet_replica.cc:333] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.082096  5410 raft_consensus.cc:2243] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.089164  5410 raft_consensus.cc:2272] T 6be297bb363c43c99d04cccfc55bec8f P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.093321  5410 tablet_replica.cc:333] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
W20260709 13:20:20.093272  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.094532  5410 raft_consensus.cc:2243] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.095314  5410 raft_consensus.cc:2272] T 0d4a8b90b128485fa69bc10112de24e9 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.098441  5410 tablet_replica.cc:333] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.100991  5410 raft_consensus.cc:2243] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.103834  5410 raft_consensus.cc:2272] T 1c3ba394f290437b90045c4a98fb75ca P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:20.104763  7200 consensus_peers.cc:597] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.107718  5410 tablet_replica.cc:333] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.108620  5410 raft_consensus.cc:2243] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.109750  5410 raft_consensus.cc:2272] T 85490b85caca40d19a25af108b1da46a P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:20.109938  7200 consensus_peers.cc:597] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.112008  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.112815  5410 tablet_replica.cc:333] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.124233  5410 raft_consensus.cc:2243] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.127836  5410 raft_consensus.cc:2272] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 3ceee54e49d041be9f7f48a2052709c5 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.134407  5410 tablet_replica.cc:333] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.136584  5410 raft_consensus.cc:2243] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.144048  5410 raft_consensus.cc:2272] T 6eb5773be2294301bdb1bbedf5765666 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.148574  5410 tablet_replica.cc:333] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.149667  5410 raft_consensus.cc:2243] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:20.150048  6637 consensus_peers.cc:597] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.148501  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.150482  5410 raft_consensus.cc:2272] T 60fb01da2cc5490ebbbca7ec57b64a20 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.153903  5410 tablet_replica.cc:333] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.154898  5410 raft_consensus.cc:2243] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.155689  5410 raft_consensus.cc:2272] T 8885ae897ec84eca940e2aa65f7311b5 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.158659  5410 tablet_replica.cc:333] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.159509  5410 raft_consensus.cc:2243] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.160761  5410 raft_consensus.cc:2272] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 3ceee54e49d041be9f7f48a2052709c5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.163718  5410 tablet_replica.cc:333] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.164575  5410 raft_consensus.cc:2243] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.165354  5410 raft_consensus.cc:2272] T 627436404a154d80a882f4900dff2948 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.168488  5410 tablet_replica.cc:333] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.169406  5410 raft_consensus.cc:2243] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.170969  5410 raft_consensus.cc:2272] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.173849  5410 tablet_replica.cc:333] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.174696  5410 raft_consensus.cc:2243] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.175974  5410 raft_consensus.cc:2272] T 389b5012cd8548a099f7dec661e47e4f P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.178748  5410 tablet_replica.cc:333] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.179553  5410 raft_consensus.cc:2243] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.180565  5410 raft_consensus.cc:2272] T 17e76729a2fc4273abedd8be9331a882 P 3ceee54e49d041be9f7f48a2052709c5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.183674  5410 tablet_replica.cc:333] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.184569  5410 raft_consensus.cc:2243] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.186134  5410 raft_consensus.cc:2272] T 835ac063488a40878ccb02b662d3898b P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.189284  5410 tablet_replica.cc:333] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.190160  5410 raft_consensus.cc:2243] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.190990  5410 raft_consensus.cc:2272] T ce959bc5e0b4481abad1a3bef3085967 P 3ceee54e49d041be9f7f48a2052709c5 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.194002  5410 tablet_replica.cc:333] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.194864  5410 raft_consensus.cc:2243] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.195945  5410 raft_consensus.cc:2272] T a758d1835d6148668ab495504660d72a P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.198971  5410 tablet_replica.cc:333] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.199774  5410 raft_consensus.cc:2243] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:20:20.200706  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.200903  5410 raft_consensus.cc:2272] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 3ceee54e49d041be9f7f48a2052709c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.203909  5410 tablet_replica.cc:333] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.204742  5410 raft_consensus.cc:2243] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.205901  5410 raft_consensus.cc:2272] T f4112f607ad043a3889d117a4fb6ca59 P 3ceee54e49d041be9f7f48a2052709c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.208616  5410 tablet_replica.cc:333] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.209453  5410 raft_consensus.cc:2243] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.210106  5410 raft_consensus.cc:2272] T d598d50f158e42a6b8d5e297f271951d P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.212869  5410 tablet_replica.cc:333] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
W20260709 13:20:20.213523  7200 consensus_peers.cc:597] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.213717  5410 raft_consensus.cc:2243] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.215119  5410 raft_consensus.cc:2272] T 1a2845fc76124deab8a20512c6112df7 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.218127  5410 tablet_replica.cc:333] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.218976  5410 raft_consensus.cc:2243] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.219622  5410 raft_consensus.cc:2272] T 7365731c56504a678dc1a5ceac3f8da4 P 3ceee54e49d041be9f7f48a2052709c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.222492  5410 tablet_replica.cc:333] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.223342  5410 raft_consensus.cc:2243] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:20.224529  5410 raft_consensus.cc:2272] T f9f5da6098b64b229cd872c6346351f3 P 3ceee54e49d041be9f7f48a2052709c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.227355  5410 tablet_replica.cc:333] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.228085  5410 raft_consensus.cc:2243] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.228698  5410 raft_consensus.cc:2272] T 56ba1578b97b4faa966168899769e067 P 3ceee54e49d041be9f7f48a2052709c5 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.231348  5410 tablet_replica.cc:333] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.232038  5410 raft_consensus.cc:2243] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.232720  5410 raft_consensus.cc:2272] T c0e818d5c3014869b85a1d5b725c58cf P 3ceee54e49d041be9f7f48a2052709c5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:20.235217  5410 tablet_replica.cc:333] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5: stopping tablet replica
I20260709 13:20:20.235853  5410 raft_consensus.cc:2243] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:20.236627  5410 raft_consensus.cc:2272] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 3ceee54e49d041be9f7f48a2052709c5 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:20.238155  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:20.277496  6637 consensus_peers.cc:597] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.320878  7200 consensus_peers.cc:597] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.361505  6561 connection.cc:466] server connection from 127.0.0.1:34536 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34536 (request call id 2264) could send its response
W20260709 13:20:20.364548  7200 consensus_peers.cc:597] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: recv got EOF from 127.5.72.130:37977 (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.367403  6637 consensus_peers.cc:597] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:20.421860  7200 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:20.450985  6637 consensus_peers.cc:597] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:20.534003  6637 consensus_peers.cc:597] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:20.571569  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:20.574734  7200 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:20.761861  6637 consensus_peers.cc:597] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:20.813797  7200 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:20.858385  6637 consensus_peers.cc:597] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:20.989452  6637 consensus_peers.cc:597] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:20.994100  6637 consensus_peers.cc:597] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:21.008463 15134 raft_consensus.cc:493] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.009300 15134 raft_consensus.cc:515] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.012418 15134 leader_election.cc:290] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.019691  7200 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.020663  7200 leader_election.cc:336] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.021596 15131 raft_consensus.cc:493] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.022100 15131 raft_consensus.cc:515] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.024016 15131 leader_election.cc:290] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.021102  7200 leader_election.cc:304] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.027498 15134 raft_consensus.cc:2749] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:21.038789  6637 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.039433  6637 leader_election.cc:336] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.039773  6637 leader_election.cc:304] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.040412 15131 raft_consensus.cc:2749] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:20:21.057453  7200 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:20:21.130697 15128 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.131192 15128 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.133226 15128 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:21.134913  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:21.140127  7200 leader_election.cc:336] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.140589  7200 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:21.141933 15128 raft_consensus.cc:2749] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:21.145896  7200 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:21.188369 15128 raft_consensus.cc:493] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.192636 15134 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.193272 15134 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.195917 15134 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:21.188951 15128 raft_consensus.cc:515] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.197391  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:21.214742 15128 leader_election.cc:290] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.217875  7200 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.218453  7200 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.223021 15135 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:21.229111  7200 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.230840 15136 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:20:21.231608  6637 consensus_peers.cc:597] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:21.231428 15136 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:21.234225  6637 consensus_peers.cc:597] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:21.235226  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:21.237221 15136 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
W20260709 13:20:21.238977  7200 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.240161  7200 leader_election.cc:336] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.240593  7200 leader_election.cc:304] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.241271 15136 raft_consensus.cc:2749] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:21.244378  7200 leader_election.cc:336] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.244837  7200 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:21.245548 15136 raft_consensus.cc:2749] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:21.247248  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:20:21.254730 15131 raft_consensus.cc:493] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.255263 15131 raft_consensus.cc:515] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.256994 15131 leader_election.cc:290] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.260991  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.262497  6637 leader_election.cc:336] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.262898  6637 leader_election.cc:304] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.263499 15131 raft_consensus.cc:2749] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:21.286420 15131 raft_consensus.cc:493] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.287449 15091 raft_consensus.cc:493] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.286963 15131 raft_consensus.cc:515] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.287928 15091 raft_consensus.cc:515] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.289780 15091 leader_election.cc:290] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.289815 15131 leader_election.cc:290] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.295214  6637 leader_election.cc:336] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.295746  6637 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.296367  6637 leader_election.cc:336] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.296777  6637 leader_election.cc:304] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.297386 15131 raft_consensus.cc:2749] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:21.297787  6637 leader_election.cc:336] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.298547  6637 leader_election.cc:304] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.299247 15131 raft_consensus.cc:2749] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:21.334976 15136 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.335614 15136 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.341324 15136 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:21.341962  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
W20260709 13:20:21.350101  7200 leader_election.cc:336] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.351081  7198 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:21.352370 15136 raft_consensus.cc:2749] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:21.357251  6637 consensus_peers.cc:597] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:21.358767 15135 raft_consensus.cc:493] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.359323 15135 raft_consensus.cc:515] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.361445 15135 leader_election.cc:290] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.365394 15131 raft_consensus.cc:493] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.366859 15091 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.367460 15091 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:21.368753  7200 leader_election.cc:336] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.369556  7200 leader_election.cc:336] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.369964  7200 leader_election.cc:304] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.371536  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:21.372565 15135 raft_consensus.cc:2749] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:21.372980  7230 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:20:21.375859  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:21.377617 15137 raft_consensus.cc:493] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:20:21.378221  6637 leader_election.cc:336] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.379771 15091 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.383499 15139 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.382723 15137 raft_consensus.cc:515] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.384358 15091 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:21.362291 15136 raft_consensus.cc:493] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.384999 15091 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.386943 15091 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:21.385861 15135 raft_consensus.cc:493] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.387822 15135 raft_consensus.cc:515] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.390354 15135 leader_election.cc:290] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.384168 15139 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.366129 15131 raft_consensus.cc:515] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.385119 15136 raft_consensus.cc:515] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.396644 15131 leader_election.cc:290] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.397923 15131 raft_consensus.cc:493] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.400131 15131 raft_consensus.cc:515] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.402148 15131 leader_election.cc:290] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.399665 15139 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.402496 15136 leader_election.cc:290] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.403792  6637 leader_election.cc:336] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.404553 15135 raft_consensus.cc:493] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.408358 15135 raft_consensus.cc:515] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.411192 15135 leader_election.cc:290] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.413115 15137 leader_election.cc:290] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.406358  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.421387  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:21.422084  7230 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
W20260709 13:20:21.424539  7200 leader_election.cc:336] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.425102  7200 leader_election.cc:304] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:21.420723  6637 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.405864  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
I20260709 13:20:21.418951 15139 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.426127 15139 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.427467 15138 raft_consensus.cc:493] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
W20260709 13:20:21.427280  6637 leader_election.cc:336] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.428320 15139 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.428030 15138 raft_consensus.cc:515] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.428419  6637 leader_election.cc:304] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.425175 15140 raft_consensus.cc:493] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.417351 15135 raft_consensus.cc:493] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.416640 15091 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.429369 15139 raft_consensus.cc:2749] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:21.429838 15135 raft_consensus.cc:515] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.426960  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:21.430557 15138 leader_election.cc:290] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.431957 15128 raft_consensus.cc:2749] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:20:21.433241 15135 leader_election.cc:290] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.433928 15139 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:21.434957  6637 leader_election.cc:336] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.438294 15139 raft_consensus.cc:493] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.438813 15139 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:21.431540 15091 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.436084  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:21.442975  7236 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:20:21.436147  7230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a761916a9242a99d2d5b2e54ffda61" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:21.444430  7230 raft_consensus.cc:3060] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:21.444314  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:21.433440 15140 raft_consensus.cc:515] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.449529 15091 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:21.449931 15091 raft_consensus.cc:493] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.450301 15091 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:21.452028 15140 leader_election.cc:290] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.465857 15139 raft_consensus.cc:515] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:21.471284  7200 leader_election.cc:336] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.471567  6637 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.477530  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f5da6098b64b229cd872c6346351f3" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:21.478193  7236 raft_consensus.cc:3060] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:21.480913  6637 leader_election.cc:336] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.481806  6637 leader_election.cc:336] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.482853  6637 leader_election.cc:336] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.485256  6637 leader_election.cc:336] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.487946  7236 raft_consensus.cc:2468] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:20:21.489496  6636 leader_election.cc:304] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:21.474472 15140 raft_consensus.cc:493] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.474963 15091 raft_consensus.cc:515] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.490332 15138 raft_consensus.cc:2804] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:21.490923 15138 raft_consensus.cc:697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:21.492189 15091 leader_election.cc:290] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.491888 15138 consensus_queue.cc:237] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.494428  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb598c0b9b54f788ae6eefe2894e3a5" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:21.495112  7234 raft_consensus.cc:3060] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:21.485816 15135 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.473838  7230 raft_consensus.cc:2468] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 21.
I20260709 13:20:21.495911 15135 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.498749 15139 leader_election.cc:290] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.498497  6636 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5
I20260709 13:20:21.498544 15135 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:21.505333 15138 raft_consensus.cc:2804] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:20:21.486518  7200 leader_election.cc:336] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.506043  7200 leader_election.cc:304] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.506150 15138 raft_consensus.cc:697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:21.507330  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "64b95375974b48d18c596a23c9cac75c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "53fee9894731461a821a131be24dbbca" is_pre_election: true
I20260709 13:20:21.508217 15138 consensus_queue.cc:237] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.507417 15140 raft_consensus.cc:515] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } attrs { replace: true } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.513800 15140 leader_election.cc:290] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.513953  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75fb632ebbc415092b08b38d0835144" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
I20260709 13:20:21.487080  6637 leader_election.cc:304] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.514649  7236 raft_consensus.cc:2468] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 24.
W20260709 13:20:21.515265  6637 leader_election.cc:336] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.516218 15135 raft_consensus.cc:2749] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:21.516700  6637 leader_election.cc:304] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
W20260709 13:20:21.517712  7200 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.515774  7234 raft_consensus.cc:2468] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 20.
I20260709 13:20:21.518950 15139 raft_consensus.cc:2749] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:21.520314  6636 leader_election.cc:304] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:21.521572 15091 raft_consensus.cc:2749] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:21.521749 15139 raft_consensus.cc:2804] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:21.522483 15139 raft_consensus.cc:697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:21.526722 15139 consensus_queue.cc:237] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:21.525292  6637 leader_election.cc:336] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.532966  6637 leader_election.cc:304] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.534423 15140 raft_consensus.cc:2749] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:21.537323 15135 raft_consensus.cc:493] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.538097 15135 raft_consensus.cc:515] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.541230 15135 leader_election.cc:290] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.549257  7200 leader_election.cc:336] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.549834  7200 leader_election.cc:304] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.550719 15128 raft_consensus.cc:2749] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:21.559573  6637 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.564543 15138 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.565106 15138 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.565614 15140 raft_consensus.cc:493] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.566306 15140 raft_consensus.cc:515] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } }
I20260709 13:20:21.567341 15138 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
I20260709 13:20:21.568615  7234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c" is_pre_election: true
W20260709 13:20:21.569329  6637 leader_election.cc:336] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.569494  7234 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 18.
I20260709 13:20:21.568763 15140 leader_election.cc:290] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.571276  6637 leader_election.cc:304] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.572279  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:21.572937 15140 raft_consensus.cc:2749] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:21.573210 15138 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:21.573910 15138 raft_consensus.cc:493] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.574308 15138 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:21.577035  7200 leader_election.cc:336] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.579617  6637 leader_election.cc:336] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.580843  6414 catalog_manager.cc:5697] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 19 to 20, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 20 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:21.581467 15138 raft_consensus.cc:515] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
W20260709 13:20:21.581915  6637 leader_election.cc:336] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.582935  6637 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.582578  6414 catalog_manager.cc:5697] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 20 to 21, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 21 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:21.584357  6414 catalog_manager.cc:5697] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 19 to 20, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 20 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:21.587641 15138 leader_election.cc:290] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Requested vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 64b95375974b48d18c596a23c9cac75c (127.5.72.132:36575)
W20260709 13:20:21.594833  7200 leader_election.cc:336] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.595417  7200 leader_election.cc:304] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.598272 15146 raft_consensus.cc:493] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.599005 15146 raft_consensus.cc:515] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.600278 15128 raft_consensus.cc:2749] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:21.601889 15146 leader_election.cc:290] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
I20260709 13:20:21.606480  7236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91a4b9d0715e4a3ba0f22cebfe819bb1" candidate_uuid: "53fee9894731461a821a131be24dbbca" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "64b95375974b48d18c596a23c9cac75c"
I20260709 13:20:21.607151  7236 raft_consensus.cc:3060] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:21.610201  6637 leader_election.cc:336] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.614923  7236 raft_consensus.cc:2468] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53fee9894731461a821a131be24dbbca in term 19.
I20260709 13:20:21.627048  6636 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 
I20260709 13:20:21.628072 15138 raft_consensus.cc:2804] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:21.628522 15138 raft_consensus.cc:697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Becoming Leader. State: Replica: 53fee9894731461a821a131be24dbbca, State: Running, Role: LEADER
I20260709 13:20:21.625978  6637 leader_election.cc:304] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca, 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.629485 15138 consensus_queue.cc:237] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.630110 15141 raft_consensus.cc:2749] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:21.628535  7200 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.631439  7200 leader_election.cc:304] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:21.632383 15147 raft_consensus.cc:2749] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:21.640813  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:21.644678  7200 leader_election.cc:336] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.645368  7200 leader_election.cc:336] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.651839  7200 leader_election.cc:336] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.652523  7200 leader_election.cc:304] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.653620 15147 raft_consensus.cc:2749] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:21.654295  7200 leader_election.cc:336] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.654980  7200 leader_election.cc:304] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.657228 15135 raft_consensus.cc:2749] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:21.660882  6637 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.661794  6637 leader_election.cc:336] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.663182  6637 leader_election.cc:336] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:21.664973  6637 leader_election.cc:336] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:21.665452  6637 leader_election.cc:304] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53fee9894731461a821a131be24dbbca; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.665781 15147 raft_consensus.cc:493] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3ceee54e49d041be9f7f48a2052709c5)
I20260709 13:20:21.666379 15147 raft_consensus.cc:515] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:21.667196 15138 raft_consensus.cc:2749] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:21.669013 15147 leader_election.cc:290] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977)
W20260709 13:20:21.678333  7200 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
W20260709 13:20:21.679092  7200 leader_election.cc:336] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:21.679519  7200 leader_election.cc:304] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 921c423406da4f289884afca51ca198f
I20260709 13:20:21.680907 15149 raft_consensus.cc:2749] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:21.691275  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
I20260709 13:20:21.700248  6414 catalog_manager.cc:5697] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca reported cstate change: term changed from 18 to 19, leader changed from 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130) to 53fee9894731461a821a131be24dbbca (127.5.72.131). New cstate: current_term: 19 leader_uuid: "53fee9894731461a821a131be24dbbca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:21.731292  7198 consensus_peers.cc:597] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:21.755870  7198 consensus_peers.cc:597] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:21.781234  7198 consensus_peers.cc:597] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:21.786331  7198 consensus_peers.cc:597] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.793641  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:21.794713  5410 tablet_replica.cc:333] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.795682  5410 raft_consensus.cc:2243] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.796490  5410 raft_consensus.cc:2272] T c7d55fddd552409bbf822686f45eda69 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.798753  5410 tablet_replica.cc:333] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.799661  5410 raft_consensus.cc:2243] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.800382  5410 raft_consensus.cc:2272] T e15a9820f1e1414f829e7fecece15dc0 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.802570  5410 tablet_replica.cc:333] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.803444  5410 raft_consensus.cc:2243] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.804374  5410 raft_consensus.cc:2272] T 520b1aa490ee4869a2814c313fac4a0e P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.807145  5410 tablet_replica.cc:333] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.807996  5410 raft_consensus.cc:2243] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.808693  5410 raft_consensus.cc:2272] T f4112f607ad043a3889d117a4fb6ca59 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:21.810708  7198 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.811507  5410 tablet_replica.cc:333] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.812269  5410 raft_consensus.cc:2243] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.812851  5410 raft_consensus.cc:2272] T 7a72647203b143708b26072192a9c102 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.815034  5410 tablet_replica.cc:333] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.815656  5410 raft_consensus.cc:2243] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.816699  5410 raft_consensus.cc:2272] T 35df31b6c7b2475cac55c1e114ba24dc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.819489  5410 tablet_replica.cc:333] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.820386  5410 raft_consensus.cc:2243] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.821029  5410 raft_consensus.cc:2272] T f67b820295ac47a4b54e21400f8b4201 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.824190  5410 tablet_replica.cc:333] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.825181  5410 raft_consensus.cc:2243] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:20:21.825870  6637 consensus_peers.cc:597] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:21.827069  5410 raft_consensus.cc:2272] T 938e1826b2ce4babab3a70e5219411a4 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.829674  5410 tablet_replica.cc:333] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.830538  5410 raft_consensus.cc:2243] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.831142  5410 raft_consensus.cc:2272] T b2c67ec2db9643088bc65fe9d83aa732 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.833978  5410 tablet_replica.cc:333] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.834734  5410 raft_consensus.cc:2243] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.835772  5410 raft_consensus.cc:2272] T 34cabfbde16f4e10affcaec358aa0dd4 P 53fee9894731461a821a131be24dbbca [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.837903  5410 tablet_replica.cc:333] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.838677  5410 raft_consensus.cc:2243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.839592  5410 raft_consensus.cc:2272] T c9bbf06685e047fdaa1b6bc881dc7a93 P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.842306  5410 tablet_replica.cc:333] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.843197  5410 raft_consensus.cc:2243] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.844309  5410 raft_consensus.cc:2272] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.846673  5410 tablet_replica.cc:333] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.847443  5410 raft_consensus.cc:2243] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.848206  5410 raft_consensus.cc:2272] T 9fc18602dfbd4fc9897b6b696b497f73 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.851204  5410 tablet_replica.cc:333] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.852077  5410 raft_consensus.cc:2243] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.852756  5410 raft_consensus.cc:2272] T b75fb632ebbc415092b08b38d0835144 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.855612  5410 tablet_replica.cc:333] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.856427  5410 raft_consensus.cc:2243] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.857386  5410 raft_consensus.cc:2272] T 91d990297ed0410d894aa69d60271fe7 P 53fee9894731461a821a131be24dbbca [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.860344  5410 tablet_replica.cc:333] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.861300  5410 raft_consensus.cc:2243] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.862357  5410 raft_consensus.cc:2272] T e1a87665f0ba4296b0ba819fcb712842 P 53fee9894731461a821a131be24dbbca [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.865211  5410 tablet_replica.cc:333] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.866103  5410 raft_consensus.cc:2243] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.866811  5410 raft_consensus.cc:2272] T c98ecd701e924e049261801a6e8b1e3b P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.869726  5410 tablet_replica.cc:333] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.870482  5410 raft_consensus.cc:2243] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.871718  5410 raft_consensus.cc:2272] T 2cbaf7daabfe40028228e412b8f9214b P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:21.873457  7198 consensus_peers.cc:597] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.874437  5410 tablet_replica.cc:333] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.875444  5410 raft_consensus.cc:2243] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.876188  5410 raft_consensus.cc:2272] T 60fb01da2cc5490ebbbca7ec57b64a20 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.878890  5410 tablet_replica.cc:333] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.879706  5410 raft_consensus.cc:2243] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.880360  5410 raft_consensus.cc:2272] T 8885ae897ec84eca940e2aa65f7311b5 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.882349  5410 tablet_replica.cc:333] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.882952  5410 raft_consensus.cc:2243] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.883791  5410 pending_rounds.cc:70] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Trying to abort 1 pending ops.
I20260709 13:20:21.883982  5410 pending_rounds.cc:77] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca: Aborting op as it isn't in flight: id { term: 21 index: 18 } timestamp: 7305638795308216320 op_type: NO_OP noop_request { }
I20260709 13:20:21.884291  5410 raft_consensus.cc:2889] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 21 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:21.884567  5410 raft_consensus.cc:2272] T e5a761916a9242a99d2d5b2e54ffda61 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.887205  5410 tablet_replica.cc:333] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.887862  5410 raft_consensus.cc:2243] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.888424  5410 raft_consensus.cc:2272] T 7fc1d5a094e64eadb9c93f5e81f30fbc P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:21.890340  7198 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.890756  5410 tablet_replica.cc:333] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.891350  5410 raft_consensus.cc:2243] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.891954  5410 raft_consensus.cc:2272] T c84605b85a2f4c4b839274e0724ef52b P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.893694  5410 tablet_replica.cc:333] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.894225  5410 raft_consensus.cc:2243] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.895080  5410 raft_consensus.cc:2272] T bab5f01bd4774a168d3c01940a615cce P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.897025  5410 tablet_replica.cc:333] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.897581  5410 raft_consensus.cc:2243] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.898413  5410 raft_consensus.cc:2272] T ff27e205e84e4edf946bc02c573a2c25 P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.900096  5410 tablet_replica.cc:333] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.900713  5410 raft_consensus.cc:2243] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.901489  5410 raft_consensus.cc:2272] T 0d4a8b90b128485fa69bc10112de24e9 P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.903301  5410 tablet_replica.cc:333] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.903899  5410 raft_consensus.cc:2243] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.904518  5410 raft_consensus.cc:2272] T 627436404a154d80a882f4900dff2948 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.906709  5410 tablet_replica.cc:333] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.907388  5410 raft_consensus.cc:2243] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.908227  5410 pending_rounds.cc:70] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Trying to abort 1 pending ops.
I20260709 13:20:21.908440  5410 pending_rounds.cc:77] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca: Aborting op as it isn't in flight: id { term: 20 index: 14 } timestamp: 7305638795458424832 op_type: NO_OP noop_request { }
I20260709 13:20:21.908778  5410 raft_consensus.cc:2889] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:21.909118  5410 raft_consensus.cc:2272] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.911355  5410 tablet_replica.cc:333] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.911891  5410 raft_consensus.cc:2243] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.912545  5410 pending_rounds.cc:70] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Trying to abort 1 pending ops.
I20260709 13:20:21.912739  5410 pending_rounds.cc:77] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca: Aborting op as it isn't in flight: id { term: 19 index: 16 } timestamp: 7305638795804381184 op_type: NO_OP noop_request { }
I20260709 13:20:21.912997  5410 raft_consensus.cc:2889] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:21.913254  5410 raft_consensus.cc:2272] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.922049  5410 tablet_replica.cc:333] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.922833  5410 raft_consensus.cc:2243] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.923671  5410 raft_consensus.cc:2272] T f8a48eeccdac45838ef8456e5e9332b5 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.925959  5410 tablet_replica.cc:333] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.926671  5410 raft_consensus.cc:2243] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:21.927078  7198 consensus_peers.cc:597] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.929000  5410 raft_consensus.cc:2272] T 389b5012cd8548a099f7dec661e47e4f P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.931295  5410 tablet_replica.cc:333] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
W20260709 13:20:21.931420  7198 consensus_peers.cc:597] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.931989  5410 raft_consensus.cc:2243] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.932516  5410 raft_consensus.cc:2272] T 17e76729a2fc4273abedd8be9331a882 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.934397  5410 tablet_replica.cc:333] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.934945  5410 raft_consensus.cc:2243] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.935664  5410 pending_rounds.cc:70] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Trying to abort 1 pending ops.
I20260709 13:20:21.935854  5410 pending_rounds.cc:77] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca: Aborting op as it isn't in flight: id { term: 20 index: 16 } timestamp: 7305638795240919040 op_type: NO_OP noop_request { }
I20260709 13:20:21.936179  5410 raft_consensus.cc:2889] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:21.936460  5410 raft_consensus.cc:2272] T f9f5da6098b64b229cd872c6346351f3 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.938647  5410 tablet_replica.cc:333] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.939277  5410 raft_consensus.cc:2243] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.939779  5410 raft_consensus.cc:2272] T a758d1835d6148668ab495504660d72a P 53fee9894731461a821a131be24dbbca [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.941406  5410 tablet_replica.cc:333] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.941893  5410 raft_consensus.cc:2243] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.942376  5410 raft_consensus.cc:2272] T ffa871a3f5334bd49b9c75539f218332 P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.944259  5410 tablet_replica.cc:333] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.944725  5410 raft_consensus.cc:2243] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.945564  5410 raft_consensus.cc:2272] T c0e818d5c3014869b85a1d5b725c58cf P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.947696  5410 tablet_replica.cc:333] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.948303  5410 raft_consensus.cc:2243] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.948741  5410 raft_consensus.cc:2272] T 835ac063488a40878ccb02b662d3898b P 53fee9894731461a821a131be24dbbca [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.950701  5410 tablet_replica.cc:333] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.951182  5410 raft_consensus.cc:2243] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.952142  5410 raft_consensus.cc:2272] T ce959bc5e0b4481abad1a3bef3085967 P 53fee9894731461a821a131be24dbbca [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.954528  5410 tablet_replica.cc:333] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.955235  5410 raft_consensus.cc:2243] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.956110  5410 raft_consensus.cc:2272] T 004d58f566fd4b6d954bc425675471fa P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:21.957257  7198 consensus_peers.cc:597] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.958540  5410 tablet_replica.cc:333] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.959147  5410 raft_consensus.cc:2243] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.959956  5410 raft_consensus.cc:2272] T 943ce377e70a422a8376b294da2840fa P 53fee9894731461a821a131be24dbbca [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.961607  5410 tablet_replica.cc:333] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.962121  5410 raft_consensus.cc:2243] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.962567  5410 raft_consensus.cc:2272] T 4c870451bc2945e8b0a88e8fedf73ce6 P 53fee9894731461a821a131be24dbbca [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.964432  5410 tablet_replica.cc:333] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.964912  5410 raft_consensus.cc:2243] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.965354  5410 raft_consensus.cc:2272] T f4f0dad199224b8182ae036d97281b72 P 53fee9894731461a821a131be24dbbca [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.967145  5410 tablet_replica.cc:333] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.967619  5410 raft_consensus.cc:2243] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:21.968410  5410 raft_consensus.cc:2272] T d598d50f158e42a6b8d5e297f271951d P 53fee9894731461a821a131be24dbbca [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.970062  5410 tablet_replica.cc:333] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.970542  5410 raft_consensus.cc:2243] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.971012  5410 raft_consensus.cc:2272] T 40b7cf612223418ea8fe2d636986bb3f P 53fee9894731461a821a131be24dbbca [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:21.972820  5410 tablet_replica.cc:333] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca: stopping tablet replica
I20260709 13:20:21.973289  5410 raft_consensus.cc:2243] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:21.973763  5410 raft_consensus.cc:2272] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 53fee9894731461a821a131be24dbbca [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:22.007992  7198 consensus_peers.cc:597] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.035832  7198 consensus_peers.cc:597] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.076305  7198 consensus_peers.cc:597] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.124787  7198 consensus_peers.cc:597] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.178642  7198 consensus_peers.cc:597] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.278945  7198 consensus_peers.cc:597] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c -> Peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Couldn't send request to peer 53fee9894731461a821a131be24dbbca. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.297505  7200 consensus_peers.cc:597] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.311846  7200 consensus_peers.cc:597] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.349596  7200 consensus_peers.cc:597] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.353367  6407 catalog_manager.cc:4755] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): ChangeConfig:REMOVE_PEER RPC failed for tablet b75fb632ebbc415092b08b38d0835144: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
W20260709 13:20:22.474745  7200 consensus_peers.cc:597] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.489809  7200 consensus_peers.cc:597] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.573853  7200 consensus_peers.cc:597] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.586293  7200 consensus_peers.cc:597] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.598177  7200 consensus_peers.cc:597] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.678051  7200 consensus_peers.cc:597] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.695461  7200 consensus_peers.cc:597] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.705365  7200 consensus_peers.cc:597] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:22.776124  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
W20260709 13:20:22.837302  7200 consensus_peers.cc:597] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:22.859992  7200 consensus_peers.cc:597] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c -> Peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Couldn't send request to peer 921c423406da4f289884afca51ca198f. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:22.867906  5410 tablet_server.cc:179] TabletServer@127.5.72.132:0 shutting down...
I20260709 13:20:22.920400  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:22.920513 15155 raft_consensus.cc:493] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:20:22.921341  5410 tablet_replica.cc:333] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.921141 15155 raft_consensus.cc:515] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:22.922328  5410 raft_consensus.cc:2243] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.923343  5410 raft_consensus.cc:2272] T 938e1826b2ce4babab3a70e5219411a4 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.923605 15155 leader_election.cc:290] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:22.924921 15155 raft_consensus.cc:493] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:20:22.925452 15155 raft_consensus.cc:515] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "921c423406da4f289884afca51ca198f" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34121 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:22.926524  5410 tablet_replica.cc:333] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.927721 15155 leader_election.cc:290] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 921c423406da4f289884afca51ca198f (127.5.72.129:34121), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:22.927740  5410 raft_consensus.cc:2243] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.929782  5410 raft_consensus.cc:2272] T b75fb632ebbc415092b08b38d0835144 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:22.930521  7200 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:22.932981  5410 tablet_replica.cc:333] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
W20260709 13:20:22.933845  7200 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 921c423406da4f289884afca51ca198f (127.5.72.129:34121): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34121: connect: Connection refused (error 111)
I20260709 13:20:22.933904  5410 raft_consensus.cc:2243] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:20:22.934859  7198 leader_election.cc:336] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111)
I20260709 13:20:22.935235  7198 leader_election.cc:304] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:22.935292  5410 raft_consensus.cc:2272] T 7a72647203b143708b26072192a9c102 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.935942 15155 raft_consensus.cc:2749] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:22.938004  7198 leader_election.cc:336] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111)
I20260709 13:20:22.938385  5410 tablet_replica.cc:333] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.938375  7198 leader_election.cc:304] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 53fee9894731461a821a131be24dbbca, 921c423406da4f289884afca51ca198f
I20260709 13:20:22.939088 15155 raft_consensus.cc:2749] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:22.939414  5410 raft_consensus.cc:2243] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.940171  5410 raft_consensus.cc:2272] T 2fb598c0b9b54f788ae6eefe2894e3a5 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.943171  5410 tablet_replica.cc:333] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.944028  5410 raft_consensus.cc:2243] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:22.945098  5410 raft_consensus.cc:2272] T 17e76729a2fc4273abedd8be9331a882 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.948179  5410 tablet_replica.cc:333] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.949079  5410 raft_consensus.cc:2243] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.949688  5410 raft_consensus.cc:2272] T 1a2845fc76124deab8a20512c6112df7 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.952718  5410 tablet_replica.cc:333] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.953548  5410 raft_consensus.cc:2243] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:20:22.954542  5410 raft_consensus.cc:2272] T 40b7cf612223418ea8fe2d636986bb3f P 64b95375974b48d18c596a23c9cac75c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.957505  5410 tablet_replica.cc:333] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.958424  5410 raft_consensus.cc:2243] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.959113  5410 raft_consensus.cc:2272] T 943ce377e70a422a8376b294da2840fa P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.961894  5410 tablet_replica.cc:333] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.962728  5410 raft_consensus.cc:2243] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.963424  5410 raft_consensus.cc:2272] T 23ba4e226865438d85ee0c061786f3db P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.966248  5410 tablet_replica.cc:333] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.967103  5410 raft_consensus.cc:2243] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.967694  5410 raft_consensus.cc:2272] T 8fc61b2fa4ee43b7a7a91b2eb2ecd874 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.970544  5410 tablet_replica.cc:333] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.971360  5410 raft_consensus.cc:2243] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.972014  5410 raft_consensus.cc:2272] T 004d58f566fd4b6d954bc425675471fa P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.974910  5410 tablet_replica.cc:333] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.975675  5410 raft_consensus.cc:2243] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:22.976642  5410 raft_consensus.cc:2272] T 8885ae897ec84eca940e2aa65f7311b5 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.979552  5410 tablet_replica.cc:333] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.980340  5410 raft_consensus.cc:2243] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.980928  5410 raft_consensus.cc:2272] T fd28ad0fdaad48d586176f4a8e3f1aa3 P 64b95375974b48d18c596a23c9cac75c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.983799  5410 tablet_replica.cc:333] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.984581  5410 raft_consensus.cc:2243] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.985296  5410 raft_consensus.cc:2272] T 34cabfbde16f4e10affcaec358aa0dd4 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.987905  5410 tablet_replica.cc:333] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.988701  5410 raft_consensus.cc:2243] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.989269  5410 raft_consensus.cc:2272] T 85490b85caca40d19a25af108b1da46a P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.992072  5410 tablet_replica.cc:333] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.992820  5410 raft_consensus.cc:2243] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:22.993480  5410 raft_consensus.cc:2272] T 6eb5773be2294301bdb1bbedf5765666 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:22.996152  5410 tablet_replica.cc:333] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:22.996944  5410 raft_consensus.cc:2243] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:22.997943  5410 raft_consensus.cc:2272] T 60fb01da2cc5490ebbbca7ec57b64a20 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.000643  5410 tablet_replica.cc:333] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.001362  5410 raft_consensus.cc:2243] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.002357  5410 raft_consensus.cc:2272] T 35df31b6c7b2475cac55c1e114ba24dc P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.005167  5410 tablet_replica.cc:333] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.005952  5410 raft_consensus.cc:2243] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.007061  5410 raft_consensus.cc:2272] T f4f0dad199224b8182ae036d97281b72 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.009663  5410 tablet_replica.cc:333] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.010448  5410 raft_consensus.cc:2243] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.011502  5410 raft_consensus.cc:2272] T 4c870451bc2945e8b0a88e8fedf73ce6 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.014240  5410 tablet_replica.cc:333] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.015048  5410 raft_consensus.cc:2243] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.016100  5410 raft_consensus.cc:2272] T b2c67ec2db9643088bc65fe9d83aa732 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.018893  5410 tablet_replica.cc:333] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.019690  5410 raft_consensus.cc:2243] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.020700  5410 raft_consensus.cc:2272] T f67b820295ac47a4b54e21400f8b4201 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.023473  5410 tablet_replica.cc:333] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.024233  5410 raft_consensus.cc:2243] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.025456  5410 raft_consensus.cc:2272] T f8a48eeccdac45838ef8456e5e9332b5 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.028113  5410 tablet_replica.cc:333] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.028847  5410 raft_consensus.cc:2243] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.029608  5410 raft_consensus.cc:2272] T f9f5da6098b64b229cd872c6346351f3 P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.032519  5410 tablet_replica.cc:333] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.033352  5410 raft_consensus.cc:2243] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.033926  5410 raft_consensus.cc:2272] T 5d9d0fb5a0864567ac4cc6ee615911b6 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.036618  5410 tablet_replica.cc:333] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.037395  5410 raft_consensus.cc:2243] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.038473  5410 raft_consensus.cc:2272] T 2c0a546cb4d749bc83ffa9cbd0f40297 P 64b95375974b48d18c596a23c9cac75c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.041198  5410 tablet_replica.cc:333] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.041945  5410 raft_consensus.cc:2243] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.042502  5410 raft_consensus.cc:2272] T 2cbaf7daabfe40028228e412b8f9214b P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.045286  5410 tablet_replica.cc:333] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.046074  5410 raft_consensus.cc:2243] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.047118  5410 raft_consensus.cc:2272] T 7365731c56504a678dc1a5ceac3f8da4 P 64b95375974b48d18c596a23c9cac75c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.049815  5410 tablet_replica.cc:333] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.050541  5410 raft_consensus.cc:2243] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.051111  5410 raft_consensus.cc:2272] T 53f3e5f2d81c4952b6ae972bdb3a18c1 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.051733 15155 raft_consensus.cc:493] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 53fee9894731461a821a131be24dbbca)
I20260709 13:20:23.052176 15155 raft_consensus.cc:515] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:23.053887  5410 tablet_replica.cc:333] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.054014 15155 leader_election.cc:290] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:23.054811  5410 raft_consensus.cc:2243] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.055459  5410 raft_consensus.cc:2272] T 17b598f21a79476cabdf3f5191b4f9a8 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.058312  5410 tablet_replica.cc:333] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
W20260709 13:20:23.058898  7198 leader_election.cc:336] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111)
I20260709 13:20:23.059134  5410 raft_consensus.cc:2243] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:23.059765  7200 leader_election.cc:336] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:23.059834  5410 raft_consensus.cc:2272] T 91a4b9d0715e4a3ba0f22cebfe819bb1 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.060122  7200 leader_election.cc:304] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
I20260709 13:20:23.060794 15155 raft_consensus.cc:2749] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:23.062959  5410 tablet_replica.cc:333] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.063854  5410 raft_consensus.cc:2243] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.064970  5410 raft_consensus.cc:2272] T a758d1835d6148668ab495504660d72a P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.067869  5410 tablet_replica.cc:333] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.068646  5410 raft_consensus.cc:2243] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.069748  5410 raft_consensus.cc:2272] T ffa871a3f5334bd49b9c75539f218332 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.072582  5410 tablet_replica.cc:333] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.073334  5410 raft_consensus.cc:2243] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.074107  5410 raft_consensus.cc:2272] T d598d50f158e42a6b8d5e297f271951d P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.076740  5410 tablet_replica.cc:333] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.077200 15155 raft_consensus.cc:493] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.077447  5410 raft_consensus.cc:2243] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.077723 15155 raft_consensus.cc:515] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3ceee54e49d041be9f7f48a2052709c5" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 37977 } } peers { permanent_uuid: "53fee9894731461a821a131be24dbbca" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38019 } } peers { permanent_uuid: "64b95375974b48d18c596a23c9cac75c" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 36575 } attrs { promote: false } }
I20260709 13:20:23.078177  5410 raft_consensus.cc:2272] T 6be297bb363c43c99d04cccfc55bec8f P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.079910 15155 leader_election.cc:290] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977), 53fee9894731461a821a131be24dbbca (127.5.72.131:38019)
I20260709 13:20:23.081046  5410 tablet_replica.cc:333] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.081848  5410 raft_consensus.cc:2243] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.083122  5410 raft_consensus.cc:2272] T d83f8b945cd641db90647da0461be4a3 P 64b95375974b48d18c596a23c9cac75c [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:23.084414  7200 leader_election.cc:336] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:20:23.085947  5410 tablet_replica.cc:333] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.086783  5410 raft_consensus.cc:2243] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.087344  5410 raft_consensus.cc:2272] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:23.087617  7198 leader_election.cc:336] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 53fee9894731461a821a131be24dbbca (127.5.72.131:38019): Network error: Client connection negotiation failed: client connection to 127.5.72.131:38019: connect: Connection refused (error 111)
I20260709 13:20:23.088037  7198 leader_election.cc:304] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 64b95375974b48d18c596a23c9cac75c; no voters: 3ceee54e49d041be9f7f48a2052709c5, 53fee9894731461a821a131be24dbbca
W20260709 13:20:23.088493  7198 raft_consensus.cc:2686] T e5a761916a9242a99d2d5b2e54ffda61 P 64b95375974b48d18c596a23c9cac75c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:20:23.090227  5410 tablet_replica.cc:333] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.091092  5410 raft_consensus.cc:2243] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.092108  5410 raft_consensus.cc:2272] T 542a321aae654f82803540bffcf7968f P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.094825  5410 tablet_replica.cc:333] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.095568  5410 raft_consensus.cc:2243] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.096225  5410 raft_consensus.cc:2272] T ff27e205e84e4edf946bc02c573a2c25 P 64b95375974b48d18c596a23c9cac75c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.099040  5410 tablet_replica.cc:333] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.099774  5410 raft_consensus.cc:2243] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:23.100205  7200 consensus_peers.cc:597] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c -> Peer 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): Couldn't send request to peer 3ceee54e49d041be9f7f48a2052709c5. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:23.100356  5410 raft_consensus.cc:2272] T bab5f01bd4774a168d3c01940a615cce P 64b95375974b48d18c596a23c9cac75c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.103432  5410 tablet_replica.cc:333] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.104156  5410 raft_consensus.cc:2243] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.105238  5410 raft_consensus.cc:2272] T 56ba1578b97b4faa966168899769e067 P 64b95375974b48d18c596a23c9cac75c [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.108104  5410 tablet_replica.cc:333] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.108798  5410 raft_consensus.cc:2243] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:23.109395  5410 raft_consensus.cc:2272] T 1c3ba394f290437b90045c4a98fb75ca P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.112090  5410 tablet_replica.cc:333] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.112754  5410 raft_consensus.cc:2243] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.113783  5410 raft_consensus.cc:2272] T 627436404a154d80a882f4900dff2948 P 64b95375974b48d18c596a23c9cac75c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.116422  5410 tablet_replica.cc:333] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c: stopping tablet replica
I20260709 13:20:23.117053  5410 raft_consensus.cc:2243] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:23.118093  5410 raft_consensus.cc:2272] T c9bbf06685e047fdaa1b6bc881dc7a93 P 64b95375974b48d18c596a23c9cac75c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:23.577013  5410 tablet_server.cc:196] TabletServer@127.5.72.132:0 shutdown complete.
I20260709 13:20:23.655193  5410 master.cc:562] Master@127.5.72.190:46511 shutting down...
W20260709 13:20:27.092830  6451 debug-util.cc:398] Leaking SignalData structure 0x7b0800177120 after lost signal to thread 5413
W20260709 13:20:27.093410  6451 debug-util.cc:398] Leaking SignalData structure 0x7b08003fa060 after lost signal to thread 6454
W20260709 13:20:44.615417  6451 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7bc0 after lost signal to thread 5413
W20260709 13:20:44.615942  6451 debug-util.cc:398] Leaking SignalData structure 0x7b080008b920 after lost signal to thread 6454
W20260709 13:21:22.391682  6407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111) [suppressed 489 similar messages]
W20260709 13:21:22.395464  6407 catalog_manager.cc:4755] TS 3ceee54e49d041be9f7f48a2052709c5 (127.5.72.130:37977): ChangeConfig:REMOVE_PEER RPC failed for tablet b75fb632ebbc415092b08b38d0835144: Network error: Client connection negotiation failed: client connection to 127.5.72.130:37977: connect: Connection refused (error 111)
I20260709 13:21:25.249220  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:25.251206  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:25.251874  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 10caaf50f65c441b844e3e6055117d18: stopping tablet replica
I20260709 13:21:25.462138  5410 master.cc:584] Master@127.5.72.190:46511 shutdown complete.
I20260709 13:21:25.516568  5410 test_util.cc:182] -----------------------------------------------
I20260709 13:21:25.516817  5410 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602945696884-5410-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (316385 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:21:25.520358  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:43355
I20260709 13:21:25.521505  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:25.527397 15168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:25.527582 15169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:25.529080 15171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:25.530332  5410 server_base.cc:1061] running on GCE node
I20260709 13:21:25.533438  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:25.533631  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:25.533828  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603285533793 us; error 0 us; skew 500 ppm
I20260709 13:21:25.534392  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:25.536883  5410 webserver.cc:533] Webserver started at http://127.5.72.190:32893/ using document root <none> and password file <none>
I20260709 13:21:25.537389  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:25.537577  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:25.537876  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:25.539247  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "63d400ef7f0049b8a99c2b99b44294f8"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.544127  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:21:25.547565 15176 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.548321  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:21:25.548622  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "63d400ef7f0049b8a99c2b99b44294f8"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.548870  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:25.561051  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:25.562264  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:25.602732  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:43355
I20260709 13:21:25.602852 15227 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:43355 every 8 connection(s)
I20260709 13:21:25.606871 15228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:25.617869 15228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8: Bootstrap starting.
I20260709 13:21:25.622819 15228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:25.627406 15228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8: No bootstrap required, opened a new log
I20260709 13:21:25.629834 15228 raft_consensus.cc:359] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d400ef7f0049b8a99c2b99b44294f8" member_type: VOTER }
I20260709 13:21:25.630352 15228 raft_consensus.cc:385] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:25.630553 15228 raft_consensus.cc:740] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d400ef7f0049b8a99c2b99b44294f8, State: Initialized, Role: FOLLOWER
I20260709 13:21:25.631220 15228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d400ef7f0049b8a99c2b99b44294f8" member_type: VOTER }
I20260709 13:21:25.631685 15228 raft_consensus.cc:399] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:21:25.631884 15228 raft_consensus.cc:493] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:21:25.632107 15228 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:25.636685 15228 raft_consensus.cc:515] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d400ef7f0049b8a99c2b99b44294f8" member_type: VOTER }
I20260709 13:21:25.637184 15228 leader_election.cc:304] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63d400ef7f0049b8a99c2b99b44294f8; no voters: 
I20260709 13:21:25.638360 15228 leader_election.cc:290] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:21:25.638825 15231 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:25.640192 15231 raft_consensus.cc:697] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 1 LEADER]: Becoming Leader. State: Replica: 63d400ef7f0049b8a99c2b99b44294f8, State: Running, Role: LEADER
I20260709 13:21:25.640938 15231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d400ef7f0049b8a99c2b99b44294f8" member_type: VOTER }
I20260709 13:21:25.641510 15228 sys_catalog.cc:565] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:21:25.652029 15232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63d400ef7f0049b8a99c2b99b44294f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d400ef7f0049b8a99c2b99b44294f8" member_type: VOTER } }
I20260709 13:21:25.652483 15233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63d400ef7f0049b8a99c2b99b44294f8. Latest consensus state: current_term: 1 leader_uuid: "63d400ef7f0049b8a99c2b99b44294f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d400ef7f0049b8a99c2b99b44294f8" member_type: VOTER } }
I20260709 13:21:25.654289 15232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:25.654435 15233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:25.658028 15241 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:21:25.668560 15241 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:21:25.670699  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:21:25.682754 15241 catalog_manager.cc:1383] Generated new cluster ID: d580230c59544d919207b92a8befa057
I20260709 13:21:25.683143 15241 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:21:25.705049 15241 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:21:25.706413 15241 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:21:25.722098 15241 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8: Generated new TSK 0
I20260709 13:21:25.722757 15241 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:21:25.739188  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:25.745639 15249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:25.746803 15250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:25.748953 15252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:25.749181  5410 server_base.cc:1061] running on GCE node
I20260709 13:21:25.752113  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:25.752331  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:25.752509  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603285752489 us; error 0 us; skew 500 ppm
I20260709 13:21:25.753098  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:25.755559  5410 webserver.cc:533] Webserver started at http://127.5.72.129:46637/ using document root <none> and password file <none>
I20260709 13:21:25.756078  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:25.756268  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:25.756553  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:25.757833  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "88eead6a17f54dd88c723744e0aa5b50"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.762427  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:25.765733 15257 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.766492  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:21:25.766867  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "88eead6a17f54dd88c723744e0aa5b50"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.767187  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:25.777734  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:25.778988  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:25.780395  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:25.782716  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:25.782929  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.783197  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:25.783380  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.825884  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:43313
I20260709 13:21:25.826015 15319 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:43313 every 8 connection(s)
I20260709 13:21:25.831328  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:25.839716 15324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:25.841708 15325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:25.842453 15327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:25.844933  5410 server_base.cc:1061] running on GCE node
I20260709 13:21:25.848263 15320 heartbeater.cc:344] Connected to a master server at 127.5.72.190:43355
I20260709 13:21:25.848680 15320 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:25.848697  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:25.849071  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:25.849251  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603285849218 us; error 0 us; skew 500 ppm
I20260709 13:21:25.849653 15320 heartbeater.cc:507] Master 127.5.72.190:43355 requested a full tablet report, sending...
I20260709 13:21:25.849951  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:25.852610 15193 ts_manager.cc:194] Registered new tserver with Master: 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:25.853391  5410 webserver.cc:533] Webserver started at http://127.5.72.130:36085/ using document root <none> and password file <none>
I20260709 13:21:25.854152  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:25.854408  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:25.854801  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:25.855660 15193 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42710
I20260709 13:21:25.856606  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.861486  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:21:25.865046 15332 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.866015  5410 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:21:25.866302  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.866587  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:25.883373  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:25.884728  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:25.886279  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:25.888850  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:25.889125  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.889402  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:25.889608  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.934302  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:46479
I20260709 13:21:25.934396 15394 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:46479 every 8 connection(s)
I20260709 13:21:25.939883  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:25.948168 15398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:25.949589 15399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:25.953423 15395 heartbeater.cc:344] Connected to a master server at 127.5.72.190:43355
W20260709 13:21:25.953522 15401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:25.954072  5410 server_base.cc:1061] running on GCE node
I20260709 13:21:25.954068 15395 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:25.955164  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:21:25.955267 15395 heartbeater.cc:507] Master 127.5.72.190:43355 requested a full tablet report, sending...
W20260709 13:21:25.955408  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:25.955765  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603285955738 us; error 0 us; skew 500 ppm
I20260709 13:21:25.956437  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:25.957531 15193 ts_manager.cc:194] Registered new tserver with Master: ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:25.959146 15193 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42716
I20260709 13:21:25.959456  5410 webserver.cc:533] Webserver started at http://127.5.72.131:33563/ using document root <none> and password file <none>
I20260709 13:21:25.960114  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:25.960357  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:25.960659  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:25.961961  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "d646a2b597824d24b522d9ef5ff5adc3"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.966450  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:21:25.969941 15406 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.970870  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:21:25.971194  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "d646a2b597824d24b522d9ef5ff5adc3"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:25.971536  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:25.994236  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:25.995604  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:25.997107  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:25.999454  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:25.999678  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:25.999917  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:26.000087  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:26.041595  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:35805
I20260709 13:21:26.041687 15468 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:35805 every 8 connection(s)
I20260709 13:21:26.046980  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:26.054824 15472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:26.055822 15473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:26.057886  5410 server_base.cc:1061] running on GCE node
W20260709 13:21:26.058449 15475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:26.059628  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:26.059921  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:26.060119  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603286060101 us; error 0 us; skew 500 ppm
I20260709 13:21:26.060518 15469 heartbeater.cc:344] Connected to a master server at 127.5.72.190:43355
I20260709 13:21:26.060828  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:26.060959 15469 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:26.062088 15469 heartbeater.cc:507] Master 127.5.72.190:43355 requested a full tablet report, sending...
I20260709 13:21:26.063898  5410 webserver.cc:533] Webserver started at http://127.5.72.132:36149/ using document root <none> and password file <none>
I20260709 13:21:26.064342 15193 ts_manager.cc:194] Registered new tserver with Master: d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:26.064438  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:26.064787  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:26.065058  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:26.065855 15193 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42728
I20260709 13:21:26.066219  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-3-root/instance:
uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
format_stamp: "Formatted at 2026-07-09 13:21:26 on dist-test-slave-n5kz"
I20260709 13:21:26.070917  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:21:26.074205 15480 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:26.074921  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:21:26.075224  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-3-root
uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
format_stamp: "Formatted at 2026-07-09 13:21:26 on dist-test-slave-n5kz"
I20260709 13:21:26.075518  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:26.116611  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:26.117918  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:26.119491  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:26.121697  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:26.121937  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:26.122184  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:26.122359  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:26.175808  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.132:41633
I20260709 13:21:26.175920 15542 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.132:41633 every 8 connection(s)
I20260709 13:21:26.195645 15543 heartbeater.cc:344] Connected to a master server at 127.5.72.190:43355
I20260709 13:21:26.196085 15543 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:26.196930 15543 heartbeater.cc:507] Master 127.5.72.190:43355 requested a full tablet report, sending...
I20260709 13:21:26.199079 15193 ts_manager.cc:194] Registered new tserver with Master: 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:26.200139  5410 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.01850655s
I20260709 13:21:26.200904 15193 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42742
I20260709 13:21:26.228963 15193 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42752:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:21:26.478003 15508 tablet_service.cc:1511] Processing CreateTablet for tablet 3942225e71d7482d869c941ce55f4971 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:26.479619 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3942225e71d7482d869c941ce55f4971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.486209 15507 tablet_service.cc:1511] Processing CreateTablet for tablet 8d531517bd614ae3af03be27016eb596 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:26.488278 15507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d531517bd614ae3af03be27016eb596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.489198 15434 tablet_service.cc:1511] Processing CreateTablet for tablet 3942225e71d7482d869c941ce55f4971 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:26.490782 15434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3942225e71d7482d869c941ce55f4971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.492089 15431 tablet_service.cc:1511] Processing CreateTablet for tablet de52504e9e244dc99cc89e67e9281f3d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:26.493546 15431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de52504e9e244dc99cc89e67e9281f3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.496047 15433 tablet_service.cc:1511] Processing CreateTablet for tablet 3288c26b19cb47dfb6035bddf30566df (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:26.497517 15433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3288c26b19cb47dfb6035bddf30566df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.500232 15432 tablet_service.cc:1511] Processing CreateTablet for tablet efc8987a44524529b4fcedae3a7ff32f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:26.501730 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc8987a44524529b4fcedae3a7ff32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.504694 15506 tablet_service.cc:1511] Processing CreateTablet for tablet efc8987a44524529b4fcedae3a7ff32f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:26.506322 15506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc8987a44524529b4fcedae3a7ff32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.511198 15430 tablet_service.cc:1511] Processing CreateTablet for tablet e8d49dec28d44105b80839cb58705961 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:26.512756 15430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d49dec28d44105b80839cb58705961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.517124 15505 tablet_service.cc:1511] Processing CreateTablet for tablet af51ee9eaabb4a29a989844d36c3f423 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:26.518707 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af51ee9eaabb4a29a989844d36c3f423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.529049 15504 tablet_service.cc:1511] Processing CreateTablet for tablet e8d49dec28d44105b80839cb58705961 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:26.531927 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 9cc957b5c5c04495a4afb4b265db88b1 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:26.532436 15282 tablet_service.cc:1511] Processing CreateTablet for tablet de52504e9e244dc99cc89e67e9281f3d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:26.533537 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc957b5c5c04495a4afb4b265db88b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.531927 15281 tablet_service.cc:1511] Processing CreateTablet for tablet af51ee9eaabb4a29a989844d36c3f423 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:26.533145 15280 tablet_service.cc:1511] Processing CreateTablet for tablet e8d49dec28d44105b80839cb58705961 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:26.535389 15283 tablet_service.cc:1511] Processing CreateTablet for tablet efc8987a44524529b4fcedae3a7ff32f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:26.535931 15285 tablet_service.cc:1511] Processing CreateTablet for tablet 3288c26b19cb47dfb6035bddf30566df (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:26.533950 15282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de52504e9e244dc99cc89e67e9281f3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.537308 15283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efc8987a44524529b4fcedae3a7ff32f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.538375 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 8d531517bd614ae3af03be27016eb596 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:26.539820 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d531517bd614ae3af03be27016eb596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.543064 15285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3288c26b19cb47dfb6035bddf30566df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.543900 15280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d49dec28d44105b80839cb58705961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.544303 15504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d49dec28d44105b80839cb58705961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.546978 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af51ee9eaabb4a29a989844d36c3f423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.574549 15503 tablet_service.cc:1511] Processing CreateTablet for tablet 9cc957b5c5c04495a4afb4b265db88b1 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:26.576592 15503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc957b5c5c04495a4afb4b265db88b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.588613 15360 tablet_service.cc:1511] Processing CreateTablet for tablet 3942225e71d7482d869c941ce55f4971 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:26.589828 15359 tablet_service.cc:1511] Processing CreateTablet for tablet 3288c26b19cb47dfb6035bddf30566df (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:26.590425 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3942225e71d7482d869c941ce55f4971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.591549 15359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3288c26b19cb47dfb6035bddf30566df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.604560 15358 tablet_service.cc:1511] Processing CreateTablet for tablet 8d531517bd614ae3af03be27016eb596 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:26.606454 15358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d531517bd614ae3af03be27016eb596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.607331 15564 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:26.618013 15564 tablet_bootstrap.cc:654] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.633667 15282 tablet_service.cc:1511] Processing CreateTablet for tablet c6bde75c2a4c44b0beb23e22159b9fa2 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:26.635130 15282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bde75c2a4c44b0beb23e22159b9fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.635231 15430 tablet_service.cc:1511] Processing CreateTablet for tablet c6bde75c2a4c44b0beb23e22159b9fa2 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:26.636584 15430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bde75c2a4c44b0beb23e22159b9fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.643934 15562 tablet_bootstrap.cc:492] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:26.643713 15279 tablet_service.cc:1511] Processing CreateTablet for tablet 12527dc46ef54e4f944ce04478f71b8d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:26.645303 15279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12527dc46ef54e4f944ce04478f71b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.645406 15357 tablet_service.cc:1511] Processing CreateTablet for tablet de52504e9e244dc99cc89e67e9281f3d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:26.646895 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de52504e9e244dc99cc89e67e9281f3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.647336 15563 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:26.652686 15356 tablet_service.cc:1511] Processing CreateTablet for tablet af51ee9eaabb4a29a989844d36c3f423 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:26.654237 15356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af51ee9eaabb4a29a989844d36c3f423. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:26.656250 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:26.656378 15564 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:26.657236 15564 ts_tablet_manager.cc:1403] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.003s
I20260709 13:21:26.660588 15355 tablet_service.cc:1511] Processing CreateTablet for tablet 9cc957b5c5c04495a4afb4b265db88b1 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:26.660872 15564 raft_consensus.cc:359] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:26.661628 15564 raft_consensus.cc:385] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:26.661945 15564 raft_consensus.cc:740] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:26.662107 15355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc957b5c5c04495a4afb4b265db88b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.669006 15354 tablet_service.cc:1511] Processing CreateTablet for tablet c6bde75c2a4c44b0beb23e22159b9fa2 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:26.670517 15354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bde75c2a4c44b0beb23e22159b9fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.672410 15563 tablet_bootstrap.cc:654] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.676561 15508 tablet_service.cc:1511] Processing CreateTablet for tablet 12527dc46ef54e4f944ce04478f71b8d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:26.677927 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12527dc46ef54e4f944ce04478f71b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.699018 15353 tablet_service.cc:1511] Processing CreateTablet for tablet 12527dc46ef54e4f944ce04478f71b8d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:26.700495 15353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12527dc46ef54e4f944ce04478f71b8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.700927 15433 tablet_service.cc:1511] Processing CreateTablet for tablet 4f14bc0f2237437888f16c60a62840de (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:26.702886 15564 consensus_queue.cc:260] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:26.704947 15433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f14bc0f2237437888f16c60a62840de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.728937 15562 tablet_bootstrap.cc:654] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.749867 15502 tablet_service.cc:1511] Processing CreateTablet for tablet 4f14bc0f2237437888f16c60a62840de (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:26.756536 15502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f14bc0f2237437888f16c60a62840de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.765020 15320 heartbeater.cc:499] Master 127.5.72.190:43355 was elected leader, sending a full tablet report...
I20260709 13:21:26.765614 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 4f14bc0f2237437888f16c60a62840de (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:26.767012 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f14bc0f2237437888f16c60a62840de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.771610 15354 tablet_service.cc:1511] Processing CreateTablet for tablet c8c2f416704b4f56b52001ddbc062bec (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:26.772981 15354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c2f416704b4f56b52001ddbc062bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.779237 15508 tablet_service.cc:1511] Processing CreateTablet for tablet c8c2f416704b4f56b52001ddbc062bec (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:26.780740 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c2f416704b4f56b52001ddbc062bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.787832 15432 tablet_service.cc:1511] Processing CreateTablet for tablet c8c2f416704b4f56b52001ddbc062bec (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:26.789427 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c2f416704b4f56b52001ddbc062bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.803285 15563 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:26.803872 15563 ts_tablet_manager.cc:1403] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.157s	user 0.047s	sys 0.044s
I20260709 13:21:26.806910 15563 raft_consensus.cc:359] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:26.807663 15563 raft_consensus.cc:385] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:26.807953 15563 raft_consensus.cc:740] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:26.808751 15563 consensus_queue.cc:260] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:26.814970 15564 ts_tablet_manager.cc:1434] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.157s	user 0.022s	sys 0.015s
I20260709 13:21:26.816319 15564 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
W20260709 13:21:26.831198 15321 tablet.cc:2406] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:21:26.832858 15568 tablet_bootstrap.cc:492] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:26.835340 15564 tablet_bootstrap.cc:654] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.837900 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 427b294c2f944e7ca234578e92b6126a (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:26.839416 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427b294c2f944e7ca234578e92b6126a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.839409 15353 tablet_service.cc:1511] Processing CreateTablet for tablet 427b294c2f944e7ca234578e92b6126a (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:26.844133 15564 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:26.844480 15353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427b294c2f944e7ca234578e92b6126a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.844759 15564 ts_tablet_manager.cc:1403] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.006s
I20260709 13:21:26.844647 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 427b294c2f944e7ca234578e92b6126a (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:26.846136 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427b294c2f944e7ca234578e92b6126a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.848505 15563 ts_tablet_manager.cc:1434] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.044s	user 0.003s	sys 0.007s
I20260709 13:21:26.848376 15284 tablet_service.cc:1511] Processing CreateTablet for tablet a6e85352a3ac49939b3bdab9c612b76e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:26.849278 15469 heartbeater.cc:499] Master 127.5.72.190:43355 was elected leader, sending a full tablet report...
I20260709 13:21:26.849862 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e85352a3ac49939b3bdab9c612b76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.854655 15562 tablet_bootstrap.cc:492] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:26.855230 15562 ts_tablet_manager.cc:1403] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.212s	user 0.016s	sys 0.000s
I20260709 13:21:26.856853 15563 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:26.848345 15564 raft_consensus.cc:359] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:26.865511 15564 raft_consensus.cc:385] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:26.865885 15564 raft_consensus.cc:740] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:26.866938 15564 consensus_queue.cc:260] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:26.868394 15563 tablet_bootstrap.cc:654] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.869931 15564 ts_tablet_manager.cc:1434] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:21:26.858445 15562 raft_consensus.cc:359] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:26.870756 15562 raft_consensus.cc:385] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:26.871084 15562 raft_consensus.cc:740] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:26.871287 15564 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:26.871860 15562 consensus_queue.cc:260] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:26.873668 15567 raft_consensus.cc:493] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:26.874189 15567 raft_consensus.cc:515] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:26.880661 15543 heartbeater.cc:499] Master 127.5.72.190:43355 was elected leader, sending a full tablet report...
I20260709 13:21:26.885322 15568 tablet_bootstrap.cc:654] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.888370 15567 leader_election.cc:290] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:26.894697 15564 tablet_bootstrap.cc:654] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.896275 15562 ts_tablet_manager.cc:1434] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.004s
I20260709 13:21:26.911588 15562 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:26.918368 15432 tablet_service.cc:1511] Processing CreateTablet for tablet 4e39f7656f2748f994ca243f51380c1c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:26.919988 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e39f7656f2748f994ca243f51380c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.919740 15357 tablet_service.cc:1511] Processing CreateTablet for tablet a6e85352a3ac49939b3bdab9c612b76e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:26.921710 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e85352a3ac49939b3bdab9c612b76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.927345 15508 tablet_service.cc:1511] Processing CreateTablet for tablet a6e85352a3ac49939b3bdab9c612b76e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:26.927562 15505 tablet_service.cc:1511] Processing CreateTablet for tablet 4e39f7656f2748f994ca243f51380c1c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:26.928820 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6e85352a3ac49939b3bdab9c612b76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.929435 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e39f7656f2748f994ca243f51380c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.930989 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 4e39f7656f2748f994ca243f51380c1c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:26.932329 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e39f7656f2748f994ca243f51380c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.945247 15433 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc719509cbf4d3aad5be6412ec80022 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:26.946127 15353 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc719509cbf4d3aad5be6412ec80022 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:26.946707 15433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc719509cbf4d3aad5be6412ec80022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.947463 15353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc719509cbf4d3aad5be6412ec80022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:26.960472 15563 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:26.960505 15564 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:26.965222 15564 ts_tablet_manager.cc:1403] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.094s	user 0.025s	sys 0.017s
I20260709 13:21:26.965373 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:26.966117 15444 raft_consensus.cc:2468] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:26.961525 15395 heartbeater.cc:499] Master 127.5.72.190:43355 was elected leader, sending a full tablet report...
I20260709 13:21:26.962844 15562 tablet_bootstrap.cc:654] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:26.967654 15261 leader_election.cc:304] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:26.963264 15568 tablet_bootstrap.cc:492] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:26.963706 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:26.968588 15567 raft_consensus.cc:2804] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:26.968978 15567 raft_consensus.cc:493] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:26.969321 15567 raft_consensus.cc:3060] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:26.969544 15568 ts_tablet_manager.cc:1403] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.137s	user 0.011s	sys 0.011s
W20260709 13:21:26.971788 15258 leader_election.cc:343] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:26.974014 15502 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc719509cbf4d3aad5be6412ec80022 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:26.975551 15563 ts_tablet_manager.cc:1403] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.119s	user 0.015s	sys 0.017s
I20260709 13:21:26.975622 15502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc719509cbf4d3aad5be6412ec80022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.008850 15568 raft_consensus.cc:359] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:27.009716 15568 raft_consensus.cc:385] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.010038 15568 raft_consensus.cc:740] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.009181 15563 raft_consensus.cc:359] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:27.021878 15562 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:27.022452 15562 ts_tablet_manager.cc:1403] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.113s	user 0.004s	sys 0.009s
I20260709 13:21:27.025635 15562 raft_consensus.cc:359] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.026363 15562 raft_consensus.cc:385] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.018432 15568 consensus_queue.cc:260] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:27.010951 15564 raft_consensus.cc:359] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.030366 15360 tablet_service.cc:1511] Processing CreateTablet for tablet f34db7828d184304ab99b4216ac1798e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:27.016404 15563 raft_consensus.cc:385] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.031234 15563 raft_consensus.cc:740] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.031025 15564 raft_consensus.cc:385] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.032174 15564 raft_consensus.cc:740] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.033187 15564 consensus_queue.cc:260] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.030995 15429 tablet_service.cc:1511] Processing CreateTablet for tablet f34db7828d184304ab99b4216ac1798e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:27.033846 15505 tablet_service.cc:1511] Processing CreateTablet for tablet e04a0fb0fb0342bd935451e239964483 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:27.034806 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f34db7828d184304ab99b4216ac1798e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.035216 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04a0fb0fb0342bd935451e239964483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.031836 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f34db7828d184304ab99b4216ac1798e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.032217 15563 consensus_queue.cc:260] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:27.033325 15567 raft_consensus.cc:515] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.044395 15281 tablet_service.cc:1511] Processing CreateTablet for tablet f34db7828d184304ab99b4216ac1798e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:27.048303 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f34db7828d184304ab99b4216ac1798e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.049156 15568 ts_tablet_manager.cc:1434] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.079s	user 0.004s	sys 0.006s
I20260709 13:21:27.056147 15568 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:27.061153 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:27.060037 15567 leader_election.cc:290] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:27.060871 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:27.061736 15444 raft_consensus.cc:3060] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:27.062945 15258 leader_election.cc:343] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.060102 15563 ts_tablet_manager.cc:1434] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.083s	user 0.004s	sys 0.007s
I20260709 13:21:27.067643 15563 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:27.070120 15444 raft_consensus.cc:2468] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:27.072829 15285 tablet_service.cc:1511] Processing CreateTablet for tablet e04a0fb0fb0342bd935451e239964483 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:27.074226 15285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04a0fb0fb0342bd935451e239964483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.075011 15261 leader_election.cc:304] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:27.075888 15567 raft_consensus.cc:2804] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:27.076436 15567 raft_consensus.cc:697] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:27.078297 15433 tablet_service.cc:1511] Processing CreateTablet for tablet 65aff9e32079465b85d18fff1c73b28a (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:27.079557 15357 tablet_service.cc:1511] Processing CreateTablet for tablet e04a0fb0fb0342bd935451e239964483 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:27.080284 15508 tablet_service.cc:1511] Processing CreateTablet for tablet 65aff9e32079465b85d18fff1c73b28a (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:27.080889 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e04a0fb0fb0342bd935451e239964483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.081617 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65aff9e32079465b85d18fff1c73b28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.084741 15433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65aff9e32079465b85d18fff1c73b28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.084789 15567 consensus_queue.cc:237] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.091467 15564 ts_tablet_manager.cc:1434] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.126s	user 0.002s	sys 0.021s
I20260709 13:21:27.092736 15564 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:27.026678 15562 raft_consensus.cc:740] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.101469 15568 tablet_bootstrap.cc:654] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.101966 15562 consensus_queue.cc:260] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.105170 15354 tablet_service.cc:1511] Processing CreateTablet for tablet 128c6f471a6b4c53bdd2c868f551b542 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:27.106729 15354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 128c6f471a6b4c53bdd2c868f551b542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.108321 15562 ts_tablet_manager.cc:1434] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.085s	user 0.006s	sys 0.000s
I20260709 13:21:27.109705 15562 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:27.111860 15563 tablet_bootstrap.cc:654] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.115648 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 65aff9e32079465b85d18fff1c73b28a (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:27.125294 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65aff9e32079465b85d18fff1c73b28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.126670 15568 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:27.127328 15568 ts_tablet_manager.cc:1403] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.071s	user 0.008s	sys 0.011s
I20260709 13:21:27.129572 15564 tablet_bootstrap.cc:654] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.132252 15568 raft_consensus.cc:359] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.134490 15568 raft_consensus.cc:385] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.135396 15568 raft_consensus.cc:740] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.137909 15562 tablet_bootstrap.cc:654] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.141134 15434 tablet_service.cc:1511] Processing CreateTablet for tablet 128c6f471a6b4c53bdd2c868f551b542 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:27.142428 15434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 128c6f471a6b4c53bdd2c868f551b542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.151674 15360 tablet_service.cc:1511] Processing CreateTablet for tablet f7b5a27333a34cfeb46c592917d72a5f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:27.153129 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b5a27333a34cfeb46c592917d72a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.155011 15280 tablet_service.cc:1511] Processing CreateTablet for tablet f7b5a27333a34cfeb46c592917d72a5f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:27.158674 15280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b5a27333a34cfeb46c592917d72a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.159595 15502 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba45fbc82014336a9bddd17603ed390 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:27.159453 15193 catalog_manager.cc:5697] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:27.160869 15502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba45fbc82014336a9bddd17603ed390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.162858 15432 tablet_service.cc:1511] Processing CreateTablet for tablet f7b5a27333a34cfeb46c592917d72a5f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:27.164278 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7b5a27333a34cfeb46c592917d72a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.151674 15508 tablet_service.cc:1511] Processing CreateTablet for tablet 128c6f471a6b4c53bdd2c868f551b542 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:27.168124 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 128c6f471a6b4c53bdd2c868f551b542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.158429 15568 consensus_queue.cc:260] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.171424 15579 raft_consensus.cc:493] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.171893 15579 raft_consensus.cc:515] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:27.173585 15568 ts_tablet_manager.cc:1434] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260709 13:21:27.175637 15579 leader_election.cc:290] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:27.187153 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba45fbc82014336a9bddd17603ed390 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:27.189150 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba45fbc82014336a9bddd17603ed390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.203632 15563 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:27.204811 15563 ts_tablet_manager.cc:1403] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.137s	user 0.009s	sys 0.005s
I20260709 13:21:27.204932 15568 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:27.212646 15568 tablet_bootstrap.cc:654] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.213182 15563 raft_consensus.cc:359] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.214032 15563 raft_consensus.cc:385] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.214335 15563 raft_consensus.cc:740] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.220687 15433 tablet_service.cc:1511] Processing CreateTablet for tablet 54afc04cc1b742d1b309764bb898bc6b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:27.222124 15433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54afc04cc1b742d1b309764bb898bc6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.235332 15505 tablet_service.cc:1511] Processing CreateTablet for tablet 54afc04cc1b742d1b309764bb898bc6b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:27.237563 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54afc04cc1b742d1b309764bb898bc6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.239180 15285 tablet_service.cc:1511] Processing CreateTablet for tablet 54afc04cc1b742d1b309764bb898bc6b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:27.240593 15285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54afc04cc1b742d1b309764bb898bc6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.246955 15563 consensus_queue.cc:260] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.248591 15570 raft_consensus.cc:493] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.249236 15570 raft_consensus.cc:515] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.251736 15434 tablet_service.cc:1511] Processing CreateTablet for tablet 43fc5b0260a048f4a581d736b3b2191e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:27.253145 15434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fc5b0260a048f4a581d736b3b2191e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.255877 15570 leader_election.cc:290] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.264155 15506 tablet_service.cc:1511] Processing CreateTablet for tablet 43fc5b0260a048f4a581d736b3b2191e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:27.265587 15506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fc5b0260a048f4a581d736b3b2191e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.278478 15564 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:27.280471 15564 ts_tablet_manager.cc:1403] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.188s	user 0.005s	sys 0.018s
I20260709 13:21:27.282882 15357 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba45fbc82014336a9bddd17603ed390 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:27.288146 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba45fbc82014336a9bddd17603ed390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.287714 15354 tablet_service.cc:1511] Processing CreateTablet for tablet 43fc5b0260a048f4a581d736b3b2191e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:27.289423 15354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fc5b0260a048f4a581d736b3b2191e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.293264 15564 raft_consensus.cc:359] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:27.304234 15567 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.307256 15567 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.309913 15567 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.311091 15353 tablet_service.cc:1511] Processing CreateTablet for tablet f133a6db45994178b473a5b1809d367f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:27.312685 15353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f133a6db45994178b473a5b1809d367f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.314114 15562 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:27.314675 15562 ts_tablet_manager.cc:1403] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.205s	user 0.022s	sys 0.025s
I20260709 13:21:27.315778 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.316139 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
W20260709 13:21:27.328146 15258 leader_election.cc:343] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.322532 15564 raft_consensus.cc:385] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.335369 15563 ts_tablet_manager.cc:1434] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.130s	user 0.014s	sys 0.004s
I20260709 13:21:27.334314 15564 raft_consensus.cc:740] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.336452 15563 tablet_bootstrap.cc:492] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:27.337122 15564 consensus_queue.cc:260] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:27.338407 15572 raft_consensus.cc:493] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.343015 15563 tablet_bootstrap.cc:654] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.342952 15572 raft_consensus.cc:515] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:27.328943 15336 leader_election.cc:343] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.345695 15564 ts_tablet_manager.cc:1434] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.003s
I20260709 13:21:27.356200 15572 leader_election.cc:290] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.363382 15567 raft_consensus.cc:493] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.371526 15567 raft_consensus.cc:515] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.373652 15567 leader_election.cc:290] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:27.374377 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.379132 15370 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:27.378230 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:27.378537 15594 raft_consensus.cc:493] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.384869 15444 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:27.384927 15594 raft_consensus.cc:515] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:27.379297 15568 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:27.386409 15568 ts_tablet_manager.cc:1403] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.182s	user 0.011s	sys 0.012s
I20260709 13:21:27.376690 15502 tablet_service.cc:1511] Processing CreateTablet for tablet 55c5fe7b4ba54f94a897b776dcc58565 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:27.383975 15284 tablet_service.cc:1511] Processing CreateTablet for tablet f133a6db45994178b473a5b1809d367f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:27.395305 15258 leader_election.cc:304] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:27.396191 15594 leader_election.cc:290] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:27.390102 15568 raft_consensus.cc:359] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.401793 15567 raft_consensus.cc:2804] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:27.402729 15567 raft_consensus.cc:493] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:27.403362 15567 raft_consensus.cc:3060] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.407270 15564 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:27.411432 15360 tablet_service.cc:1511] Processing CreateTablet for tablet 55c5fe7b4ba54f94a897b776dcc58565 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:27.439020 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:27.440901 15333 leader_election.cc:343] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.441447 15333 leader_election.cc:304] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:27.441951 15508 tablet_service.cc:1511] Processing CreateTablet for tablet 67f48e3fa27c4c6487669b0d45130af7 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:27.442510 15579 raft_consensus.cc:2749] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:27.400604 15568 raft_consensus.cc:385] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.443491 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f48e3fa27c4c6487669b0d45130af7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.445123 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c5fe7b4ba54f94a897b776dcc58565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.449741 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 55c5fe7b4ba54f94a897b776dcc58565 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:27.444902 15568 raft_consensus.cc:740] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.461349 15502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c5fe7b4ba54f94a897b776dcc58565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.457046 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f133a6db45994178b473a5b1809d367f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.465065 15432 tablet_service.cc:1511] Processing CreateTablet for tablet f133a6db45994178b473a5b1809d367f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:27.465554 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55c5fe7b4ba54f94a897b776dcc58565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.467969 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:27.468727 15518 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:27.470044 15258 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:27.471642 15580 raft_consensus.cc:2804] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:27.471488 15355 tablet_service.cc:1511] Processing CreateTablet for tablet f9cf658f5d1340238f97e4d01fa8c07e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:27.472074 15580 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:27.472438 15580 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.472936 15355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9cf658f5d1340238f97e4d01fa8c07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.475212 15568 consensus_queue.cc:260] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.477455 15568 ts_tablet_manager.cc:1434] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.000s
I20260709 13:21:27.466454 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f133a6db45994178b473a5b1809d367f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.463814 15562 raft_consensus.cc:359] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.457659 15564 tablet_bootstrap.cc:654] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.484263 15562 raft_consensus.cc:385] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.493620 15563 tablet_bootstrap.cc:492] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:27.493285 15434 tablet_service.cc:1511] Processing CreateTablet for tablet 67f48e3fa27c4c6487669b0d45130af7 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:27.495654 15429 tablet_service.cc:1511] Processing CreateTablet for tablet f9cf658f5d1340238f97e4d01fa8c07e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:27.495647 15506 tablet_service.cc:1511] Processing CreateTablet for tablet f9cf658f5d1340238f97e4d01fa8c07e (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:27.503218 15506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9cf658f5d1340238f97e4d01fa8c07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.500078 15568 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:27.498752 15562 raft_consensus.cc:740] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.501338 15434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f48e3fa27c4c6487669b0d45130af7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.508908 15580 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.509866 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:27.501500 15285 tablet_service.cc:1511] Processing CreateTablet for tablet 67f48e3fa27c4c6487669b0d45130af7 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:27.511442 15285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f48e3fa27c4c6487669b0d45130af7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.507103 15563 ts_tablet_manager.cc:1403] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.171s	user 0.021s	sys 0.018s
W20260709 13:21:27.511549 15484 leader_election.cc:343] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.514974 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9cf658f5d1340238f97e4d01fa8c07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.517833 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:27.518477 15518 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.518813 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:27.518848 15562 consensus_queue.cc:260] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:27.522271 15258 leader_election.cc:343] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.531092 15562 ts_tablet_manager.cc:1434] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.216s	user 0.006s	sys 0.003s
I20260709 13:21:27.532295 15562 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:27.533950 15518 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:27.521010 15563 raft_consensus.cc:359] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.535141 15563 raft_consensus.cc:385] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.535494 15563 raft_consensus.cc:740] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.535763 15579 raft_consensus.cc:493] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.536299 15579 raft_consensus.cc:515] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.537528 15563 consensus_queue.cc:260] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.540045 15579 leader_election.cc:290] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:27.542869 15258 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:27.544497 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:27.545604 15568 tablet_bootstrap.cc:654] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.546003 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.546689 15370 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
W20260709 13:21:27.549511 15336 leader_election.cc:343] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.551522 15567 raft_consensus.cc:515] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.552955 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.553675 15567 leader_election.cc:290] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:27.554904 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:27.555487 15444 raft_consensus.cc:3060] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.520727 15580 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.559417 15580 raft_consensus.cc:2804] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:27.559906 15580 raft_consensus.cc:697] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:27.560716 15580 consensus_queue.cc:237] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.564926 15563 ts_tablet_manager.cc:1434] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:21:27.566013 15563 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
W20260709 13:21:27.567142 15481 leader_election.cc:343] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.569732 15481 leader_election.cc:304] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:27.548852 15407 leader_election.cc:304] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:27.575814 15572 raft_consensus.cc:2749] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:27.579591 15505 tablet_service.cc:1511] Processing CreateTablet for tablet 0b6805b1cee04dff9be875c8d6d74e7d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:27.580519 15564 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:27.583559 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b6805b1cee04dff9be875c8d6d74e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.583953 15564 ts_tablet_manager.cc:1403] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.185s	user 0.009s	sys 0.008s
I20260709 13:21:27.591329 15564 raft_consensus.cc:359] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.592192 15564 raft_consensus.cc:385] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.592545 15564 raft_consensus.cc:740] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.595553 15357 tablet_service.cc:1511] Processing CreateTablet for tablet c124e9e01448409d9a64892dddf7c610 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:27.607729 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c124e9e01448409d9a64892dddf7c610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.596863 15562 tablet_bootstrap.cc:654] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.597486 15444 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:27.599727 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d49dec28d44105b80839cb58705961" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:27.608806 15360 tablet_service.cc:1511] Processing CreateTablet for tablet 0b6805b1cee04dff9be875c8d6d74e7d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:27.622509 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b6805b1cee04dff9be875c8d6d74e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.623211 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d49dec28d44105b80839cb58705961" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:27.626124 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:27.620332 15594 raft_consensus.cc:2764] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260709 13:21:27.627414 15410 leader_election.cc:343] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:27.624538 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 81763.7us
I20260709 13:21:27.628865 15281 tablet_service.cc:1511] Processing CreateTablet for tablet c124e9e01448409d9a64892dddf7c610 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:27.630143 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:27.629451 15580 raft_consensus.cc:493] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.630844 15370 raft_consensus.cc:3060] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.630940 15580 raft_consensus.cc:515] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:27.634483 15580 leader_election.cc:290] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:27.627431 15295 raft_consensus.cc:2393] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:27.635296 15564 consensus_queue.cc:260] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.636173 15261 leader_election.cc:304] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:27.636596 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
W20260709 13:21:27.635134 15407 leader_election.cc:343] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.637467 15567 raft_consensus.cc:2804] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:27.637737 15564 ts_tablet_manager.cc:1434] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.053s	user 0.001s	sys 0.004s
I20260709 13:21:27.637668 15407 leader_election.cc:304] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:27.638039 15567 raft_consensus.cc:697] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
W20260709 13:21:27.638171 15261 leader_election.cc:343] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.638931 15567 consensus_queue.cc:237] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.647357 15594 raft_consensus.cc:2749] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:27.643532 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c124e9e01448409d9a64892dddf7c610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.649088 15433 tablet_service.cc:1511] Processing CreateTablet for tablet 1c87965874f942deb1c496acbfed4a1f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:27.650413 15433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c87965874f942deb1c496acbfed4a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.650553 15429 tablet_service.cc:1511] Processing CreateTablet for tablet c124e9e01448409d9a64892dddf7c610 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:27.651947 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c124e9e01448409d9a64892dddf7c610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.656605 15563 tablet_bootstrap.cc:654] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.654289 15502 tablet_service.cc:1511] Processing CreateTablet for tablet 1c87965874f942deb1c496acbfed4a1f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:27.657784 15502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c87965874f942deb1c496acbfed4a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.660199 15370 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:27.655455 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 0b6805b1cee04dff9be875c8d6d74e7d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:27.661056 15564 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:27.661296 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:27.666095 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b6805b1cee04dff9be875c8d6d74e7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.668303 15568 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:27.668861 15568 ts_tablet_manager.cc:1403] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.184s	user 0.014s	sys 0.027s
I20260709 13:21:27.672286 15568 raft_consensus.cc:359] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.682911 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:27.684170 15258 consensus_peers.cc:597] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:27.688081 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:27.693595 15355 tablet_service.cc:1511] Processing CreateTablet for tablet f585eec076224b80b0b6d6447d292eeb (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:27.696592 15355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f585eec076224b80b0b6d6447d292eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.697603 15432 tablet_service.cc:1511] Processing CreateTablet for tablet f585eec076224b80b0b6d6447d292eeb (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:27.698967 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f585eec076224b80b0b6d6447d292eeb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:27.704352 15258 leader_election.cc:343] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.704917 15258 leader_election.cc:304] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:27.705771 15567 raft_consensus.cc:2749] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:27.701952 15336 leader_election.cc:343] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.706604 15336 leader_election.cc:304] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:27.707894 15579 raft_consensus.cc:2749] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:27.708315 15191 catalog_manager.cc:5697] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:27.703545 15568 raft_consensus.cc:385] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.716583 15603 consensus_queue.cc:1048] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:21:27.718678 15285 tablet_service.cc:1511] Processing CreateTablet for tablet 1c87965874f942deb1c496acbfed4a1f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:27.720036 15285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c87965874f942deb1c496acbfed4a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.722487 15564 tablet_bootstrap.cc:654] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.723681 15568 raft_consensus.cc:740] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.724476 15508 tablet_service.cc:1511] Processing CreateTablet for tablet f585eec076224b80b0b6d6447d292eeb (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:27.724579 15568 consensus_queue.cc:260] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.725836 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f585eec076224b80b0b6d6447d292eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.726780 15281 tablet_service.cc:1511] Processing CreateTablet for tablet e5546361e1c64e36951aeb942521a72b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:27.727448 15563 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:27.728055 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5546361e1c64e36951aeb942521a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.729534 15568 ts_tablet_manager.cc:1434] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.001s
I20260709 13:21:27.710551 15191 catalog_manager.cc:5697] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:27.727975 15563 ts_tablet_manager.cc:1403] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.162s	user 0.018s	sys 0.017s
I20260709 13:21:27.739434 15568 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:27.743592 15357 tablet_service.cc:1511] Processing CreateTablet for tablet e5546361e1c64e36951aeb942521a72b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:27.745450 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5546361e1c64e36951aeb942521a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.750103 15433 tablet_service.cc:1511] Processing CreateTablet for tablet e5546361e1c64e36951aeb942521a72b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:27.751025 15505 tablet_service.cc:1511] Processing CreateTablet for tablet ffef2ecb155c41daba889a3314b66527 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:27.752422 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffef2ecb155c41daba889a3314b66527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.760916 15563 raft_consensus.cc:359] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.761732 15563 raft_consensus.cc:385] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.762032 15563 raft_consensus.cc:740] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.763310 15433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5546361e1c64e36951aeb942521a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.770246 15284 tablet_service.cc:1511] Processing CreateTablet for tablet ffef2ecb155c41daba889a3314b66527 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:27.771662 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffef2ecb155c41daba889a3314b66527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.774928 15562 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:27.775552 15562 ts_tablet_manager.cc:1403] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.243s	user 0.011s	sys 0.015s
I20260709 13:21:27.777422 15594 raft_consensus.cc:493] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.778172 15594 raft_consensus.cc:515] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.783658 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.784416 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:27.785210 15407 leader_election.cc:343] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.785408 15518 raft_consensus.cc:2468] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:27.767850 15563 consensus_queue.cc:260] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:27.786757 15594 leader_election.cc:290] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.787660 15407 leader_election.cc:304] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:27.788772 15355 tablet_service.cc:1511] Processing CreateTablet for tablet ffef2ecb155c41daba889a3314b66527 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:27.790166 15355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffef2ecb155c41daba889a3314b66527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.790524 15562 raft_consensus.cc:359] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:27.791304 15562 raft_consensus.cc:385] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:27.791652 15562 raft_consensus.cc:740] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:27.791054 15594 raft_consensus.cc:2804] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:27.793362 15594 raft_consensus.cc:493] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:27.793912 15594 raft_consensus.cc:3060] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.798785 15568 tablet_bootstrap.cc:654] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.792536 15562 consensus_queue.cc:260] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:27.807623 15562 ts_tablet_manager.cc:1434] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 13:21:27.808712 15562 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:27.810257 15572 raft_consensus.cc:493] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.810994 15572 raft_consensus.cc:515] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.807276 15432 tablet_service.cc:1511] Processing CreateTablet for tablet d4539a0cb3c54de2862153a2f08eba04 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:27.815444 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4539a0cb3c54de2862153a2f08eba04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.819614 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.820935 15280 tablet_service.cc:1511] Processing CreateTablet for tablet d4539a0cb3c54de2862153a2f08eba04 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:27.822284 15280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4539a0cb3c54de2862153a2f08eba04. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:27.823724 15481 leader_election.cc:343] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.821130 15572 leader_election.cc:290] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.837409 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:27.838339 15353 tablet_service.cc:1511] Processing CreateTablet for tablet d81a8d43d2a34dc38f37e6c6c03c29c3 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:27.839711 15353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d81a8d43d2a34dc38f37e6c6c03c29c3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:27.839565 15484 leader_election.cc:343] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.840335 15484 leader_election.cc:304] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:27.841634 15429 tablet_service.cc:1511] Processing CreateTablet for tablet d81a8d43d2a34dc38f37e6c6c03c29c3 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:27.843179 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d81a8d43d2a34dc38f37e6c6c03c29c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.844046 15508 tablet_service.cc:1511] Processing CreateTablet for tablet d4539a0cb3c54de2862153a2f08eba04 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:27.841899 15572 raft_consensus.cc:2749] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:27.858820 15508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4539a0cb3c54de2862153a2f08eba04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.868690 15506 tablet_service.cc:1511] Processing CreateTablet for tablet d81a8d43d2a34dc38f37e6c6c03c29c3 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:27.896973 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 2588887022d8411bba3692826b2fe934 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:27.898294 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2588887022d8411bba3692826b2fe934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.898725 15506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d81a8d43d2a34dc38f37e6c6c03c29c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.899359 15605 raft_consensus.cc:493] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.899899 15605 raft_consensus.cc:515] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.894969 15354 tablet_service.cc:1511] Processing CreateTablet for tablet 2588887022d8411bba3692826b2fe934 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:27.909685 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:27.909781 15605 leader_election.cc:290] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.911505 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.912284 15370 raft_consensus.cc:2468] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:27.913652 15258 leader_election.cc:304] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
W20260709 13:21:27.914633 15258 leader_election.cc:343] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.916350 15568 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:27.916467 15605 raft_consensus.cc:2804] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:27.916919 15605 raft_consensus.cc:493] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:27.917009 15568 ts_tablet_manager.cc:1403] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.178s	user 0.017s	sys 0.005s
I20260709 13:21:27.910130 15354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2588887022d8411bba3692826b2fe934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.908298 15562 tablet_bootstrap.cc:654] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.917310 15605 raft_consensus.cc:3060] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.925868 15434 tablet_service.cc:1511] Processing CreateTablet for tablet 2588887022d8411bba3692826b2fe934 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:27.927451 15434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2588887022d8411bba3692826b2fe934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.928351 15564 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:27.928928 15564 ts_tablet_manager.cc:1403] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.283s	user 0.039s	sys 0.078s
I20260709 13:21:27.931324 15507 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed4922079864d94a4a771fb9466d63c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:27.931721 15563 ts_tablet_manager.cc:1434] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.195s	user 0.013s	sys 0.018s
I20260709 13:21:27.937095 15280 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed4922079864d94a4a771fb9466d63c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:27.934470 15507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed4922079864d94a4a771fb9466d63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.938674 15280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed4922079864d94a4a771fb9466d63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.938356 15430 tablet_service.cc:1511] Processing CreateTablet for tablet 677797bb8c284c9d8a85ce81e8052fed (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:27.939694 15430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677797bb8c284c9d8a85ce81e8052fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.942507 15285 tablet_service.cc:1511] Processing CreateTablet for tablet 677797bb8c284c9d8a85ce81e8052fed (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:27.943284 15504 tablet_service.cc:1511] Processing CreateTablet for tablet 677797bb8c284c9d8a85ce81e8052fed (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:27.943894 15285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677797bb8c284c9d8a85ce81e8052fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.944762 15504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677797bb8c284c9d8a85ce81e8052fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.943799 15355 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed4922079864d94a4a771fb9466d63c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:27.947335 15355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed4922079864d94a4a771fb9466d63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.948777 15594 raft_consensus.cc:515] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:27.950695 15594 leader_election.cc:290] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.951812 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:27.952425 15518 raft_consensus.cc:3060] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:27.952972 15563 tablet_bootstrap.cc:492] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:27.956673 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:27.958978 15594 raft_consensus.cc:493] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.959380 15360 tablet_service.cc:1511] Processing CreateTablet for tablet 100da53293964979b47f6fe528856727 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:27.959512 15594 raft_consensus.cc:515] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:27.960377 15407 leader_election.cc:343] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.960731 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100da53293964979b47f6fe528856727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.962322 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:27.963850 15410 leader_election.cc:343] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:27.962190 15284 tablet_service.cc:1511] Processing CreateTablet for tablet f542e50c9876452b88a3f17ea981a62c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:27.963725 15431 tablet_service.cc:1511] Processing CreateTablet for tablet 100da53293964979b47f6fe528856727 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:27.965749 15603 consensus_queue.cc:1048] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:27.966652 15431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100da53293964979b47f6fe528856727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.967170 15594 leader_election.cc:290] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:27.966653 15503 tablet_service.cc:1511] Processing CreateTablet for tablet 100da53293964979b47f6fe528856727 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:27.970235 15503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 100da53293964979b47f6fe528856727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.965353 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f542e50c9876452b88a3f17ea981a62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:27.971709 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:27.972417 15370 raft_consensus.cc:2468] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:27.973954 15563 tablet_bootstrap.cc:654] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:27.975133 15358 tablet_service.cc:1511] Processing CreateTablet for tablet f542e50c9876452b88a3f17ea981a62c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:27.976426 15358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f542e50c9876452b88a3f17ea981a62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.007396 15428 tablet_service.cc:1511] Processing CreateTablet for tablet f542e50c9876452b88a3f17ea981a62c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:28.010437 15407 leader_election.cc:304] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:28.011412 15568 raft_consensus.cc:359] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.012428 15568 raft_consensus.cc:385] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.012799 15568 raft_consensus.cc:740] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.024358 15562 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:28.025031 15562 ts_tablet_manager.cc:1403] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.217s	user 0.015s	sys 0.005s
I20260709 13:21:28.025161 15613 raft_consensus.cc:2804] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:28.025662 15613 raft_consensus.cc:493] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.026014 15613 raft_consensus.cc:3060] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.013811 15568 consensus_queue.cc:260] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.030846 15428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f542e50c9876452b88a3f17ea981a62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.037547 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 7bad35d090e945218853dd63c340674b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:28.040124 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bad35d090e945218853dd63c340674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.062260 15562 raft_consensus.cc:359] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.063153 15562 raft_consensus.cc:385] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.063442 15562 raft_consensus.cc:740] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.064236 15562 consensus_queue.cc:260] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.065769 15518 raft_consensus.cc:2468] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:28.066946 15562 ts_tablet_manager.cc:1434] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 13:21:28.068243 15562 tablet_bootstrap.cc:492] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:28.071329 15572 raft_consensus.cc:493] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.071841 15572 raft_consensus.cc:515] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.074000 15572 leader_election.cc:290] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:28.074457 15563 tablet_bootstrap.cc:492] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:28.074982 15563 ts_tablet_manager.cc:1403] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.122s	user 0.012s	sys 0.039s
I20260709 13:21:28.076092 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:28.076781 15295 raft_consensus.cc:2468] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:28.078220 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:28.078547 15484 leader_election.cc:304] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:28.079577 15572 raft_consensus.cc:2804] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:28.079957 15572 raft_consensus.cc:493] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.080272 15572 raft_consensus.cc:3060] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:28.082111 15484 leader_election.cc:343] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.089730 15572 raft_consensus.cc:515] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.092751 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:28.078053 15563 raft_consensus.cc:359] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.093354 15295 raft_consensus.cc:3060] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.093638 15563 raft_consensus.cc:385] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.093983 15563 raft_consensus.cc:740] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.094928 15563 consensus_queue.cc:260] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.098251 15563 ts_tablet_manager.cc:1434] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:21:28.100881 15295 raft_consensus.cc:2468] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:28.102324 15484 leader_election.cc:304] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:28.052261 15603 consensus_queue.cc:1048] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.058s
I20260709 13:21:28.119800 15568 ts_tablet_manager.cc:1434] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.202s	user 0.008s	sys 0.003s
I20260709 13:21:28.120375 15407 leader_election.cc:304] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:28.113687 15564 raft_consensus.cc:359] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.121357 15564 raft_consensus.cc:385] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.122036 15564 raft_consensus.cc:740] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.125152 15564 consensus_queue.cc:260] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.127110 15594 raft_consensus.cc:2804] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:28.127642 15568 tablet_bootstrap.cc:492] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:28.129205 15564 ts_tablet_manager.cc:1434] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.200s	user 0.007s	sys 0.003s
I20260709 13:21:28.129772 15503 tablet_service.cc:1511] Processing CreateTablet for tablet 7bad35d090e945218853dd63c340674b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:28.133677 15564 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:28.137701 15605 raft_consensus.cc:515] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.156940 15563 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:28.167044 15503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bad35d090e945218853dd63c340674b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.179093 15360 tablet_service.cc:1511] Processing CreateTablet for tablet 7bad35d090e945218853dd63c340674b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:28.180334 15572 leader_election.cc:290] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
W20260709 13:21:28.182276 15258 consensus_peers.cc:597] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:28.184216 15428 tablet_service.cc:1511] Processing CreateTablet for tablet a49a8fabd3bc4946993584f83f34f587 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:28.185333 15567 consensus_queue.cc:1048] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:28.168681 15613 raft_consensus.cc:515] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:28.194582 15613 leader_election.cc:290] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:28.197268 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:28.198998 15281 tablet_service.cc:1511] Processing CreateTablet for tablet a49a8fabd3bc4946993584f83f34f587 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:28.200204 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a49a8fabd3bc4946993584f83f34f587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.201023 15505 tablet_service.cc:1511] Processing CreateTablet for tablet e02c064babd84e109140e2fa50a9ad94 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:28.201812 15353 tablet_service.cc:1511] Processing CreateTablet for tablet e02c064babd84e109140e2fa50a9ad94 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:28.202263 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e02c064babd84e109140e2fa50a9ad94. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:28.204668 15410 leader_election.cc:343] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.191149 15428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a49a8fabd3bc4946993584f83f34f587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.180958 15572 raft_consensus.cc:2804] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:28.220464 15568 tablet_bootstrap.cc:654] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:28.176656 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 74435.5us
I20260709 13:21:28.224025 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:28.224527 15605 leader_election.cc:290] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:28.226035 15572 raft_consensus.cc:697] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:28.227017 15572 consensus_queue.cc:237] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
W20260709 13:21:28.230546 15484 leader_election.cc:343] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.231864 15562 tablet_bootstrap.cc:654] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.238137 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:28.239822 15506 tablet_service.cc:1511] Processing CreateTablet for tablet a49a8fabd3bc4946993584f83f34f587 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:28.240602 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:28.241757 15369 raft_consensus.cc:3060] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.245178 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:28.246686 15370 raft_consensus.cc:3060] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.245857 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bad35d090e945218853dd63c340674b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:28.241801 15258 leader_election.cc:343] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.215444 15613 raft_consensus.cc:493] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.255975 15613 raft_consensus.cc:515] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.263041 15369 raft_consensus.cc:2468] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:28.258383 15613 leader_election.cc:290] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:28.267334 15506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a49a8fabd3bc4946993584f83f34f587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.267724 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 260309c1df6b490ea4502dd13e14f692 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:28.264387 15594 raft_consensus.cc:697] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:28.271399 15353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e02c064babd84e109140e2fa50a9ad94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.273079 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260309c1df6b490ea4502dd13e14f692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.267488 15563 tablet_bootstrap.cc:654] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.268388 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:28.258605 15429 tablet_service.cc:1511] Processing CreateTablet for tablet e02c064babd84e109140e2fa50a9ad94 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:28.271503 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:28.283623 15258 leader_election.cc:304] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:28.271601 15594 consensus_queue.cc:237] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.286460 15367 raft_consensus.cc:2468] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:28.288128 15620 raft_consensus.cc:493] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.288757 15620 raft_consensus.cc:515] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.291865 15620 leader_election.cc:290] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:28.292932 15567 raft_consensus.cc:2804] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:28.293560 15567 raft_consensus.cc:697] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:28.294498 15567 consensus_queue.cc:237] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:28.294973 15407 leader_election.cc:343] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.295114 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d49dec28d44105b80839cb58705961" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:28.296005 15295 raft_consensus.cc:2468] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:28.297267 15484 leader_election.cc:304] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:28.298135 15620 raft_consensus.cc:2804] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:28.298494 15620 raft_consensus.cc:493] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.298875 15620 raft_consensus.cc:3060] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.298949 15407 leader_election.cc:304] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:28.318816 15564 tablet_bootstrap.cc:654] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.326068 15568 tablet_bootstrap.cc:492] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:28.326850 15568 ts_tablet_manager.cc:1403] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.199s	user 0.010s	sys 0.011s
I20260709 13:21:28.304291 15613 raft_consensus.cc:2804] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:28.333878 15613 raft_consensus.cc:493] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.337565 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e02c064babd84e109140e2fa50a9ad94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.346208 15504 tablet_service.cc:1511] Processing CreateTablet for tablet 318aa120381647db94d137ca2de5d87b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:28.346637 15568 raft_consensus.cc:359] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.347402 15568 raft_consensus.cc:385] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.347504 15504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 318aa120381647db94d137ca2de5d87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.347719 15568 raft_consensus.cc:740] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.348677 15568 consensus_queue.cc:260] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.351333 15568 ts_tablet_manager.cc:1434] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.002s
I20260709 13:21:28.352380 15568 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:28.341322 15370 raft_consensus.cc:2468] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:28.362936 15564 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:28.363511 15564 ts_tablet_manager.cc:1403] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.230s	user 0.013s	sys 0.011s
I20260709 13:21:28.362960 15430 tablet_service.cc:1511] Processing CreateTablet for tablet 260309c1df6b490ea4502dd13e14f692 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:28.362753 15407 leader_election.cc:304] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:28.347636 15357 tablet_service.cc:1511] Processing CreateTablet for tablet 260309c1df6b490ea4502dd13e14f692 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:28.370302 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 318aa120381647db94d137ca2de5d87b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:28.380323 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 318aa120381647db94d137ca2de5d87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.370750 15594 raft_consensus.cc:2804] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:28.385185 15594 raft_consensus.cc:697] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:28.368129 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d49dec28d44105b80839cb58705961" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:28.387444 15443 raft_consensus.cc:2468] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:28.386034 15594 consensus_queue.cc:237] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:28.342590 15613 raft_consensus.cc:3060] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.361372 15568 tablet_bootstrap.cc:654] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.367378 15430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260309c1df6b490ea4502dd13e14f692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.392592 15579 raft_consensus.cc:493] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.399829 15579 raft_consensus.cc:515] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.403704 15579 leader_election.cc:290] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:28.404125 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:28.405663 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3942225e71d7482d869c941ce55f4971" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:28.406512 15518 raft_consensus.cc:2393] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:28.371232 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260309c1df6b490ea4502dd13e14f692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.423233 15333 leader_election.cc:304] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:28.425051 15191 catalog_manager.cc:5697] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:28.426047 15620 raft_consensus.cc:515] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.366593 15564 raft_consensus.cc:359] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:28.415565 15579 raft_consensus.cc:493] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.443131 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d49dec28d44105b80839cb58705961" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:28.443351 15504 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9fb81bccaf408cba956547d37caf3f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:28.443837 15442 raft_consensus.cc:3060] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.444691 15504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9fb81bccaf408cba956547d37caf3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.447918 15431 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9fb81bccaf408cba956547d37caf3f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:28.449267 15431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9fb81bccaf408cba956547d37caf3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.450776 15192 catalog_manager.cc:5697] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:28.439932 15564 raft_consensus.cc:385] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.463222 15564 raft_consensus.cc:740] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.466887 15564 consensus_queue.cc:260] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:28.467871 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9fb81bccaf408cba956547d37caf3f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:28.469157 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9fb81bccaf408cba956547d37caf3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.469749 15564 ts_tablet_manager.cc:1434] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.106s	user 0.005s	sys 0.002s
I20260709 13:21:28.453856 15579 raft_consensus.cc:515] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.480598 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8d49dec28d44105b80839cb58705961" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:28.481804 15579 leader_election.cc:290] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:28.485123 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:28.481261 15295 raft_consensus.cc:3060] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.473287 15620 leader_election.cc:290] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:28.487497 15613 raft_consensus.cc:515] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.490996 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:28.491587 15370 raft_consensus.cc:3060] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.492619 15295 raft_consensus.cc:2468] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:28.493956 15484 leader_election.cc:304] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:28.494896 15620 raft_consensus.cc:2804] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:28.495766 15620 raft_consensus.cc:697] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:28.496048 15353 tablet_service.cc:1511] Processing CreateTablet for tablet 318aa120381647db94d137ca2de5d87b (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:28.496742 15620 consensus_queue.cc:237] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.497884 15353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 318aa120381647db94d137ca2de5d87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.482524 15579 raft_consensus.cc:3060] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.522969 15564 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:28.523797 15370 raft_consensus.cc:2468] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
W20260709 13:21:28.509042 15333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 85702.1us
I20260709 13:21:28.535349 15613 leader_election.cc:290] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:28.544490 15407 leader_election.cc:304] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:28.554692 15563 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:28.555202 15563 ts_tablet_manager.cc:1403] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.398s	user 0.009s	sys 0.009s
I20260709 13:21:28.555095 15429 tablet_service.cc:1511] Processing CreateTablet for tablet cd5f83d74ae74d00b588a9fd88c8b0b0 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:28.556449 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd5f83d74ae74d00b588a9fd88c8b0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.544698 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:28.546087 15503 tablet_service.cc:1511] Processing CreateTablet for tablet cd5f83d74ae74d00b588a9fd88c8b0b0 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:28.560206 15503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd5f83d74ae74d00b588a9fd88c8b0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.561710 15568 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:28.562255 15568 ts_tablet_manager.cc:1403] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.210s	user 0.009s	sys 0.024s
I20260709 13:21:28.562645 15594 raft_consensus.cc:2804] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:28.563385 15594 raft_consensus.cc:697] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:28.564204 15594 consensus_queue.cc:237] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.566150 15442 raft_consensus.cc:2468] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:28.551147 15562 tablet_bootstrap.cc:492] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
W20260709 13:21:28.572088 15336 leader_election.cc:343] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.572177 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:28.575224 15407 leader_election.cc:343] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.575697 15517 raft_consensus.cc:2468] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:28.578209 15358 tablet_service.cc:1511] Processing CreateTablet for tablet 3593559594274cc7aef7c38772401110 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:28.580051 15358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3593559594274cc7aef7c38772401110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.583478 15281 tablet_service.cc:1511] Processing CreateTablet for tablet d95073a3b15444a4880921aaad3f82a1 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:28.584693 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d95073a3b15444a4880921aaad3f82a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.574810 15562 ts_tablet_manager.cc:1403] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.507s	user 0.050s	sys 0.033s
I20260709 13:21:28.588163 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 3593559594274cc7aef7c38772401110 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:28.589303 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3593559594274cc7aef7c38772401110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.590950 15333 leader_election.cc:304] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:28.590811 15562 raft_consensus.cc:359] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.592047 15635 raft_consensus.cc:2804] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:28.592624 15635 raft_consensus.cc:493] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.593053 15635 raft_consensus.cc:3060] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.565138 15568 raft_consensus.cc:359] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.574381 15355 tablet_service.cc:1511] Processing CreateTablet for tablet cd5f83d74ae74d00b588a9fd88c8b0b0 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:28.598878 15568 raft_consensus.cc:385] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.599351 15568 raft_consensus.cc:740] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.574731 15564 tablet_bootstrap.cc:654] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.600308 15605 raft_consensus.cc:493] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.600461 15568 consensus_queue.cc:260] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.600818 15605 raft_consensus.cc:515] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:28.548270 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 3593559594274cc7aef7c38772401110 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:28.605363 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3593559594274cc7aef7c38772401110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.599676 15355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd5f83d74ae74d00b588a9fd88c8b0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.612304 15506 tablet_service.cc:1511] Processing CreateTablet for tablet d95073a3b15444a4880921aaad3f82a1 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:28.592015 15562 raft_consensus.cc:385] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.613180 15562 raft_consensus.cc:740] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.613693 15506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d95073a3b15444a4880921aaad3f82a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.613965 15562 consensus_queue.cc:260] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.558171 15563 raft_consensus.cc:359] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.624782 15579 raft_consensus.cc:2749] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:28.625036 15563 raft_consensus.cc:385] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.626315 15563 raft_consensus.cc:740] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.627672 15563 consensus_queue.cc:260] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.637524 15572 raft_consensus.cc:493] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.638273 15572 raft_consensus.cc:515] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.651741 15613 raft_consensus.cc:493] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.652509 15613 raft_consensus.cc:515] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:28.657871 15562 ts_tablet_manager.cc:1434] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.071s	user 0.015s	sys 0.015s
I20260709 13:21:28.659202 15562 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:28.659092 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:28.660025 15294 raft_consensus.cc:2393] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:28.661993 15613 leader_election.cc:290] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:28.663750 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:28.665503 15407 leader_election.cc:304] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:28.666541 15562 tablet_bootstrap.cc:654] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.668879 15635 raft_consensus.cc:515] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.672034 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
W20260709 13:21:28.673375 15336 leader_election.cc:343] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:28.682955 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:28.683137 15605 leader_election.cc:290] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:28.683842 15368 raft_consensus.cc:2393] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:28.688167 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3288c26b19cb47dfb6035bddf30566df" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:28.691972 15357 tablet_service.cc:1511] Processing CreateTablet for tablet d95073a3b15444a4880921aaad3f82a1 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:28.692687 15261 leader_election.cc:304] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:28.693310 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d95073a3b15444a4880921aaad3f82a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.694888 15635 leader_election.cc:290] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:28.696440 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:28.697034 15517 raft_consensus.cc:3060] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.699792 15567 raft_consensus.cc:3060] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.699772 15603 consensus_queue.cc:1048] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:28.706840 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1ddcb87ba840e68a9c62551336aa0c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:28.708132 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1ddcb87ba840e68a9c62551336aa0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.708379 15193 catalog_manager.cc:5697] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:28.679965 15594 raft_consensus.cc:3060] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.736379 15563 ts_tablet_manager.cc:1434] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.181s	user 0.009s	sys 0.000s
I20260709 13:21:28.744855 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:28.745760 15370 raft_consensus.cc:2393] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:28.748086 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1ddcb87ba840e68a9c62551336aa0c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:28.749300 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1ddcb87ba840e68a9c62551336aa0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.749478 15563 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:28.754889 15572 leader_election.cc:290] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:28.725759 15613 consensus_queue.cc:1048] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 13:21:28.758606 15563 tablet_bootstrap.cc:654] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.759474 15517 raft_consensus.cc:2468] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:28.769040 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cc957b5c5c04495a4afb4b265db88b1" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:28.777278 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 500dbd579fd848328ce9984276eb0e04 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:28.778214 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 500dbd579fd848328ce9984276eb0e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.782997 15484 leader_election.cc:304] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:28.790993 15594 raft_consensus.cc:2749] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:28.791604 15572 raft_consensus.cc:3060] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.792915 15333 leader_election.cc:304] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:28.793659 15358 tablet_service.cc:1511] Processing CreateTablet for tablet 500dbd579fd848328ce9984276eb0e04 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:28.795025 15358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 500dbd579fd848328ce9984276eb0e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.796212 15503 tablet_service.cc:1511] Processing CreateTablet for tablet 500dbd579fd848328ce9984276eb0e04 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:28.797567 15503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 500dbd579fd848328ce9984276eb0e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.798740 15567 raft_consensus.cc:2749] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:28.805387 15504 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1ddcb87ba840e68a9c62551336aa0c (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:28.806946 15504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1ddcb87ba840e68a9c62551336aa0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.809930 15620 consensus_queue.cc:1048] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:28.810534 15368 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:28.811410 15635 raft_consensus.cc:2804] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:28.812354 15635 raft_consensus.cc:697] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:28.813377 15635 consensus_queue.cc:237] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.822535 15613 consensus_queue.cc:1048] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:28.822753 15622 consensus_queue.cc:1048] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:28.824116 15564 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:28.824635 15564 ts_tablet_manager.cc:1403] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.349s	user 0.007s	sys 0.027s
I20260709 13:21:28.785888 15562 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
W20260709 13:21:28.827720 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:21:28.834347 15258 consensus_peers.cc:597] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 -> Peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Couldn't send request to peer 1b7f36fa6b484a7b8859acbcd86c4f0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:28.836187 15355 tablet_service.cc:1511] Processing CreateTablet for tablet 979c935f63114ce6b2c31919b88a250f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:28.836509 15564 raft_consensus.cc:359] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.837196 15564 raft_consensus.cc:385] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.837494 15564 raft_consensus.cc:740] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.839648 15564 consensus_queue.cc:260] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:28.847843 15564 ts_tablet_manager.cc:1434] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:21:28.851994 15563 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:28.852717 15563 ts_tablet_manager.cc:1403] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.000s
I20260709 13:21:28.855931 15355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979c935f63114ce6b2c31919b88a250f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.838624 15562 ts_tablet_manager.cc:1403] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.180s	user 0.021s	sys 0.009s
I20260709 13:21:28.863380 15620 consensus_queue.cc:1048] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:28.866269 15281 tablet_service.cc:1511] Processing CreateTablet for tablet 979c935f63114ce6b2c31919b88a250f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:28.867933 15281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979c935f63114ce6b2c31919b88a250f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.867727 15357 tablet_service.cc:1511] Processing CreateTablet for tablet 4d4413c641a248a9a0cf781768951edc (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:28.867830 15506 tablet_service.cc:1511] Processing CreateTablet for tablet 4d4413c641a248a9a0cf781768951edc (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:28.868963 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4413c641a248a9a0cf781768951edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.869190 15506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4413c641a248a9a0cf781768951edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.873572 15505 tablet_service.cc:1511] Processing CreateTablet for tablet b99ac0daff22426fb7dfa484f28c548d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:28.879261 15190 catalog_manager.cc:5697] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:28.880995 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 979c935f63114ce6b2c31919b88a250f (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:28.882141 15358 tablet_service.cc:1511] Processing CreateTablet for tablet 73732c746e334164b459508a1eeda5ec (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:28.883605 15358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73732c746e334164b459508a1eeda5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.883384 15431 tablet_service.cc:1511] Processing CreateTablet for tablet b99ac0daff22426fb7dfa484f28c548d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:28.884390 15280 tablet_service.cc:1511] Processing CreateTablet for tablet b99ac0daff22426fb7dfa484f28c548d (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:28.885274 15284 tablet_service.cc:1511] Processing CreateTablet for tablet 4d4413c641a248a9a0cf781768951edc (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:28.885659 15280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99ac0daff22426fb7dfa484f28c548d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.882197 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979c935f63114ce6b2c31919b88a250f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.886371 15562 raft_consensus.cc:359] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.846745 15572 raft_consensus.cc:2749] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:28.887316 15562 raft_consensus.cc:385] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.879280 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99ac0daff22426fb7dfa484f28c548d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.887710 15562 raft_consensus.cc:740] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.888691 15568 ts_tablet_manager.cc:1434] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.326s	user 0.021s	sys 0.049s
I20260709 13:21:28.889313 15562 consensus_queue.cc:260] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.889791 15568 tablet_bootstrap.cc:492] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:28.886397 15284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4413c641a248a9a0cf781768951edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.900422 15568 tablet_bootstrap.cc:654] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.910501 15503 tablet_service.cc:1511] Processing CreateTablet for tablet 73732c746e334164b459508a1eeda5ec (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:28.939445 15503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73732c746e334164b459508a1eeda5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.939319 15430 tablet_service.cc:1511] Processing CreateTablet for tablet 73732c746e334164b459508a1eeda5ec (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:28.941361 15285 tablet_service.cc:1511] Processing CreateTablet for tablet 06ec5b047f9d4549ac7e3bf2d5f71bb2 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:28.942605 15285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06ec5b047f9d4549ac7e3bf2d5f71bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.937551 15562 ts_tablet_manager.cc:1434] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 13:21:28.938114 15431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99ac0daff22426fb7dfa484f28c548d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.945849 15563 raft_consensus.cc:359] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.949422 15563 raft_consensus.cc:385] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:28.949851 15563 raft_consensus.cc:740] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:28.961175 15360 tablet_service.cc:1511] Processing CreateTablet for tablet 06ec5b047f9d4549ac7e3bf2d5f71bb2 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:28.965440 15562 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:28.974702 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 06ec5b047f9d4549ac7e3bf2d5f71bb2 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:28.957329 15563 consensus_queue.cc:260] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:28.962503 15360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06ec5b047f9d4549ac7e3bf2d5f71bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.985138 15564 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:28.988732 15430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73732c746e334164b459508a1eeda5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.997079 15562 tablet_bootstrap.cc:654] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:28.997191 15507 tablet_service.cc:1511] Processing CreateTablet for tablet 20d52192ec7445f4a801452a645af6ba (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:29.003969 15280 tablet_service.cc:1511] Processing CreateTablet for tablet 20d52192ec7445f4a801452a645af6ba (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:29.013350 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06ec5b047f9d4549ac7e3bf2d5f71bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.998566 15507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d52192ec7445f4a801452a645af6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.052609 15563 ts_tablet_manager.cc:1434] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.200s	user 0.012s	sys 0.001s
I20260709 13:21:29.054033 15563 tablet_bootstrap.cc:492] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.064038 15283 tablet_service.cc:1511] Processing CreateTablet for tablet 69858931805f420db0a026c73d7b0f06 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:29.041858 15579 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:29.033375 15613 consensus_queue.cc:1048] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:21:29.083216 15280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d52192ec7445f4a801452a645af6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.085286 15505 tablet_service.cc:1511] Processing CreateTablet for tablet 69858931805f420db0a026c73d7b0f06 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:29.085119 15357 tablet_service.cc:1511] Processing CreateTablet for tablet 20d52192ec7445f4a801452a645af6ba (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:29.088857 15505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69858931805f420db0a026c73d7b0f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.089301 15357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d52192ec7445f4a801452a645af6ba. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:29.096766 15407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 107610us
I20260709 13:21:29.110769 15283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69858931805f420db0a026c73d7b0f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.113008 15563 tablet_bootstrap.cc:654] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.115547 15429 tablet_service.cc:1511] Processing CreateTablet for tablet 69858931805f420db0a026c73d7b0f06 (DEFAULT_TABLE table=test-workload [id=3a7cce0320cb4dd19347ea14a96f2761]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:29.116773 15429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69858931805f420db0a026c73d7b0f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:29.116475 15662 consensus_queue.cc:1048] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:29.079150 15579 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.136662 15613 consensus_queue.cc:1048] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:29.138749 15579 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:29.138978 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:29.140237 15564 tablet_bootstrap.cc:654] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:29.147102 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 106883us
I20260709 13:21:29.153323 15193 catalog_manager.cc:5697] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:29.156181 15193 catalog_manager.cc:5697] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.168123 15568 tablet_bootstrap.cc:492] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.169817 15568 ts_tablet_manager.cc:1403] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.280s	user 0.034s	sys 0.034s
I20260709 13:21:29.177416 15562 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.177945 15562 ts_tablet_manager.cc:1403] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.213s	user 0.012s	sys 0.003s
I20260709 13:21:29.178105 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:29.179340 15572 consensus_queue.cc:1048] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:29.201870 15333 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
W20260709 13:21:29.168826 15410 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 136803us
I20260709 13:21:29.205942 15572 raft_consensus.cc:493] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.206430 15579 raft_consensus.cc:2749] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:29.218451 15568 raft_consensus.cc:359] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.221356 15562 raft_consensus.cc:359] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.223345 15568 raft_consensus.cc:385] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.223358 15562 raft_consensus.cc:385] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.224081 15568 raft_consensus.cc:740] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.223834 15572 raft_consensus.cc:515] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.225273 15568 consensus_queue.cc:260] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.227124 15572 leader_election.cc:290] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:29.224086 15562 raft_consensus.cc:740] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.230295 15562 consensus_queue.cc:260] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.231070 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:29.231869 15568 ts_tablet_manager.cc:1434] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.001s
I20260709 13:21:29.232632 15562 ts_tablet_manager.cc:1434] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.054s	user 0.003s	sys 0.003s
I20260709 13:21:29.233060 15568 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
W20260709 13:21:29.232944 15484 leader_election.cc:343] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.233592 15562 tablet_bootstrap.cc:492] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.240525 15563 tablet_bootstrap.cc:492] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.241047 15563 ts_tablet_manager.cc:1403] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.187s	user 0.033s	sys 0.018s
I20260709 13:21:29.242122 15664 consensus_queue.cc:1048] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:29.247387 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:29.247795 15563 raft_consensus.cc:359] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.248140 15443 raft_consensus.cc:2468] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:29.248525 15563 raft_consensus.cc:385] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.248826 15563 raft_consensus.cc:740] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.251119 15484 leader_election.cc:304] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:29.251734 15562 tablet_bootstrap.cc:654] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.253970 15666 raft_consensus.cc:2804] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.249660 15563 consensus_queue.cc:260] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.254431 15666 raft_consensus.cc:493] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.254850 15666 raft_consensus.cc:3060] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.256678 15563 ts_tablet_manager.cc:1434] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:21:29.257887 15563 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.259956 15568 tablet_bootstrap.cc:654] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.268878 15564 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:29.269367 15564 ts_tablet_manager.cc:1403] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.285s	user 0.014s	sys 0.010s
I20260709 13:21:29.276177 15517 raft_consensus.cc:3060] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.297609 15563 tablet_bootstrap.cc:654] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.303772 15605 raft_consensus.cc:493] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.310511 15579 consensus_queue.cc:1048] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:29.312716 15564 raft_consensus.cc:359] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.313761 15564 raft_consensus.cc:385] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.314179 15564 raft_consensus.cc:740] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.315172 15564 consensus_queue.cc:260] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.319185 15564 ts_tablet_manager.cc:1434] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:21:29.316322 15666 raft_consensus.cc:515] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.304514 15605 raft_consensus.cc:515] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.323813 15605 leader_election.cc:290] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:29.328763 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:29.329412 15442 raft_consensus.cc:3060] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.330753 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:29.324018 15666 leader_election.cc:290] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:29.334834 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:29.335594 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d531517bd614ae3af03be27016eb596" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:29.335666 15294 raft_consensus.cc:3060] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.336575 15562 tablet_bootstrap.cc:492] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.337074 15562 ts_tablet_manager.cc:1403] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.104s	user 0.016s	sys 0.043s
I20260709 13:21:29.339860 15562 raft_consensus.cc:359] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.340523 15562 raft_consensus.cc:385] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.340785 15562 raft_consensus.cc:740] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.341588 15562 consensus_queue.cc:260] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.344120 15562 ts_tablet_manager.cc:1434] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:21:29.345016 15562 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.351012 15562 tablet_bootstrap.cc:654] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.347033 15258 leader_election.cc:304] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:29.351998 15605 raft_consensus.cc:2749] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.355535 15564 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:29.369923 15193 catalog_manager.cc:5697] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.374940 15563 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.375470 15563 ts_tablet_manager.cc:1403] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.118s	user 0.026s	sys 0.023s
I20260709 13:21:29.378399 15564 tablet_bootstrap.cc:654] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.379004 15568 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.379519 15568 ts_tablet_manager.cc:1403] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.147s	user 0.025s	sys 0.013s
I20260709 13:21:29.382058 15294 raft_consensus.cc:2468] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:29.382604 15442 raft_consensus.cc:2468] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:29.383777 15563 raft_consensus.cc:359] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.384124 15484 leader_election.cc:304] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:29.385070 15662 raft_consensus.cc:2804] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.385156 15563 raft_consensus.cc:385] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.385646 15662 raft_consensus.cc:697] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:29.385704 15563 raft_consensus.cc:740] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.386693 15563 consensus_queue.cc:260] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.388535 15563 ts_tablet_manager.cc:1434] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:21:29.389477 15563 tablet_bootstrap.cc:492] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.389624 15562 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.390184 15562 ts_tablet_manager.cc:1403] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.013s
I20260709 13:21:29.386673 15662 consensus_queue.cc:237] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.389278 15568 raft_consensus.cc:359] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.393550 15568 raft_consensus.cc:385] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.393839 15568 raft_consensus.cc:740] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.394495 15568 consensus_queue.cc:260] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.396625 15568 ts_tablet_manager.cc:1434] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260709 13:21:29.397553 15568 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:29.400044 15294 raft_consensus.cc:3060] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.402837 15562 raft_consensus.cc:359] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.403591 15562 raft_consensus.cc:385] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.403856 15562 raft_consensus.cc:740] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.404671 15562 consensus_queue.cc:260] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.406023 15563 tablet_bootstrap.cc:654] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.418463 15562 ts_tablet_manager.cc:1434] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.028s	user 0.001s	sys 0.005s
I20260709 13:21:29.420810 15568 tablet_bootstrap.cc:654] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.424268 15562 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.432281 15563 tablet_bootstrap.cc:492] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.433068 15563 ts_tablet_manager.cc:1403] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.044s	user 0.020s	sys 0.001s
I20260709 13:21:29.441278 15562 tablet_bootstrap.cc:654] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.447350 15635 consensus_queue.cc:1048] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:29.452914 15563 raft_consensus.cc:359] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.455466 15568 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.455747 15564 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:29.456068 15568 ts_tablet_manager.cc:1403] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.000s
I20260709 13:21:29.456327 15564 ts_tablet_manager.cc:1403] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.101s	user 0.034s	sys 0.034s
I20260709 13:21:29.454170 15563 raft_consensus.cc:385] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.460336 15563 raft_consensus.cc:740] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.460785 15564 raft_consensus.cc:359] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.461062 15563 consensus_queue.cc:260] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.462177 15562 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.462733 15562 ts_tablet_manager.cc:1403] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.003s
I20260709 13:21:29.463141 15563 ts_tablet_manager.cc:1434] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.004s
I20260709 13:21:29.461486 15564 raft_consensus.cc:385] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.463589 15564 raft_consensus.cc:740] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.465026 15564 consensus_queue.cc:260] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.465408 15562 raft_consensus.cc:359] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.466073 15562 raft_consensus.cc:385] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.466351 15562 raft_consensus.cc:740] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.467459 15564 ts_tablet_manager.cc:1434] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:29.468531 15564 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:29.464177 15563 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.463891 15568 raft_consensus.cc:359] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.469319 15568 raft_consensus.cc:385] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.468531 15562 consensus_queue.cc:260] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.469635 15568 raft_consensus.cc:740] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.471500 15562 ts_tablet_manager.cc:1434] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:21:29.472527 15562 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.470371 15568 consensus_queue.cc:260] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.478039 15562 tablet_bootstrap.cc:654] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.478674 15568 ts_tablet_manager.cc:1434] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:21:29.479641 15568 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:29.487304 15605 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.487864 15605 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.489130 15562 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.489319 15564 tablet_bootstrap.cc:654] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.489612 15562 ts_tablet_manager.cc:1403] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.002s
I20260709 13:21:29.489934 15605 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:29.491945 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:29.492350 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:29.492605 15562 raft_consensus.cc:359] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.492996 15517 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:29.493279 15562 raft_consensus.cc:385] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.493578 15562 raft_consensus.cc:740] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
W20260709 13:21:29.493990 15258 leader_election.cc:343] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.494254 15562 consensus_queue.cc:260] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.494917 15258 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:29.496747 15562 ts_tablet_manager.cc:1434] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:29.497434 15563 tablet_bootstrap.cc:654] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.497539 15562 tablet_bootstrap.cc:492] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.498413 15605 raft_consensus.cc:2804] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.498857 15605 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.499214 15605 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.486114 15190 catalog_manager.cc:5697] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:29.502424 15568 tablet_bootstrap.cc:654] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.504297 15562 tablet_bootstrap.cc:654] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.509822 15563 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.510283 15563 ts_tablet_manager.cc:1403] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.046s	user 0.001s	sys 0.009s
I20260709 13:21:29.513235 15563 raft_consensus.cc:359] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.513517 15605 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.514191 15563 raft_consensus.cc:385] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.514631 15563 raft_consensus.cc:740] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.515476 15563 consensus_queue.cc:260] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.519032 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:29.519657 15517 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.520879 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
W20260709 13:21:29.522053 15258 leader_election.cc:343] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.524022 15563 ts_tablet_manager.cc:1434] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.004s
I20260709 13:21:29.524993 15563 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.526911 15605 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:29.533056 15368 raft_consensus.cc:3060] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.535398 15517 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:29.539741 15659 raft_consensus.cc:493] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.540226 15659 raft_consensus.cc:515] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.541877 15563 tablet_bootstrap.cc:654] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.545450 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
W20260709 13:21:29.547003 15481 leader_election.cc:343] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.547717 15564 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:29.542443 15659 leader_election.cc:290] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:29.548393 15564 ts_tablet_manager.cc:1403] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.023s
I20260709 13:21:29.546931 15258 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:29.556056 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:29.559041 15484 leader_election.cc:343] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.559614 15568 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.559499 15484 leader_election.cc:304] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:29.560078 15568 ts_tablet_manager.cc:1403] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.009s
I20260709 13:21:29.561136 15659 raft_consensus.cc:2749] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.555656 15605 raft_consensus.cc:2804] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.565126 15605 raft_consensus.cc:697] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:29.565724 15562 tablet_bootstrap.cc:492] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.562857 15568 raft_consensus.cc:359] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.566664 15568 raft_consensus.cc:385] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.568006 15568 raft_consensus.cc:740] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.566399 15605 consensus_queue.cc:237] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.569037 15568 consensus_queue.cc:260] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.566383 15562 ts_tablet_manager.cc:1403] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.024s
I20260709 13:21:29.557329 15564 raft_consensus.cc:359] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.572590 15564 raft_consensus.cc:385] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.572969 15564 raft_consensus.cc:740] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.573302 15568 ts_tablet_manager.cc:1434] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:21:29.575551 15568 tablet_bootstrap.cc:492] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:29.575845 15563 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.576321 15563 ts_tablet_manager.cc:1403] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.015s
I20260709 13:21:29.573925 15564 consensus_queue.cc:260] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.579020 15563 raft_consensus.cc:359] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.579658 15563 raft_consensus.cc:385] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.579918 15563 raft_consensus.cc:740] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.580476 15564 ts_tablet_manager.cc:1434] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.003s
I20260709 13:21:29.580660 15563 consensus_queue.cc:260] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.581430 15564 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:29.582935 15563 ts_tablet_manager.cc:1434] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:21:29.585546 15568 tablet_bootstrap.cc:654] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.586118 15563 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.588238 15564 tablet_bootstrap.cc:654] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.576455 15562 raft_consensus.cc:359] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.591444 15562 raft_consensus.cc:385] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.591791 15562 raft_consensus.cc:740] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.592643 15562 consensus_queue.cc:260] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.595026 15562 ts_tablet_manager.cc:1434] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:21:29.596369 15562 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.596537 15563 tablet_bootstrap.cc:654] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.600095 15635 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.600600 15635 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.604184 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:29.604962 15517 raft_consensus.cc:2393] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:29.605649 15563 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.606140 15563 ts_tablet_manager.cc:1403] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.006s
I20260709 13:21:29.606212 15562 tablet_bootstrap.cc:654] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.605679 15564 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:29.607122 15564 ts_tablet_manager.cc:1403] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.004s
I20260709 13:21:29.608878 15563 raft_consensus.cc:359] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.609543 15563 raft_consensus.cc:385] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.609881 15563 raft_consensus.cc:740] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.609592 15635 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:29.609725 15564 raft_consensus.cc:359] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.612625 15564 raft_consensus.cc:385] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.612922 15564 raft_consensus.cc:740] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.612867 15563 consensus_queue.cc:260] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.613782 15564 consensus_queue.cc:260] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.615628 15563 ts_tablet_manager.cc:1434] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:21:29.615772 15564 ts_tablet_manager.cc:1434] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:21:29.616585 15563 tablet_bootstrap.cc:492] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.613042 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:29.618494 15336 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:29.619302 15635 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.608497 15192 catalog_manager.cc:5697] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:29.625314 15563 tablet_bootstrap.cc:654] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.616585 15564 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:29.633680 15635 raft_consensus.cc:2749] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.635669 15564 tablet_bootstrap.cc:654] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.651211 15659 raft_consensus.cc:493] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.651993 15659 raft_consensus.cc:515] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.654202 15564 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:29.654925 15564 ts_tablet_manager.cc:1403] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.002s
I20260709 13:21:29.655435 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:29.655853 15659 leader_election.cc:290] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:29.656149 15368 raft_consensus.cc:2468] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:29.657496 15481 leader_election.cc:304] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:29.657703 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:29.658390 15442 raft_consensus.cc:2468] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:29.658717 15659 raft_consensus.cc:2804] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.659097 15659 raft_consensus.cc:493] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.659689 15564 raft_consensus.cc:359] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.660645 15564 raft_consensus.cc:385] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.660983 15564 raft_consensus.cc:740] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.661851 15564 consensus_queue.cc:260] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.663240 15659 raft_consensus.cc:3060] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.665405 15563 tablet_bootstrap.cc:492] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.666064 15563 ts_tablet_manager.cc:1403] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.010s
I20260709 13:21:29.667467 15568 tablet_bootstrap.cc:492] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.668190 15568 ts_tablet_manager.cc:1403] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.005s
I20260709 13:21:29.669637 15563 raft_consensus.cc:359] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.670590 15563 raft_consensus.cc:385] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.670334 15659 raft_consensus.cc:515] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.670992 15563 raft_consensus.cc:740] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.671151 15568 raft_consensus.cc:359] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.671938 15563 consensus_queue.cc:260] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.673095 15622 raft_consensus.cc:493] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.673219 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:29.673822 15368 raft_consensus.cc:3060] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.673604 15622 raft_consensus.cc:515] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.675153 15564 ts_tablet_manager.cc:1434] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:21:29.676522 15564 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:29.676725 15622 leader_election.cc:290] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:29.678290 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427b294c2f944e7ca234578e92b6126a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:29.680756 15294 raft_consensus.cc:2468] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:29.682519 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:29.683099 15442 raft_consensus.cc:3060] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.683022 15410 leader_election.cc:304] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:29.684121 15664 raft_consensus.cc:2804] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.684466 15664 raft_consensus.cc:493] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.679909 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427b294c2f944e7ca234578e92b6126a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:29.684718 15563 ts_tablet_manager.cc:1434] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.005s
I20260709 13:21:29.685117 15370 raft_consensus.cc:2468] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:29.684775 15664 raft_consensus.cc:3060] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.688149 15563 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.697161 15563 tablet_bootstrap.cc:654] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.698230 15368 raft_consensus.cc:2468] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:29.699621 15481 leader_election.cc:304] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:29.700402 15662 raft_consensus.cc:2804] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.700908 15662 raft_consensus.cc:697] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:29.701159 15664 raft_consensus.cc:515] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.671949 15568 raft_consensus.cc:385] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.702731 15568 raft_consensus.cc:740] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.686858 15564 tablet_bootstrap.cc:654] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.701771 15662 consensus_queue.cc:237] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.704429 15568 consensus_queue.cc:260] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.672773 15659 leader_election.cc:290] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:29.707144 15568 ts_tablet_manager.cc:1434] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:21:29.708050 15568 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:29.710705 15442 raft_consensus.cc:2468] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:29.729869 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427b294c2f944e7ca234578e92b6126a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:29.730839 15294 raft_consensus.cc:3060] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.736215 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "427b294c2f944e7ca234578e92b6126a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:29.736873 15368 raft_consensus.cc:3060] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.738581 15568 tablet_bootstrap.cc:654] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.739187 15294 raft_consensus.cc:2468] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:29.739364 15191 catalog_manager.cc:5697] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.740507 15410 leader_election.cc:304] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:29.741430 15622 raft_consensus.cc:2804] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.742053 15622 raft_consensus.cc:697] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:29.732585 15664 leader_election.cc:290] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:29.743132 15622 consensus_queue.cc:237] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.745402 15563 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.745882 15563 ts_tablet_manager.cc:1403] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.017s
I20260709 13:21:29.743984 15368 raft_consensus.cc:2468] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:29.753686 15563 raft_consensus.cc:359] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.754290 15563 raft_consensus.cc:385] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.754530 15563 raft_consensus.cc:740] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.755334 15563 consensus_queue.cc:260] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.757773 15562 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.757791 15563 ts_tablet_manager.cc:1434] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:21:29.758569 15562 ts_tablet_manager.cc:1403] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.162s	user 0.036s	sys 0.057s
I20260709 13:21:29.758961 15568 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.760098 15563 tablet_bootstrap.cc:492] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.759482 15568 ts_tablet_manager.cc:1403] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.005s
I20260709 13:21:29.761607 15562 raft_consensus.cc:359] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.762737 15562 raft_consensus.cc:385] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.762991 15562 raft_consensus.cc:740] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.763574 15568 raft_consensus.cc:359] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.763661 15562 consensus_queue.cc:260] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.764387 15568 raft_consensus.cc:385] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.764739 15568 raft_consensus.cc:740] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.765419 15562 ts_tablet_manager.cc:1434] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.002s
I20260709 13:21:29.765528 15568 consensus_queue.cc:260] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.768297 15568 ts_tablet_manager.cc:1434] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:29.767385 15563 tablet_bootstrap.cc:654] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.769467 15568 tablet_bootstrap.cc:492] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:29.773458 15564 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:29.774040 15564 ts_tablet_manager.cc:1403] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.003s
I20260709 13:21:29.776124 15568 tablet_bootstrap.cc:654] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.778348 15662 consensus_queue.cc:1048] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:29.780009 15191 catalog_manager.cc:5697] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:29.786572 15564 raft_consensus.cc:359] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.795063 15622 raft_consensus.cc:493] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.795670 15622 raft_consensus.cc:515] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.792984 15563 tablet_bootstrap.cc:492] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:29.798521 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:29.799204 15368 raft_consensus.cc:2468] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:29.799841 15562 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.800123 15568 tablet_bootstrap.cc:492] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.800450 15563 ts_tablet_manager.cc:1403] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.006s
I20260709 13:21:29.800591 15568 ts_tablet_manager.cc:1403] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.008s
I20260709 13:21:29.800716 15407 leader_election.cc:304] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:29.801802 15594 raft_consensus.cc:493] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.802243 15594 raft_consensus.cc:515] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.803495 15563 raft_consensus.cc:359] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.804456 15563 raft_consensus.cc:385] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.804490 15594 leader_election.cc:290] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:29.805009 15563 raft_consensus.cc:740] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.806699 15562 tablet_bootstrap.cc:654] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.807291 15664 raft_consensus.cc:2804] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.807499 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e39f7656f2748f994ca243f51380c1c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:29.807891 15664 raft_consensus.cc:493] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.808385 15664 raft_consensus.cc:3060] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.806468 15568 raft_consensus.cc:359] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.808444 15294 raft_consensus.cc:2468] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:29.807562 15563 consensus_queue.cc:260] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.809898 15410 leader_election.cc:304] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:29.811702 15568 raft_consensus.cc:385] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.812120 15568 raft_consensus.cc:740] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.812743 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e39f7656f2748f994ca243f51380c1c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:29.813632 15517 raft_consensus.cc:2468] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:29.815295 15568 consensus_queue.cc:260] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.815968 15594 raft_consensus.cc:2804] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.816366 15712 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.818012 15712 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.816931 15564 raft_consensus.cc:385] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.819164 15594 raft_consensus.cc:493] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.820122 15662 raft_consensus.cc:493] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.821476 15712 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:29.827492 15662 raft_consensus.cc:515] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.829599 15662 leader_election.cc:290] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:29.821683 15568 ts_tablet_manager.cc:1434] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.002s
I20260709 13:21:29.821952 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:29.830039 15664 raft_consensus.cc:515] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.830793 15517 raft_consensus.cc:2468] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:29.832012 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:29.823565 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:29.834558 15410 leader_election.cc:343] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.835047 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:29.835624 15594 raft_consensus.cc:3060] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.835755 15370 raft_consensus.cc:2468] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:29.826833 15622 leader_election.cc:290] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:29.838498 15481 leader_election.cc:304] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:29.826323 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
W20260709 13:21:29.840147 15407 leader_election.cc:343] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.840615 15407 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:29.841385 15712 raft_consensus.cc:2749] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.841496 15662 raft_consensus.cc:2804] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.841998 15662 raft_consensus.cc:493] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.842341 15662 raft_consensus.cc:3060] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.851536 15443 raft_consensus.cc:2393] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:29.852363 15568 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:29.857223 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:29.858054 15368 raft_consensus.cc:3060] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.863543 15564 raft_consensus.cc:740] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.866077 15664 leader_election.cc:290] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:29.867204 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
W20260709 13:21:29.870433 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:29.871371 15563 ts_tablet_manager.cc:1434] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.071s	user 0.017s	sys 0.044s
I20260709 13:21:29.872440 15563 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:29.873584 15562 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.870496 15564 consensus_queue.cc:260] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.874166 15562 ts_tablet_manager.cc:1403] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.008s
I20260709 13:21:29.877110 15562 raft_consensus.cc:359] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.877816 15562 raft_consensus.cc:385] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.878152 15562 raft_consensus.cc:740] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.878849 15562 consensus_queue.cc:260] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.880882 15562 ts_tablet_manager.cc:1434] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:21:29.889748 15562 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:29.883450 15662 raft_consensus.cc:515] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.893185 15564 ts_tablet_manager.cc:1434] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.119s	user 0.007s	sys 0.000s
I20260709 13:21:29.894182 15564 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:29.890838 15666 consensus_queue.cc:1048] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:29.901655 15563 tablet_bootstrap.cc:654] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.905714 15568 tablet_bootstrap.cc:654] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.909693 15517 raft_consensus.cc:2393] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:29.910230 15662 leader_election.cc:290] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:29.914053 15564 tablet_bootstrap.cc:654] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.919934 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:29.925010 15594 raft_consensus.cc:515] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.926805 15368 raft_consensus.cc:2468] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:29.940801 15562 tablet_bootstrap.cc:654] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:29.932879 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc719509cbf4d3aad5be6412ec80022" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:29.942528 15443 raft_consensus.cc:2393] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:29.945833 15484 leader_election.cc:304] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:29.946130 15568 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:29.944986 15623 raft_consensus.cc:493] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.947170 15662 raft_consensus.cc:2749] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:29.947166 15623 raft_consensus.cc:515] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.948168 15407 leader_election.cc:304] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:29.948968 15664 raft_consensus.cc:2804] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.949455 15664 raft_consensus.cc:697] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:29.950095 15623 leader_election.cc:290] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:29.952014 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:29.952677 15442 raft_consensus.cc:2468] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:29.950268 15664 consensus_queue.cc:237] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.954036 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e39f7656f2748f994ca243f51380c1c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:29.955109 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:29.946733 15568 ts_tablet_manager.cc:1403] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.095s	user 0.019s	sys 0.008s
I20260709 13:21:29.955775 15370 raft_consensus.cc:2468] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:29.956234 15517 raft_consensus.cc:3060] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.956497 15594 leader_election.cc:290] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:29.963028 15579 raft_consensus.cc:493] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.965466 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e39f7656f2748f994ca243f51380c1c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:29.966082 15295 raft_consensus.cc:3060] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.965940 15568 raft_consensus.cc:359] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.966656 15568 raft_consensus.cc:385] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.966936 15568 raft_consensus.cc:740] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.968645 15564 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:29.969135 15564 ts_tablet_manager.cc:1403] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.075s	user 0.015s	sys 0.015s
I20260709 13:21:29.962453 15258 leader_election.cc:304] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:29.963564 15579 raft_consensus.cc:515] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:29.972105 15605 raft_consensus.cc:2804] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.972606 15605 raft_consensus.cc:493] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.973031 15605 raft_consensus.cc:3060] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.974560 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128c6f471a6b4c53bdd2c868f551b542" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:29.973245 15579 leader_election.cc:290] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
W20260709 13:21:29.976807 15333 leader_election.cc:343] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:29.976526 15568 consensus_queue.cc:260] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:29.977139 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128c6f471a6b4c53bdd2c868f551b542" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:29.978637 15295 raft_consensus.cc:2468] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:29.978883 15442 raft_consensus.cc:2468] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:29.980801 15410 leader_election.cc:304] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:29.981467 15568 ts_tablet_manager.cc:1434] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:21:29.981614 15594 raft_consensus.cc:2804] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:29.982116 15594 raft_consensus.cc:697] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:29.982342 15336 leader_election.cc:304] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:29.982964 15594 consensus_queue.cc:237] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:29.984323 15579 raft_consensus.cc:2804] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:29.984716 15579 raft_consensus.cc:493] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.985067 15579 raft_consensus.cc:3060] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:29.988502 15564 raft_consensus.cc:359] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.989319 15564 raft_consensus.cc:385] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:29.989619 15564 raft_consensus.cc:740] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:29.990392 15564 consensus_queue.cc:260] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:29.993045 15517 raft_consensus.cc:2468] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:29.996011 15568 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:29.997223 15562 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:29.997751 15562 ts_tablet_manager.cc:1403] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.110s	user 0.008s	sys 0.016s
I20260709 13:21:29.999876 15579 raft_consensus.cc:515] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.004750 15568 tablet_bootstrap.cc:654] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.006721 15605 raft_consensus.cc:515] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.009295 15563 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:30.009825 15563 ts_tablet_manager.cc:1403] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.138s	user 0.019s	sys 0.005s
I20260709 13:21:30.010809 15567 consensus_queue.cc:1048] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.012373 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:30.012984 15442 raft_consensus.cc:3060] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.015671 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:30.016513 15370 raft_consensus.cc:3060] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.018371 15567 consensus_queue.cc:1048] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.012691 15563 raft_consensus.cc:359] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.023157 15563 raft_consensus.cc:385] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.023051 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128c6f471a6b4c53bdd2c868f551b542" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:30.023669 15563 raft_consensus.cc:740] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.023893 15443 raft_consensus.cc:3060] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.024390 15563 consensus_queue.cc:260] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.024887 15562 raft_consensus.cc:359] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.025537 15562 raft_consensus.cc:385] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.025774 15562 raft_consensus.cc:740] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.026422 15562 consensus_queue.cc:260] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.031294 15442 raft_consensus.cc:2468] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:30.032516 15261 leader_election.cc:304] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:30.033931 15563 ts_tablet_manager.cc:1434] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 13:21:30.036147 15563 tablet_bootstrap.cc:492] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:30.040601 15721 raft_consensus.cc:2804] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.023175 15579 leader_election.cc:290] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:30.034854 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128c6f471a6b4c53bdd2c868f551b542" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:30.041801 15605 leader_election.cc:290] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.048506 15721 raft_consensus.cc:697] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
W20260709 13:21:30.049569 15333 leader_election.cc:343] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.049431 15721 consensus_queue.cc:237] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.051072 15370 raft_consensus.cc:2468] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:30.054395 15562 ts_tablet_manager.cc:1434] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:21:30.044886 15563 tablet_bootstrap.cc:654] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.044728 15193 catalog_manager.cc:5697] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.059648 15662 raft_consensus.cc:493] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.060182 15662 raft_consensus.cc:515] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.062263 15662 leader_election.cc:290] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:30.064158 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54afc04cc1b742d1b309764bb898bc6b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:30.064879 15442 raft_consensus.cc:2468] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:30.066089 15484 leader_election.cc:304] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:30.066913 15662 raft_consensus.cc:2804] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.067178 15662 raft_consensus.cc:493] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.067384 15662 raft_consensus.cc:3060] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.067437 15563 tablet_bootstrap.cc:492] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:30.067951 15563 ts_tablet_manager.cc:1403] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.032s	user 0.006s	sys 0.010s
I20260709 13:21:30.074077 15562 tablet_bootstrap.cc:492] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.075201 15563 raft_consensus.cc:359] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.081949 15563 raft_consensus.cc:385] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.082274 15563 raft_consensus.cc:740] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.076215 15564 ts_tablet_manager.cc:1434] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.000s
I20260709 13:21:30.076951 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54afc04cc1b742d1b309764bb898bc6b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:30.082983 15662 raft_consensus.cc:515] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.082998 15563 consensus_queue.cc:260] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.089480 15563 ts_tablet_manager.cc:1434] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
W20260709 13:21:30.090480 15484 leader_election.cc:343] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.092896 15443 raft_consensus.cc:2468] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:30.094129 15336 leader_election.cc:304] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:30.094928 15594 raft_consensus.cc:493] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.096838 15594 raft_consensus.cc:515] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.098575 15563 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:30.099011 15594 leader_election.cc:290] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:30.095055 15579 raft_consensus.cc:2804] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.100203 15579 raft_consensus.cc:697] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:30.085309 15662 leader_election.cc:290] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:30.101052 15579 consensus_queue.cc:237] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.096275 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54afc04cc1b742d1b309764bb898bc6b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:30.095340 15664 raft_consensus.cc:493] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.103675 15664 raft_consensus.cc:515] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.104615 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.106211 15563 tablet_bootstrap.cc:654] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:30.106585 15407 leader_election.cc:343] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.107548 15664 leader_election.cc:290] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:30.085991 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54afc04cc1b742d1b309764bb898bc6b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:30.087203 15564 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:30.109804 15442 raft_consensus.cc:3060] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.111086 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:30.112432 15407 leader_election.cc:343] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.112898 15407 leader_election.cc:304] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:30.113574 15568 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:30.107939 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65aff9e32079465b85d18fff1c73b28a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:30.114483 15594 raft_consensus.cc:2749] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.114926 15517 raft_consensus.cc:2468] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:30.115062 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65aff9e32079465b85d18fff1c73b28a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:30.115343 15564 tablet_bootstrap.cc:654] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.115908 15568 ts_tablet_manager.cc:1403] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.007s
W20260709 13:21:30.116330 15410 leader_election.cc:343] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.117081 15407 leader_election.cc:304] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:30.118214 15664 raft_consensus.cc:2804] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.094609 15193 catalog_manager.cc:5697] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.118660 15664 raft_consensus.cc:493] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.119174 15664 raft_consensus.cc:3060] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.120508 15442 raft_consensus.cc:2468] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:30.122149 15484 leader_election.cc:304] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:30.123245 15662 raft_consensus.cc:2804] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.085685 15562 tablet_bootstrap.cc:654] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.123713 15662 raft_consensus.cc:697] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
W20260709 13:21:30.124454 15484 leader_election.cc:343] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.124635 15662 consensus_queue.cc:237] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.130394 15564 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:30.130590 15562 tablet_bootstrap.cc:492] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:30.130920 15564 ts_tablet_manager.cc:1403] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.000s
I20260709 13:21:30.131197 15562 ts_tablet_manager.cc:1403] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.002s
I20260709 13:21:30.133668 15564 raft_consensus.cc:359] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.133865 15562 raft_consensus.cc:359] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.134344 15564 raft_consensus.cc:385] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.134649 15562 raft_consensus.cc:385] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.134796 15564 raft_consensus.cc:740] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.136909 15623 raft_consensus.cc:493] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.137089 15562 raft_consensus.cc:740] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.137308 15666 consensus_queue.cc:1048] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.137560 15623 raft_consensus.cc:515] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.137625 15564 consensus_queue.cc:260] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.138043 15562 consensus_queue.cc:260] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.139806 15623 leader_election.cc:290] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:30.140192 15562 ts_tablet_manager.cc:1434] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:21:30.136533 15563 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:30.140954 15563 ts_tablet_manager.cc:1403] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.043s	user 0.002s	sys 0.011s
I20260709 13:21:30.141381 15562 tablet_bootstrap.cc:492] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.141947 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.142633 15367 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:30.143963 15258 leader_election.cc:304] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:30.144397 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:30.145076 15517 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:30.143801 15563 raft_consensus.cc:359] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.146638 15563 raft_consensus.cc:385] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.146930 15563 raft_consensus.cc:740] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.118611 15568 raft_consensus.cc:359] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.147625 15568 raft_consensus.cc:385] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.147902 15568 raft_consensus.cc:740] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.148239 15562 tablet_bootstrap.cc:654] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.149156 15623 raft_consensus.cc:2804] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.149608 15623 raft_consensus.cc:493] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.149773 15563 consensus_queue.cc:260] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.150105 15623 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.148736 15664 raft_consensus.cc:515] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.139815 15564 ts_tablet_manager.cc:1434] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:21:30.148695 15568 consensus_queue.cc:260] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.152490 15564 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:30.155721 15190 catalog_manager.cc:5697] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.163414 15564 tablet_bootstrap.cc:654] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.172760 15563 ts_tablet_manager.cc:1434] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:21:30.174154 15563 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:30.176262 15568 ts_tablet_manager.cc:1434] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.001s
I20260709 13:21:30.177597 15568 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:30.180608 15623 raft_consensus.cc:515] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.180931 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65aff9e32079465b85d18fff1c73b28a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:30.185384 15662 raft_consensus.cc:493] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.185895 15662 raft_consensus.cc:515] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.189095 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128c6f471a6b4c53bdd2c868f551b542" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.189581 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128c6f471a6b4c53bdd2c868f551b542" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:30.190397 15444 raft_consensus.cc:2393] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:30.190580 15568 tablet_bootstrap.cc:654] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.183949 15517 raft_consensus.cc:3060] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.191754 15484 leader_election.cc:304] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:30.192649 15666 raft_consensus.cc:3060] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.196976 15664 leader_election.cc:290] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:30.195397 15662 leader_election.cc:290] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.199105 15623 leader_election.cc:290] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:30.190474 15594 consensus_queue.cc:1048] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:21:30.201650 15664 raft_consensus.cc:493] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.202143 15664 raft_consensus.cc:515] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.205080 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9cf658f5d1340238f97e4d01fa8c07e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.204444 15666 raft_consensus.cc:2749] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:30.207062 15407 leader_election.cc:343] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.200416 15191 catalog_manager.cc:5697] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.211189 15664 leader_election.cc:290] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:30.216218 15562 tablet_bootstrap.cc:492] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:30.216710 15562 ts_tablet_manager.cc:1403] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.076s	user 0.015s	sys 0.015s
I20260709 13:21:30.219501 15562 raft_consensus.cc:359] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.220185 15562 raft_consensus.cc:385] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.220446 15562 raft_consensus.cc:740] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.220496 15563 tablet_bootstrap.cc:654] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.221163 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65aff9e32079465b85d18fff1c73b28a" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
W20260709 13:21:30.223186 15410 leader_election.cc:343] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.221164 15562 consensus_queue.cc:260] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.226984 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9cf658f5d1340238f97e4d01fa8c07e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:30.228973 15407 leader_election.cc:343] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.229501 15407 leader_election.cc:304] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:30.230895 15594 raft_consensus.cc:2749] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.230064 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:30.232066 15518 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.235536 15568 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:30.236135 15568 ts_tablet_manager.cc:1403] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.000s
I20260709 13:21:30.238634 15562 ts_tablet_manager.cc:1434] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.002s
I20260709 13:21:30.239594 15562 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.241809 15518 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:30.246045 15594 consensus_queue.cc:1048] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.246490 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:30.247931 15370 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.254310 15564 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:30.254987 15564 ts_tablet_manager.cc:1403] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.000s
I20260709 13:21:30.255484 15192 catalog_manager.cc:5697] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.256958 15258 leader_election.cc:304] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:30.257761 15605 raft_consensus.cc:2804] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.257949 15659 consensus_queue.cc:1048] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:21:30.261281 15568 raft_consensus.cc:359] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.261956 15568 raft_consensus.cc:385] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.262200 15568 raft_consensus.cc:740] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.257592 15564 raft_consensus.cc:359] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.263273 15564 raft_consensus.cc:385] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.264070 15562 tablet_bootstrap.cc:654] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.264485 15605 raft_consensus.cc:697] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:30.267443 15564 raft_consensus.cc:740] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.262923 15568 consensus_queue.cc:260] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.269672 15564 consensus_queue.cc:260] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.275637 15568 ts_tablet_manager.cc:1434] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:21:30.276636 15568 tablet_bootstrap.cc:492] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:30.277251 15564 ts_tablet_manager.cc:1434] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.003s
I20260709 13:21:30.277995 15564 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:30.280795 15370 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:30.287153 15605 consensus_queue.cc:237] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.288727 15563 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:30.289227 15563 ts_tablet_manager.cc:1403] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.115s	user 0.010s	sys 0.014s
I20260709 13:21:30.284122 15517 raft_consensus.cc:2468] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:30.299101 15563 raft_consensus.cc:359] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.300987 15563 raft_consensus.cc:385] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.301301 15563 raft_consensus.cc:740] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.301936 15563 consensus_queue.cc:260] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.307262 15407 leader_election.cc:304] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:30.309051 15709 raft_consensus.cc:2804] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.309656 15709 raft_consensus.cc:697] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:30.310500 15709 consensus_queue.cc:237] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.315325 15568 tablet_bootstrap.cc:654] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.319567 15579 raft_consensus.cc:493] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.320119 15579 raft_consensus.cc:515] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.322299 15579 leader_election.cc:290] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.331588 15564 tablet_bootstrap.cc:654] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.336030 15659 raft_consensus.cc:493] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.336606 15659 raft_consensus.cc:515] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.340291 15666 raft_consensus.cc:493] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.323329 15563 ts_tablet_manager.cc:1434] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
I20260709 13:21:30.324026 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f133a6db45994178b473a5b1809d367f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:30.340803 15666 raft_consensus.cc:515] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.341581 15442 raft_consensus.cc:2468] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:30.342854 15336 leader_election.cc:304] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:30.343039 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f133a6db45994178b473a5b1809d367f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:30.340344 15721 raft_consensus.cc:493] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.344018 15579 raft_consensus.cc:2804] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.344461 15579 raft_consensus.cc:493] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.344719 15659 leader_election.cc:290] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.344879 15579 raft_consensus.cc:3060] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.342885 15666 leader_election.cc:290] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:30.343788 15721 raft_consensus.cc:515] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.345757 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.346578 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:30.347244 15442 raft_consensus.cc:2468] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
W20260709 13:21:30.347393 15481 leader_election.cc:343] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.345728 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.348435 15484 leader_election.cc:304] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:30.349059 15721 leader_election.cc:290] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:30.349324 15666 raft_consensus.cc:2804] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.349769 15666 raft_consensus.cc:493] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.350147 15666 raft_consensus.cc:3060] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.350500 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54afc04cc1b742d1b309764bb898bc6b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:30.350776 15336 leader_election.cc:343] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:30.349089 15481 leader_election.cc:343] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.352655 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:30.354956 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54afc04cc1b742d1b309764bb898bc6b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:30.341598 15563 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:30.355793 15442 raft_consensus.cc:2393] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:30.354373 15564 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:30.356586 15564 ts_tablet_manager.cc:1403] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.003s
I20260709 13:21:30.349843 15721 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.359206 15721 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.360126 15258 leader_election.cc:304] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:30.360921 15623 raft_consensus.cc:3060] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.362187 15564 raft_consensus.cc:359] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.364269 15564 raft_consensus.cc:385] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.362993 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:30.363626 15721 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.364893 15568 tablet_bootstrap.cc:492] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:30.366277 15442 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:30.362661 15666 raft_consensus.cc:515] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.366489 15568 ts_tablet_manager.cc:1403] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.002s
I20260709 13:21:30.367920 15261 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:30.370651 15568 raft_consensus.cc:359] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.372699 15623 raft_consensus.cc:2749] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.372852 15709 consensus_queue.cc:1048] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:30.374089 15484 leader_election.cc:343] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.371316 15568 raft_consensus.cc:385] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.374550 15484 leader_election.cc:304] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:30.374907 15568 raft_consensus.cc:740] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.375607 15568 consensus_queue.cc:260] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.376134 15659 raft_consensus.cc:2749] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.378015 15568 ts_tablet_manager.cc:1434] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:21:30.378993 15568 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:30.371925 15721 raft_consensus.cc:2804] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.379560 15721 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.380069 15721 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.383780 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.385838 15709 consensus_queue.cc:1048] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.400527 15368 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:30.403126 15189 catalog_manager.cc:5697] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.407371 15721 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.410532 15721 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.411417 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:30.412027 15368 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.412380 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:30.412992 15444 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.387934 15563 tablet_bootstrap.cc:654] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.391338 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:30.418919 15367 raft_consensus.cc:3060] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.420399 15368 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:30.421739 15258 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:30.392038 15666 leader_election.cc:290] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.367388 15564 raft_consensus.cc:740] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.423823 15721 raft_consensus.cc:2804] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.424548 15721 raft_consensus.cc:697] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:30.424434 15564 consensus_queue.cc:260] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.425403 15721 consensus_queue.cc:237] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.392244 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:30.429946 15442 raft_consensus.cc:3060] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.393054 15568 tablet_bootstrap.cc:654] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.426535 15564 ts_tablet_manager.cc:1434] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.000s
I20260709 13:21:30.434283 15711 consensus_queue.cc:1048] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.435393 15564 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:30.451040 15562 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:30.451754 15562 ts_tablet_manager.cc:1403] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.212s	user 0.045s	sys 0.047s
I20260709 13:21:30.452327 15711 raft_consensus.cc:493] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.452849 15711 raft_consensus.cc:515] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.506394 15562 raft_consensus.cc:359] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.507196 15562 raft_consensus.cc:385] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.507465 15562 raft_consensus.cc:740] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.508170 15562 consensus_queue.cc:260] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.512512 15605 consensus_queue.cc:1048] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.512837 15562 ts_tablet_manager.cc:1434] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.025s
I20260709 13:21:30.513943 15562 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.514940 15192 catalog_manager.cc:5697] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.520390 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f48e3fa27c4c6487669b0d45130af7" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:30.520853 15562 tablet_bootstrap.cc:654] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.521065 15444 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
W20260709 13:21:30.522001 15407 leader_election.cc:343] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.523216 15711 leader_election.cc:290] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:30.524868 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f48e3fa27c4c6487669b0d45130af7" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:30.527665 15410 leader_election.cc:343] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.528414 15410 leader_election.cc:304] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:30.531661 15709 raft_consensus.cc:2749] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.541788 15709 consensus_queue.cc:1048] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.547149 15666 consensus_queue.cc:1048] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.558846 15367 raft_consensus.cc:2468] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:30.576370 15442 raft_consensus.cc:2468] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:30.513015 15579 raft_consensus.cc:515] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.588482 15666 raft_consensus.cc:493] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.594915 15623 consensus_queue.cc:1048] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.609529 15564 tablet_bootstrap.cc:654] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.641465 15682 consensus_queue.cc:1048] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.642962 15579 leader_election.cc:290] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:30.624598 15481 leader_election.cc:304] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:30.645061 15662 raft_consensus.cc:2804] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.633744 15563 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:30.645717 15662 raft_consensus.cc:697] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:30.646062 15563 ts_tablet_manager.cc:1403] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.305s	user 0.011s	sys 0.004s
I20260709 13:21:30.636435 15666 raft_consensus.cc:515] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.646651 15662 consensus_queue.cc:237] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.648885 15563 raft_consensus.cc:359] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.649619 15563 raft_consensus.cc:385] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.649297 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c5fe7b4ba54f94a897b776dcc58565" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:30.649923 15563 raft_consensus.cc:740] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.644980 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f133a6db45994178b473a5b1809d367f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:30.652735 15444 raft_consensus.cc:3060] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.652984 15563 consensus_queue.cc:260] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.657799 15562 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:30.631197 15659 consensus_queue.cc:1048] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.616663 15721 raft_consensus.cc:493] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:30.651671 15484 leader_election.cc:343] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.644105 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f133a6db45994178b473a5b1809d367f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:30.658401 15562 ts_tablet_manager.cc:1403] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.145s	user 0.065s	sys 0.054s
I20260709 13:21:30.662045 15666 leader_election.cc:290] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:30.662833 15721 raft_consensus.cc:515] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.663254 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c5fe7b4ba54f94a897b776dcc58565" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.664191 15444 raft_consensus.cc:2468] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:30.666884 15568 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
W20260709 13:21:30.673257 15481 leader_election.cc:343] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.673774 15481 leader_election.cc:304] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:30.674094 15568 ts_tablet_manager.cc:1403] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.295s	user 0.081s	sys 0.083s
I20260709 13:21:30.674966 15662 raft_consensus.cc:2749] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.675343 15721 leader_election.cc:290] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:30.668442 15563 ts_tablet_manager.cc:1434] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.022s	user 0.001s	sys 0.005s
I20260709 13:21:30.676442 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65aff9e32079465b85d18fff1c73b28a" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:30.677305 15518 raft_consensus.cc:2393] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:30.663286 15562 raft_consensus.cc:359] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:30.663306 15336 leader_election.cc:343] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.679262 15336 leader_election.cc:304] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:30.680316 15682 raft_consensus.cc:2804] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.676975 15563 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:30.680827 15562 raft_consensus.cc:385] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.681228 15562 raft_consensus.cc:740] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.682801 15664 raft_consensus.cc:493] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.681879 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65aff9e32079465b85d18fff1c73b28a" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:30.683382 15664 raft_consensus.cc:515] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.682753 15562 consensus_queue.cc:260] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.685469 15664 leader_election.cc:290] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:30.686182 15562 ts_tablet_manager.cc:1434] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:21:30.687387 15562 tablet_bootstrap.cc:492] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.688043 15261 leader_election.cc:304] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:30.679804 15568 raft_consensus.cc:359] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.691452 15605 raft_consensus.cc:3060] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.692735 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c124e9e01448409d9a64892dddf7c610" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.693836 15562 tablet_bootstrap.cc:654] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:30.694737 15407 leader_election.cc:343] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.700042 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c124e9e01448409d9a64892dddf7c610" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:30.700541 15564 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:30.691672 15568 raft_consensus.cc:385] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.702035 15568 raft_consensus.cc:740] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.702837 15568 consensus_queue.cc:260] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:30.703873 15410 leader_election.cc:343] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.704571 15410 leader_election.cc:304] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:30.709452 15563 tablet_bootstrap.cc:654] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.707536 15709 raft_consensus.cc:2749] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.716006 15769 consensus_queue.cc:1048] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:30.716336 15682 raft_consensus.cc:697] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:30.719825 15682 consensus_queue.cc:237] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.726537 15605 raft_consensus.cc:2749] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.718581 15568 ts_tablet_manager.cc:1434] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.002s
I20260709 13:21:30.729223 15568 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:30.725503 15190 catalog_manager.cc:5697] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.736159 15568 tablet_bootstrap.cc:654] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.744021 15563 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:30.747282 15563 ts_tablet_manager.cc:1403] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.004s
I20260709 13:21:30.701251 15564 ts_tablet_manager.cc:1403] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.266s	user 0.014s	sys 0.002s
I20260709 13:21:30.716988 15709 consensus_queue.cc:1048] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:30.760812 15336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 53980.6us
I20260709 13:21:30.762312 15564 raft_consensus.cc:359] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.765108 15564 raft_consensus.cc:385] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.765645 15564 raft_consensus.cc:740] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.775141 15562 tablet_bootstrap.cc:492] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:30.776124 15562 ts_tablet_manager.cc:1403] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.089s	user 0.054s	sys 0.022s
I20260709 13:21:30.774698 15192 catalog_manager.cc:5697] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:30.782850 15721 raft_consensus.cc:493] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.801357 15721 raft_consensus.cc:515] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.803643 15721 leader_election.cc:290] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:30.767035 15564 consensus_queue.cc:260] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.805682 15564 ts_tablet_manager.cc:1434] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.000s
I20260709 13:21:30.806564 15564 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:30.781556 15562 raft_consensus.cc:359] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.808806 15562 raft_consensus.cc:385] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.809216 15562 raft_consensus.cc:740] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.810058 15562 consensus_queue.cc:260] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.812337 15564 tablet_bootstrap.cc:654] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.812917 15562 ts_tablet_manager.cc:1434] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 13:21:30.814188 15562 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.767482 15563 raft_consensus.cc:359] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.800578 15568 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:30.814790 15563 raft_consensus.cc:385] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.815028 15568 ts_tablet_manager.cc:1403] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.086s	user 0.034s	sys 0.020s
I20260709 13:21:30.815071 15563 raft_consensus.cc:740] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.815984 15563 consensus_queue.cc:260] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.817996 15563 ts_tablet_manager.cc:1434] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 13:21:30.817968 15568 raft_consensus.cc:359] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.818814 15568 raft_consensus.cc:385] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.819094 15563 tablet_bootstrap.cc:492] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:30.819082 15568 raft_consensus.cc:740] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.819871 15568 consensus_queue.cc:260] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.821789 15568 ts_tablet_manager.cc:1434] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:21:30.823173 15568 tablet_bootstrap.cc:492] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:30.825145 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:30.825455 15563 tablet_bootstrap.cc:654] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.825839 15518 raft_consensus.cc:2468] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:30.828727 15568 tablet_bootstrap.cc:654] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.837000 15562 tablet_bootstrap.cc:654] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.845046 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:30.845779 15368 raft_consensus.cc:2468] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:30.842351 15664 consensus_queue.cc:1048] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:30.860395 15258 leader_election.cc:304] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:30.861646 15605 raft_consensus.cc:2804] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.862154 15605 raft_consensus.cc:493] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.862543 15605 raft_consensus.cc:3060] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.874778 15605 raft_consensus.cc:515] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.880736 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:30.881551 15517 raft_consensus.cc:3060] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.882802 15562 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:30.883327 15562 ts_tablet_manager.cc:1403] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.069s	user 0.023s	sys 0.009s
I20260709 13:21:30.886065 15562 raft_consensus.cc:359] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.886734 15562 raft_consensus.cc:385] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.886998 15562 raft_consensus.cc:740] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.887692 15562 consensus_queue.cc:260] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.889334 15517 raft_consensus.cc:2468] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:30.893182 15765 raft_consensus.cc:493] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.893682 15258 leader_election.cc:304] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:30.893729 15765 raft_consensus.cc:515] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.896677 15765 leader_election.cc:290] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:30.897647 15562 ts_tablet_manager.cc:1434] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:21:30.898528 15605 leader_election.cc:290] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:30.898689 15562 tablet_bootstrap.cc:492] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.900020 15756 raft_consensus.cc:2804] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.905316 15756 raft_consensus.cc:697] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:30.901721 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c5fe7b4ba54f94a897b776dcc58565" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:30.906443 15756 consensus_queue.cc:237] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.907389 15295 raft_consensus.cc:2468] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:30.910723 15563 tablet_bootstrap.cc:492] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:30.911244 15563 ts_tablet_manager.cc:1403] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.030s
I20260709 13:21:30.902554 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c5fe7b4ba54f94a897b776dcc58565" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:30.903345 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:30.912732 15368 raft_consensus.cc:3060] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.916208 15564 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:30.916738 15564 ts_tablet_manager.cc:1403] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.110s	user 0.010s	sys 0.005s
I20260709 13:21:30.919564 15563 raft_consensus.cc:359] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.896803 15568 tablet_bootstrap.cc:492] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:30.922012 15568 ts_tablet_manager.cc:1403] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.099s	user 0.008s	sys 0.009s
I20260709 13:21:30.922910 15368 raft_consensus.cc:2468] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:30.925499 15564 raft_consensus.cc:359] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.926256 15564 raft_consensus.cc:385] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.920337 15563 raft_consensus.cc:385] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.926858 15517 raft_consensus.cc:2468] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:30.926589 15568 raft_consensus.cc:359] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.927573 15562 tablet_bootstrap.cc:654] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.927654 15568 raft_consensus.cc:385] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.928157 15568 raft_consensus.cc:740] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.926996 15563 raft_consensus.cc:740] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.929013 15568 consensus_queue.cc:260] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.929231 15563 consensus_queue.cc:260] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.926530 15564 raft_consensus.cc:740] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.928202 15333 leader_election.cc:304] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:30.931257 15765 raft_consensus.cc:2804] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:30.931692 15765 raft_consensus.cc:493] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.931948 15563 ts_tablet_manager.cc:1434] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:21:30.932237 15765 raft_consensus.cc:3060] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:30.936780 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:30.937595 15564 consensus_queue.cc:260] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:30.940007 15564 ts_tablet_manager.cc:1434] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:21:30.940474 15563 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:30.941264 15564 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:30.939221 15190 catalog_manager.cc:5697] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:30.948937 15564 tablet_bootstrap.cc:654] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.949929 15756 consensus_queue.cc:1048] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.953985 15721 consensus_queue.cc:1048] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:30.964210 15709 raft_consensus.cc:493] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.964852 15709 raft_consensus.cc:515] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.967720 15709 leader_election.cc:290] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:30.969489 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f585eec076224b80b0b6d6447d292eeb" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:30.970541 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f585eec076224b80b0b6d6447d292eeb" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
W20260709 13:21:30.971283 15407 leader_election.cc:343] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:30.972102 15407 leader_election.cc:343] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:30.972571 15407 leader_election.cc:304] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:30.973560 15711 raft_consensus.cc:2749] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:30.976428 15568 ts_tablet_manager.cc:1434] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 13:21:30.978541 15564 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:30.979072 15564 ts_tablet_manager.cc:1403] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.024s
I20260709 13:21:30.980298 15562 tablet_bootstrap.cc:492] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:30.980832 15562 ts_tablet_manager.cc:1403] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.013s
I20260709 13:21:30.975545 15765 raft_consensus.cc:515] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:30.981779 15564 raft_consensus.cc:359] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.982784 15564 raft_consensus.cc:385] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.983144 15564 raft_consensus.cc:740] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.984006 15564 consensus_queue.cc:260] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:30.985453 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c5fe7b4ba54f94a897b776dcc58565" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:30.986079 15516 raft_consensus.cc:3060] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.986690 15564 ts_tablet_manager.cc:1434] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:21:30.988019 15564 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:30.994035 15562 raft_consensus.cc:359] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:30.994781 15562 raft_consensus.cc:385] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.995048 15562 raft_consensus.cc:740] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.995476 15516 raft_consensus.cc:2468] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:30.996098 15564 tablet_bootstrap.cc:654] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.996886 15333 leader_election.cc:304] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:30.997776 15682 raft_consensus.cc:2804] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.998175 15563 tablet_bootstrap.cc:654] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.998273 15682 raft_consensus.cc:697] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:30.999280 15682 consensus_queue.cc:237] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.001062 15765 leader_election.cc:290] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:31.001938 15562 consensus_queue.cc:260] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.004540 15562 ts_tablet_manager.cc:1434] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:21:31.005539 15562 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:30.984990 15568 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:31.009811 15605 consensus_queue.cc:1048] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:31.011574 15562 tablet_bootstrap.cc:654] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.017161 15662 raft_consensus.cc:493] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.022714 15666 raft_consensus.cc:493] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.022589 15662 raft_consensus.cc:515] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.023247 15666 raft_consensus.cc:515] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.024937 15662 leader_election.cc:290] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.025291 15666 leader_election.cc:290] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:31.027987 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9cf658f5d1340238f97e4d01fa8c07e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:31.028069 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f48e3fa27c4c6487669b0d45130af7" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:31.028313 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55c5fe7b4ba54f94a897b776dcc58565" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:31.028719 15443 raft_consensus.cc:2468] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:31.028878 15295 raft_consensus.cc:3060] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.030109 15484 leader_election.cc:304] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:31.028805 15444 raft_consensus.cc:2468] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:31.031078 15666 raft_consensus.cc:2804] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.030122 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9cf658f5d1340238f97e4d01fa8c07e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:31.032027 15367 raft_consensus.cc:2468] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:31.031481 15666 raft_consensus.cc:493] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.033730 15666 raft_consensus.cc:3060] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.035392 15484 leader_election.cc:304] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:31.038105 15659 consensus_queue.cc:1048] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:31.038105 15662 raft_consensus.cc:2804] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.042466 15662 raft_consensus.cc:493] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.042268 15605 consensus_queue.cc:1048] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:31.043125 15662 raft_consensus.cc:3060] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.049665 15295 raft_consensus.cc:2468] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:30.993724 15192 catalog_manager.cc:5697] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:31.064261 15568 tablet_bootstrap.cc:654] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.071842 15662 raft_consensus.cc:515] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.083304 15563 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:31.083986 15563 ts_tablet_manager.cc:1403] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.144s	user 0.021s	sys 0.013s
I20260709 13:21:31.087342 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f48e3fa27c4c6487669b0d45130af7" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:31.088014 15295 raft_consensus.cc:2468] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:31.074415 15666 raft_consensus.cc:515] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.091320 15659 raft_consensus.cc:493] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.100584 15666 leader_election.cc:290] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.100677 15659 raft_consensus.cc:515] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.102456 15562 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:31.102952 15562 ts_tablet_manager.cc:1403] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.001s
I20260709 13:21:31.105572 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9cf658f5d1340238f97e4d01fa8c07e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:31.105665 15562 raft_consensus.cc:359] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.108804 15562 raft_consensus.cc:385] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.109081 15562 raft_consensus.cc:740] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.114117 15659 leader_election.cc:290] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.117702 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f585eec076224b80b0b6d6447d292eeb" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:31.118391 15441 raft_consensus.cc:2468] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:31.118772 15659 consensus_queue.cc:1048] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:31.121414 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f585eec076224b80b0b6d6447d292eeb" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:31.121654 15563 raft_consensus.cc:359] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.122339 15563 raft_consensus.cc:385] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.122740 15563 raft_consensus.cc:740] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.124476 15563 consensus_queue.cc:260] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.126288 15568 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:31.126722 15443 raft_consensus.cc:3060] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.126825 15568 ts_tablet_manager.cc:1403] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.148s	user 0.016s	sys 0.006s
I20260709 13:21:31.129743 15568 raft_consensus.cc:359] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.130352 15568 raft_consensus.cc:385] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.130609 15568 raft_consensus.cc:740] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.126482 15563 ts_tablet_manager.cc:1434] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:21:31.133908 15568 consensus_queue.cc:260] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.131184 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9cf658f5d1340238f97e4d01fa8c07e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:31.135999 15568 ts_tablet_manager.cc:1434] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:21:31.136091 15368 raft_consensus.cc:3060] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.136904 15568 tablet_bootstrap.cc:492] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
W20260709 13:21:31.139451 15481 leader_election.cc:343] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.143491 15568 tablet_bootstrap.cc:654] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.143563 15484 leader_election.cc:304] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:31.145139 15620 raft_consensus.cc:2804] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.145526 15620 raft_consensus.cc:493] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.145836 15620 raft_consensus.cc:3060] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.153530 15562 consensus_queue.cc:260] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.152065 15666 raft_consensus.cc:493] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.158130 15666 raft_consensus.cc:515] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.161578 15563 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:31.174691 15765 raft_consensus.cc:493] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.175441 15765 raft_consensus.cc:515] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.178164 15765 leader_election.cc:290] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.182283 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c124e9e01448409d9a64892dddf7c610" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:31.183324 15441 raft_consensus.cc:2468] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:31.185458 15336 leader_election.cc:304] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:31.186697 15768 raft_consensus.cc:2804] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.187163 15768 raft_consensus.cc:493] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.187315 15666 leader_election.cc:290] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:31.187510 15768 raft_consensus.cc:3060] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.189339 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:31.200320 15295 raft_consensus.cc:3060] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:31.200687 15481 leader_election.cc:343] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.201862 15765 consensus_queue.cc:1048] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:31.204563 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f48e3fa27c4c6487669b0d45130af7" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:31.205436 15444 raft_consensus.cc:3060] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.196074 15562 ts_tablet_manager.cc:1434] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.093s	user 0.005s	sys 0.003s
I20260709 13:21:31.210233 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c124e9e01448409d9a64892dddf7c610" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:31.183185 15788 log.cc:927] Time spent T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:21:31.215608 15562 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:31.218004 15193 catalog_manager.cc:5697] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:31.182101 15368 raft_consensus.cc:2468] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:31.219936 15443 raft_consensus.cc:2468] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:31.237742 15605 raft_consensus.cc:493] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.244503 15564 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:31.247233 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:31.247570 15605 raft_consensus.cc:515] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.248116 15564 ts_tablet_manager.cc:1403] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.260s	user 0.041s	sys 0.044s
I20260709 13:21:31.249751 15605 leader_election.cc:290] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.250561 15709 raft_consensus.cc:493] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.251127 15709 raft_consensus.cc:515] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.251214 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f133a6db45994178b473a5b1809d367f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:31.253280 15709 leader_election.cc:290] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:31.255345 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:31.256095 15516 raft_consensus.cc:2468] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:31.256331 15564 raft_consensus.cc:359] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.257296 15564 raft_consensus.cc:385] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.257648 15564 raft_consensus.cc:740] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.257697 15407 leader_election.cc:304] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:31.258750 15709 raft_consensus.cc:2804] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.259231 15709 raft_consensus.cc:493] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.259616 15709 raft_consensus.cc:3060] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.260161 15568 tablet_bootstrap.cc:492] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:31.260663 15568 ts_tablet_manager.cc:1403] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.124s	user 0.039s	sys 0.036s
I20260709 13:21:31.228214 15662 leader_election.cc:290] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
W20260709 13:21:31.263718 15336 leader_election.cc:343] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.267517 15484 leader_election.cc:304] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:31.268101 15768 raft_consensus.cc:515] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.269817 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:31.258536 15564 consensus_queue.cc:260] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.275631 15562 tablet_bootstrap.cc:654] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.278875 15568 raft_consensus.cc:359] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.279520 15568 raft_consensus.cc:385] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.279358 15444 raft_consensus.cc:2468] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:31.279742 15568 raft_consensus.cc:740] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.283617 15568 consensus_queue.cc:260] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.241137 15620 raft_consensus.cc:515] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.287654 15568 ts_tablet_manager.cc:1434] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:21:31.289827 15564 ts_tablet_manager.cc:1434] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.003s
I20260709 13:21:31.282033 15793 raft_consensus.cc:2804] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:31.292744 15564 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
W20260709 13:21:31.288498 15410 leader_election.cc:343] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.295184 15768 leader_election.cc:290] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.295429 15620 leader_election.cc:290] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.282862 15765 consensus_queue.cc:1048] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:21:31.300804 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f133a6db45994178b473a5b1809d367f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:31.301364 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f585eec076224b80b0b6d6447d292eeb" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:31.302089 15367 raft_consensus.cc:3060] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.303853 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c124e9e01448409d9a64892dddf7c610" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:31.304531 15444 raft_consensus.cc:3060] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.309945 15367 raft_consensus.cc:2468] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
W20260709 13:21:31.310839 15484 leader_election.cc:343] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.311189 15481 leader_election.cc:304] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:31.312155 15659 raft_consensus.cc:2804] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:31.312635 15659 raft_consensus.cc:697] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:31.311409 15484 leader_election.cc:304] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:31.316397 15659 consensus_queue.cc:237] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.285382 15709 raft_consensus.cc:515] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.320937 15568 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:31.324980 15444 raft_consensus.cc:2468] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:31.325812 15662 raft_consensus.cc:2749] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:31.314956 15793 raft_consensus.cc:697] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:31.329909 15793 consensus_queue.cc:237] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.340433 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:31.341079 15516 raft_consensus.cc:3060] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.343415 15258 leader_election.cc:304] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:31.345000 15721 raft_consensus.cc:2749] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:31.345868 15484 leader_election.cc:304] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:31.346778 15662 raft_consensus.cc:2804] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:31.347448 15662 raft_consensus.cc:697] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:31.348584 15662 consensus_queue.cc:237] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.352164 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f585eec076224b80b0b6d6447d292eeb" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:31.352857 15441 raft_consensus.cc:3060] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.363874 15563 tablet_bootstrap.cc:654] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.376600 15709 leader_election.cc:290] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:31.377789 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67f48e3fa27c4c6487669b0d45130af7" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:31.378983 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:31.378504 15293 raft_consensus.cc:3060] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:31.386445 15410 leader_election.cc:343] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.393985 15336 leader_election.cc:304] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:31.394901 15768 raft_consensus.cc:2804] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:31.398667 15768 raft_consensus.cc:697] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:31.399523 15768 consensus_queue.cc:237] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.422609 15564 tablet_bootstrap.cc:654] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.410068 15293 raft_consensus.cc:2468] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:31.432612 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c124e9e01448409d9a64892dddf7c610" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:31.436256 15294 raft_consensus.cc:3060] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.438170 15563 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:31.438684 15563 ts_tablet_manager.cc:1403] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.307s	user 0.015s	sys 0.000s
I20260709 13:21:31.440657 15568 tablet_bootstrap.cc:654] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.444974 15294 raft_consensus.cc:2468] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:31.444694 15563 raft_consensus.cc:359] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.450803 15563 raft_consensus.cc:385] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.451146 15563 raft_consensus.cc:740] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.451833 15563 consensus_queue.cc:260] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.453433 15682 consensus_queue.cc:1048] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:31.450922 15562 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:31.455650 15562 ts_tablet_manager.cc:1403] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.241s	user 0.024s	sys 0.027s
I20260709 13:21:31.458423 15562 raft_consensus.cc:359] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.459128 15562 raft_consensus.cc:385] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.459647 15562 raft_consensus.cc:740] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.458874 15190 catalog_manager.cc:5697] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:31.459167 15765 consensus_queue.cc:1048] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:31.453378 15441 raft_consensus.cc:2468] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:31.460402 15562 consensus_queue.cc:260] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.463472 15516 raft_consensus.cc:2468] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:31.470265 15562 ts_tablet_manager.cc:1434] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:21:31.470681 15407 leader_election.cc:304] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:31.465521 15190 catalog_manager.cc:5697] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:31.471251 15562 tablet_bootstrap.cc:492] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:31.465981 15563 ts_tablet_manager.cc:1434] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 13:21:31.473395 15563 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:31.472821 15190 catalog_manager.cc:5697] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:31.476686 15664 raft_consensus.cc:2804] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:31.477221 15664 raft_consensus.cc:697] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:31.478060 15664 consensus_queue.cc:237] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.489766 15563 tablet_bootstrap.cc:654] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.489696 15189 catalog_manager.cc:5697] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:31.497285 15564 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:31.497996 15564 ts_tablet_manager.cc:1403] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.207s	user 0.012s	sys 0.000s
I20260709 13:21:31.502226 15711 raft_consensus.cc:493] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.502947 15711 raft_consensus.cc:515] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.506978 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:31.507439 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:31.509018 15711 leader_election.cc:290] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
W20260709 13:21:31.509222 15410 leader_election.cc:343] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:31.510390 15407 leader_election.cc:343] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.511053 15407 leader_election.cc:304] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:31.513062 15712 raft_consensus.cc:2749] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:31.502007 15564 raft_consensus.cc:359] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.532341 15564 raft_consensus.cc:385] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.532749 15564 raft_consensus.cc:740] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.536695 15564 consensus_queue.cc:260] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.553470 15564 ts_tablet_manager.cc:1434] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.055s	user 0.012s	sys 0.000s
I20260709 13:21:31.554049 15190 catalog_manager.cc:5697] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:31.554950 15564 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:31.561725 15721 consensus_queue.cc:1048] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:31.563627 15605 consensus_queue.cc:1048] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:31.565676 15563 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:31.566206 15563 ts_tablet_manager.cc:1403] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.093s	user 0.018s	sys 0.026s
I20260709 13:21:31.577713 15568 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:31.577661 15563 raft_consensus.cc:359] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.578303 15568 ts_tablet_manager.cc:1403] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.260s	user 0.034s	sys 0.063s
I20260709 13:21:31.578397 15563 raft_consensus.cc:385] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.578769 15563 raft_consensus.cc:740] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.579528 15563 consensus_queue.cc:260] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.587456 15568 raft_consensus.cc:359] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.588227 15568 raft_consensus.cc:385] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.588495 15568 raft_consensus.cc:740] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.589290 15568 consensus_queue.cc:260] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.591101 15568 ts_tablet_manager.cc:1434] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:21:31.592199 15568 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:31.596019 15563 ts_tablet_manager.cc:1434] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:21:31.601190 15562 tablet_bootstrap.cc:654] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.603070 15563 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:31.621773 15564 tablet_bootstrap.cc:654] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.633771 15564 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:31.634378 15564 ts_tablet_manager.cc:1403] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.007s
I20260709 13:21:31.637501 15564 raft_consensus.cc:359] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.638343 15564 raft_consensus.cc:385] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.638720 15564 raft_consensus.cc:740] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.639448 15564 consensus_queue.cc:260] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.641240 15563 tablet_bootstrap.cc:654] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.642691 15564 ts_tablet_manager.cc:1434] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:21:31.643738 15564 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:31.651610 15564 tablet_bootstrap.cc:654] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.673213 15568 tablet_bootstrap.cc:654] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.686682 15562 tablet_bootstrap.cc:492] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:31.687446 15562 ts_tablet_manager.cc:1403] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.216s	user 0.012s	sys 0.003s
I20260709 13:21:31.689675 15664 raft_consensus.cc:493] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.690398 15664 raft_consensus.cc:515] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.694275 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:31.695217 15516 raft_consensus.cc:2468] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:31.696926 15407 leader_election.cc:304] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:31.698038 15712 raft_consensus.cc:2804] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.698644 15712 raft_consensus.cc:493] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.699102 15712 raft_consensus.cc:3060] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.701110 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:31.703006 15410 leader_election.cc:343] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.704047 15664 leader_election.cc:290] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:31.708819 15562 raft_consensus.cc:359] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.709545 15562 raft_consensus.cc:385] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.709810 15562 raft_consensus.cc:740] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.712980 15563 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:31.713533 15563 ts_tablet_manager.cc:1403] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.111s	user 0.034s	sys 0.034s
I20260709 13:21:31.716267 15563 raft_consensus.cc:359] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.716909 15563 raft_consensus.cc:385] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.717192 15563 raft_consensus.cc:740] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.717835 15563 consensus_queue.cc:260] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.720232 15563 ts_tablet_manager.cc:1434] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:21:31.721485 15563 tablet_bootstrap.cc:492] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:31.729911 15563 tablet_bootstrap.cc:654] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.740914 15712 raft_consensus.cc:515] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.742843 15563 tablet_bootstrap.cc:492] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:31.743367 15712 leader_election.cc:290] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:31.744587 15563 ts_tablet_manager.cc:1403] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.005s
I20260709 13:21:31.746723 15564 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:31.746855 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:31.747401 15564 ts_tablet_manager.cc:1403] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.104s	user 0.039s	sys 0.047s
W20260709 13:21:31.748426 15410 leader_election.cc:343] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.747411 15563 raft_consensus.cc:359] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.749056 15563 raft_consensus.cc:385] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.749372 15563 raft_consensus.cc:740] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.750031 15563 consensus_queue.cc:260] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.752296 15563 ts_tablet_manager.cc:1434] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:21:31.753393 15662 consensus_queue.cc:1048] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:31.753913 15563 tablet_bootstrap.cc:492] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:31.750542 15564 raft_consensus.cc:359] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.754998 15564 raft_consensus.cc:385] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.755293 15564 raft_consensus.cc:740] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.756023 15564 consensus_queue.cc:260] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.758376 15564 ts_tablet_manager.cc:1434] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:21:31.759317 15564 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:31.761374 15563 tablet_bootstrap.cc:654] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.763410 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:31.764086 15516 raft_consensus.cc:3060] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.764969 15564 tablet_bootstrap.cc:654] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.775672 15563 tablet_bootstrap.cc:492] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:31.776175 15563 ts_tablet_manager.cc:1403] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.007s
I20260709 13:21:31.778703 15563 raft_consensus.cc:359] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.779431 15563 raft_consensus.cc:385] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.779708 15563 raft_consensus.cc:740] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.780815 15568 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:31.781419 15568 ts_tablet_manager.cc:1403] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.190s	user 0.011s	sys 0.000s
I20260709 13:21:31.780855 15563 consensus_queue.cc:260] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.711796 15562 consensus_queue.cc:260] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:31.784122 15568 raft_consensus.cc:359] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.784897 15568 raft_consensus.cc:385] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.785187 15568 raft_consensus.cc:740] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.785905 15662 consensus_queue.cc:1048] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:31.786830 15563 ts_tablet_manager.cc:1434] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:21:31.786881 15568 consensus_queue.cc:260] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.788270 15563 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:31.789748 15568 ts_tablet_manager.cc:1434] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:21:31.790740 15568 tablet_bootstrap.cc:492] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:31.795073 15562 ts_tablet_manager.cc:1434] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.107s	user 0.006s	sys 0.002s
I20260709 13:21:31.796110 15562 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:31.798848 15563 tablet_bootstrap.cc:654] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.803751 15568 tablet_bootstrap.cc:654] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.813344 15564 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:31.813469 15563 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:31.814008 15563 ts_tablet_manager.cc:1403] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.007s
I20260709 13:21:31.813860 15564 ts_tablet_manager.cc:1403] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.018s
I20260709 13:21:31.816887 15563 raft_consensus.cc:359] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.817159 15564 raft_consensus.cc:359] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.817763 15563 raft_consensus.cc:385] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.818130 15563 raft_consensus.cc:740] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.817929 15564 raft_consensus.cc:385] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.818977 15564 raft_consensus.cc:740] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.818773 15563 consensus_queue.cc:260] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.819670 15564 consensus_queue.cc:260] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.820986 15563 ts_tablet_manager.cc:1434] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:31.821187 15564 ts_tablet_manager.cc:1434] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:21:31.822000 15563 tablet_bootstrap.cc:492] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:31.822175 15564 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:31.823302 15756 raft_consensus.cc:493] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.823848 15756 raft_consensus.cc:515] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.826193 15756 leader_election.cc:290] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.828579 15563 tablet_bootstrap.cc:654] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.833432 15564 tablet_bootstrap.cc:654] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.842468 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:31.858408 15441 raft_consensus.cc:2468] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:31.860389 15261 leader_election.cc:304] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:31.861647 15756 raft_consensus.cc:2804] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.862190 15756 raft_consensus.cc:493] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.863217 15756 raft_consensus.cc:3060] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.855656 15662 consensus_queue.cc:1048] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:21:31.872572 15564 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:31.873109 15564 ts_tablet_manager.cc:1403] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.000s
I20260709 13:21:31.877472 15564 raft_consensus.cc:359] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.878180 15564 raft_consensus.cc:385] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.878446 15564 raft_consensus.cc:740] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.879171 15564 consensus_queue.cc:260] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.890059 15564 ts_tablet_manager.cc:1434] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:21:31.891115 15564 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:31.897099 15564 tablet_bootstrap.cc:654] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.904088 15682 consensus_queue.cc:1048] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:31.851786 15793 consensus_queue.cc:1048] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:31.857403 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:31.914939 15605 raft_consensus.cc:493] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.915529 15605 raft_consensus.cc:515] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:31.917176 15757 consensus_queue.cc:1048] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:31.918041 15605 leader_election.cc:290] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
W20260709 13:21:31.922572 15258 leader_election.cc:343] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:31.926112 15562 tablet_bootstrap.cc:654] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.927665 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:31.933548 15605 raft_consensus.cc:493] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.934682 15605 raft_consensus.cc:515] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:31.928618 15368 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:31.939949 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:31.941318 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:31.945418 15518 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:31.943130 15605 leader_election.cc:290] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:31.947041 15756 raft_consensus.cc:515] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:31.965214 15258 leader_election.cc:304] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:31.904726 15516 raft_consensus.cc:2468] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:31.949211 15756 leader_election.cc:290] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:31.970047 15623 raft_consensus.cc:2804] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:31.975297 15623 raft_consensus.cc:493] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.975757 15623 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.979302 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:31.980260 15443 raft_consensus.cc:3060] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:31.967178 15682 raft_consensus.cc:493] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.968788 15662 raft_consensus.cc:493] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.995376 15756 raft_consensus.cc:493] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.989519 15768 consensus_queue.cc:1048] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:21:32.011854 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.012656 15516 raft_consensus.cc:2393] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:32.014991 15623 raft_consensus.cc:515] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.016101 15564 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:32.016778 15564 ts_tablet_manager.cc:1403] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.126s	user 0.024s	sys 0.009s
I20260709 13:21:32.007467 15407 leader_election.cc:304] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:32.019322 15564 raft_consensus.cc:359] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.019685 15711 raft_consensus.cc:2804] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.020010 15564 raft_consensus.cc:385] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.020427 15564 raft_consensus.cc:740] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.021425 15564 consensus_queue.cc:260] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.022272 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:31.992130 15682 raft_consensus.cc:515] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.023548 15564 ts_tablet_manager.cc:1434] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:21:32.024099 15664 raft_consensus.cc:493] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.024497 15564 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:32.024780 15682 leader_election.cc:290] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:32.026963 15293 raft_consensus.cc:3060] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.028096 15623 leader_election.cc:290] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:31.994197 15662 raft_consensus.cc:515] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.028096 15712 consensus_queue.cc:1048] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:21:32.032843 15564 tablet_bootstrap.cc:654] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.032938 15662 leader_election.cc:290] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:32.034722 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:32.039762 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 72864.2us
I20260709 13:21:32.040407 15721 raft_consensus.cc:493] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.040416 15711 raft_consensus.cc:697] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:32.041668 15721 raft_consensus.cc:515] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:32.041968 15333 leader_election.cc:343] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.042120 15711 consensus_queue.cc:237] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.042810 15563 tablet_bootstrap.cc:492] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:32.043360 15563 ts_tablet_manager.cc:1403] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.222s	user 0.036s	sys 0.050s
I20260709 13:21:32.043782 15721 leader_election.cc:290] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:32.044468 15258 leader_election.cc:304] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:32.045135 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100da53293964979b47f6fe528856727" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:32.045936 15563 raft_consensus.cc:359] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
W20260709 13:21:32.046671 15481 leader_election.cc:343] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.047748 15563 raft_consensus.cc:385] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.048241 15563 raft_consensus.cc:740] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.050001 15563 consensus_queue.cc:260] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.024549 15664 raft_consensus.cc:515] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.060240 15664 leader_election.cc:290] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:32.062042 15562 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:32.062562 15562 ts_tablet_manager.cc:1403] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.267s	user 0.020s	sys 0.027s
I20260709 13:21:32.062870 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:32.063546 15294 raft_consensus.cc:2468] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:32.063871 15443 raft_consensus.cc:2468] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:32.065174 15410 leader_election.cc:304] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:32.011323 15756 raft_consensus.cc:515] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.067302 15261 leader_election.cc:304] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:32.067746 15756 leader_election.cc:290] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:32.068360 15568 tablet_bootstrap.cc:492] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:32.062389 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:32.068823 15568 ts_tablet_manager.cc:1403] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.278s	user 0.054s	sys 0.031s
I20260709 13:21:32.068981 15518 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.069345 15562 raft_consensus.cc:359] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.070328 15562 raft_consensus.cc:385] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.070766 15562 raft_consensus.cc:740] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.072283 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:32.070726 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100da53293964979b47f6fe528856727" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:32.075680 15441 raft_consensus.cc:2468] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:32.077695 15563 ts_tablet_manager.cc:1434] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:21:32.077602 15562 consensus_queue.cc:260] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.078858 15563 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:32.079591 15484 leader_election.cc:304] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:32.071610 15664 raft_consensus.cc:2804] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:32.084671 15562 ts_tablet_manager.cc:1434] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:21:32.087616 15568 raft_consensus.cc:359] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.088186 15568 raft_consensus.cc:385] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.088466 15568 raft_consensus.cc:740] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.089197 15568 consensus_queue.cc:260] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.080279 15756 raft_consensus.cc:2804] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.086486 15659 consensus_queue.cc:1048] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:21:32.084784 15664 raft_consensus.cc:493] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.091540 15664 raft_consensus.cc:3060] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.085937 15757 raft_consensus.cc:2804] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:32.092005 15757 raft_consensus.cc:493] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.092350 15757 raft_consensus.cc:3060] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:32.080154 15407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 57724.5us
I20260709 13:21:32.098956 15568 ts_tablet_manager.cc:1434] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:21:32.099786 15562 tablet_bootstrap.cc:492] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:32.101806 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:32.093748 15721 raft_consensus.cc:2749] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.118258 15568 tablet_bootstrap.cc:492] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:32.098840 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.099035 15756 raft_consensus.cc:697] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
W20260709 13:21:32.107769 15336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 78532.9us
W20260709 13:21:32.148182 15258 leader_election.cc:343] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.150862 15756 consensus_queue.cc:237] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.138746 15516 raft_consensus.cc:2468] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:32.169942 15562 tablet_bootstrap.cc:654] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.172436 15564 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:32.173691 15564 ts_tablet_manager.cc:1403] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.149s	user 0.021s	sys 0.002s
I20260709 13:21:32.151031 15712 raft_consensus.cc:493] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.180783 15712 raft_consensus.cc:515] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.189684 15712 leader_election.cc:290] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:32.190899 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:32.191589 15294 raft_consensus.cc:2468] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:32.165098 15711 consensus_queue.cc:1048] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:21:32.201789 15258 leader_election.cc:304] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:32.203188 15623 raft_consensus.cc:2804] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:32.203567 15623 raft_consensus.cc:493] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.203909 15623 raft_consensus.cc:3060] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:32.204203 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:32.209314 15711 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.210183 15836 raft_consensus.cc:493] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.210016 15711 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.210716 15836 raft_consensus.cc:515] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.212641 15711 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:32.212738 15836 leader_election.cc:290] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:32.216573 15292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2588887022d8411bba3692826b2fe934" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:32.219786 15407 leader_election.cc:343] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.222760 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:32.223603 15369 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.224543 15336 leader_election.cc:304] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:32.226584 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:32.227959 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:32.228689 15365 raft_consensus.cc:2468] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:32.229176 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.231477 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2588887022d8411bba3692826b2fe934" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:32.233052 15369 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:32.233719 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d81a8d43d2a34dc38f37e6c6c03c29c3" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.238878 15835 raft_consensus.cc:2804] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:32.239288 15835 raft_consensus.cc:493] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.239624 15835 raft_consensus.cc:3060] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.244717 15664 raft_consensus.cc:515] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.245994 15444 raft_consensus.cc:2393] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:32.248785 15564 raft_consensus.cc:359] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.249603 15564 raft_consensus.cc:385] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.249452 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d81a8d43d2a34dc38f37e6c6c03c29c3" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:32.250185 15367 raft_consensus.cc:2468] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:32.254189 15518 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:32.259359 15568 tablet_bootstrap.cc:654] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.261515 15258 leader_election.cc:304] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:32.263568 15721 raft_consensus.cc:493] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.264038 15721 raft_consensus.cc:515] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.266032 15721 leader_election.cc:290] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
W20260709 13:21:32.270521 15258 leader_election.cc:343] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.249930 15564 raft_consensus.cc:740] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.200853 15563 tablet_bootstrap.cc:654] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.275565 15564 consensus_queue.cc:260] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.282503 15757 raft_consensus.cc:515] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.297053 15564 ts_tablet_manager.cc:1434] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.123s	user 0.006s	sys 0.001s
I20260709 13:21:32.298319 15564 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:32.266541 15721 raft_consensus.cc:2804] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.280730 15712 raft_consensus.cc:493] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:32.312925 15407 leader_election.cc:343] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:32.315653 15407 leader_election.cc:343] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:32.317371 15407 leader_election.cc:343] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.317963 15407 leader_election.cc:304] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:32.291841 15292 raft_consensus.cc:2468] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:32.313524 15294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:32.315377 15712 raft_consensus.cc:515] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.323566 15712 leader_election.cc:290] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:32.324298 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:32.319540 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
W20260709 13:21:32.325917 15407 leader_election.cc:343] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.326879 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.327627 15518 raft_consensus.cc:2393] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:32.326066 15193 catalog_manager.cc:5697] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:32.314196 15721 raft_consensus.cc:697] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:32.351719 15410 leader_election.cc:304] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:32.369154 15664 leader_election.cc:290] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:32.388415 15841 raft_consensus.cc:2804] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:32.388882 15841 raft_consensus.cc:493] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.389227 15841 raft_consensus.cc:3060] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.389422 15562 tablet_bootstrap.cc:492] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:32.389921 15562 ts_tablet_manager.cc:1403] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.290s	user 0.019s	sys 0.009s
W20260709 13:21:32.390018 15410 leader_election.cc:343] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.390753 15410 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:32.373240 15721 consensus_queue.cc:237] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.392753 15562 raft_consensus.cc:359] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.393455 15562 raft_consensus.cc:385] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.393744 15562 raft_consensus.cc:740] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.395113 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:32.376014 15623 raft_consensus.cc:515] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.394414 15562 consensus_queue.cc:260] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
W20260709 13:21:32.385411 15261 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 101864us
I20260709 13:21:32.383567 15712 raft_consensus.cc:2804] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:21:32.406390 15407 leader_election.cc:343] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.405953 15834 raft_consensus.cc:2749] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.402550 15623 leader_election.cc:290] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:32.421398 15757 leader_election.cc:290] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:32.427737 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.429210 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:32.429705 15564 tablet_bootstrap.cc:654] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.413723 15712 raft_consensus.cc:493] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:32.435889 15407 leader_election.cc:343] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.436498 15407 leader_election.cc:304] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:32.398131 15835 raft_consensus.cc:515] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.423842 15568 tablet_bootstrap.cc:492] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:32.433178 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:32.438344 15711 raft_consensus.cc:2749] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.439147 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100da53293964979b47f6fe528856727" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:32.439937 15712 raft_consensus.cc:3060] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.444976 15261 leader_election.cc:304] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:32.447475 15568 ts_tablet_manager.cc:1403] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.330s	user 0.010s	sys 0.017s
W20260709 13:21:32.448436 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 62523.5us
W20260709 13:21:32.459486 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 54662.4us
I20260709 13:21:32.462072 15562 ts_tablet_manager.cc:1434] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.072s	user 0.003s	sys 0.012s
I20260709 13:21:32.466857 15562 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:32.467082 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100da53293964979b47f6fe528856727" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:32.467703 15441 raft_consensus.cc:3060] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.468659 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:32.469286 15516 raft_consensus.cc:3060] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.479801 15368 raft_consensus.cc:2393] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:32.480326 15568 raft_consensus.cc:359] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.481047 15568 raft_consensus.cc:385] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.481331 15568 raft_consensus.cc:740] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.459193 15293 raft_consensus.cc:3060] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.512893 15261 leader_election.cc:304] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
W20260709 13:21:32.513334 15481 leader_election.cc:343] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.520217 15756 raft_consensus.cc:3060] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.525772 15712 raft_consensus.cc:515] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:32.533432 15407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 53364.8us
I20260709 13:21:32.554709 15563 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:32.555240 15563 ts_tablet_manager.cc:1403] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.477s	user 0.014s	sys 0.007s
I20260709 13:21:32.555552 15516 raft_consensus.cc:2468] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:32.561686 15258 leader_election.cc:304] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:32.562985 15563 raft_consensus.cc:359] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.563625 15563 raft_consensus.cc:385] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.563798 15563 raft_consensus.cc:740] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.564960 15623 raft_consensus.cc:2804] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.566336 15623 raft_consensus.cc:697] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:32.567185 15623 consensus_queue.cc:237] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.568760 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2588887022d8411bba3692826b2fe934" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:32.569402 15291 raft_consensus.cc:3060] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.570677 15835 leader_election.cc:290] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:32.571116 15756 raft_consensus.cc:2749] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.573244 15568 consensus_queue.cc:260] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.577083 15712 leader_election.cc:290] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:32.579113 15568 ts_tablet_manager.cc:1434] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.122s	user 0.009s	sys 0.001s
I20260709 13:21:32.579676 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2588887022d8411bba3692826b2fe934" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:32.585105 15568 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:32.587146 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:32.588210 15189 catalog_manager.cc:5697] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:32.590876 15568 tablet_bootstrap.cc:654] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:32.592166 15407 leader_election.cc:343] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.593673 15293 raft_consensus.cc:2468] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:32.594225 15721 raft_consensus.cc:2749] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.596487 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:32.597311 15517 raft_consensus.cc:2393] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:32.600227 15410 leader_election.cc:304] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:32.600679 15563 consensus_queue.cc:260] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.601467 15712 raft_consensus.cc:2804] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.602042 15712 raft_consensus.cc:697] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:32.603000 15712 consensus_queue.cc:237] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.603444 15841 raft_consensus.cc:515] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.606479 15841 leader_election.cc:290] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:32.614085 15562 tablet_bootstrap.cc:654] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.614285 15563 ts_tablet_manager.cc:1434] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.001s
I20260709 13:21:32.615800 15563 tablet_bootstrap.cc:492] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:32.617944 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d81a8d43d2a34dc38f37e6c6c03c29c3" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:32.618665 15517 raft_consensus.cc:3060] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.626107 15441 raft_consensus.cc:2468] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:32.627242 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d81a8d43d2a34dc38f37e6c6c03c29c3" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:32.626106 15291 raft_consensus.cc:2468] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:32.627940 15364 raft_consensus.cc:3060] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.629120 15484 leader_election.cc:304] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:32.632966 15855 raft_consensus.cc:2804] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.633178 15517 raft_consensus.cc:2468] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:32.633497 15855 raft_consensus.cc:697] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:32.634665 15855 consensus_queue.cc:237] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.636210 15333 leader_election.cc:304] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:32.641399 15407 leader_election.cc:304] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:32.649395 15410 leader_election.cc:304] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:32.649896 15841 raft_consensus.cc:2804] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.650480 15841 raft_consensus.cc:697] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:32.650926 15858 raft_consensus.cc:2804] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:32.651538 15858 raft_consensus.cc:697] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:32.651367 15841 consensus_queue.cc:237] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.652762 15858 consensus_queue.cc:237] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.650499 15834 consensus_queue.cc:1048] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:32.655616 15192 catalog_manager.cc:5697] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:32.658775 15768 raft_consensus.cc:2749] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.659654 15664 consensus_queue.cc:1048] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:32.660018 15682 raft_consensus.cc:493] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.660530 15682 raft_consensus.cc:515] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.663280 15682 leader_election.cc:290] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:32.677522 15563 tablet_bootstrap.cc:654] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.677762 15568 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:32.678282 15568 ts_tablet_manager.cc:1403] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.095s	user 0.010s	sys 0.005s
I20260709 13:21:32.681475 15568 raft_consensus.cc:359] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.682286 15568 raft_consensus.cc:385] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.682587 15568 raft_consensus.cc:740] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.683351 15568 consensus_queue.cc:260] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.691704 15568 ts_tablet_manager.cc:1434] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:21:32.707160 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100da53293964979b47f6fe528856727" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:32.708122 15441 raft_consensus.cc:2393] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:32.709889 15568 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:32.712013 15364 raft_consensus.cc:2468] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:32.713006 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "100da53293964979b47f6fe528856727" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.713377 15564 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:32.713858 15564 ts_tablet_manager.cc:1403] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.416s	user 0.033s	sys 0.022s
I20260709 13:21:32.717563 15336 leader_election.cc:304] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:32.718504 15835 raft_consensus.cc:3060] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.764832 15562 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:32.777588 15562 ts_tablet_manager.cc:1403] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.311s	user 0.009s	sys 0.019s
I20260709 13:21:32.768388 15193 catalog_manager.cc:5697] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:32.728488 15854 consensus_queue.cc:1048] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260709 13:21:32.780371 15562 raft_consensus.cc:359] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
W20260709 13:21:32.782215 15258 consensus_peers.cc:597] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:32.762662 15568 tablet_bootstrap.cc:654] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.781878 15562 raft_consensus.cc:385] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.785452 15562 raft_consensus.cc:740] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.786240 15562 consensus_queue.cc:260] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.736115 15835 raft_consensus.cc:2749] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.788534 15562 ts_tablet_manager.cc:1434] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:32.764734 15563 tablet_bootstrap.cc:492] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:32.790388 15563 ts_tablet_manager.cc:1403] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.175s	user 0.007s	sys 0.009s
I20260709 13:21:32.792348 15568 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:32.792918 15562 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:32.793012 15568 ts_tablet_manager.cc:1403] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.001s
I20260709 13:21:32.797366 15568 raft_consensus.cc:359] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.798024 15568 raft_consensus.cc:385] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.798302 15568 raft_consensus.cc:740] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.800148 15562 tablet_bootstrap.cc:654] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.800719 15568 consensus_queue.cc:260] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.802953 15568 ts_tablet_manager.cc:1434] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:21:32.803890 15568 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:32.810089 15562 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:32.811026 15562 ts_tablet_manager.cc:1403] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 13:21:32.811553 15568 tablet_bootstrap.cc:654] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.813632 15562 raft_consensus.cc:359] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.814338 15562 raft_consensus.cc:385] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.722981 15564 raft_consensus.cc:359] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.815271 15564 raft_consensus.cc:385] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.815533 15564 raft_consensus.cc:740] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.816218 15564 consensus_queue.cc:260] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.818157 15564 ts_tablet_manager.cc:1434] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.104s	user 0.004s	sys 0.000s
I20260709 13:21:32.772015 15192 catalog_manager.cc:5697] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:32.823616 15568 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:32.824296 15568 ts_tablet_manager.cc:1403] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.004s
I20260709 13:21:32.824529 15192 catalog_manager.cc:5697] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:32.814605 15562 raft_consensus.cc:740] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.826936 15568 raft_consensus.cc:359] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.827639 15568 raft_consensus.cc:385] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.829183 15568 raft_consensus.cc:740] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.826102 15564 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:32.829898 15568 consensus_queue.cc:260] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.822597 15563 raft_consensus.cc:359] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.830803 15563 raft_consensus.cc:385] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.831156 15563 raft_consensus.cc:740] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.827521 15562 consensus_queue.cc:260] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.833876 15562 ts_tablet_manager.cc:1434] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:21:32.882516 15562 tablet_bootstrap.cc:492] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:32.831938 15563 consensus_queue.cc:260] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.889189 15855 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.889786 15855 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.891757 15563 ts_tablet_manager.cc:1434] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.101s	user 0.012s	sys 0.000s
I20260709 13:21:32.891994 15568 ts_tablet_manager.cc:1434] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:21:32.892328 15721 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.897508 15721 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.898272 15568 tablet_bootstrap.cc:492] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:32.902276 15563 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:32.903277 15564 tablet_bootstrap.cc:654] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.904624 15568 tablet_bootstrap.cc:654] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.909026 15721 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:32.910261 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.910843 15563 tablet_bootstrap.cc:654] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.910990 15517 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:32.912374 15258 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:32.915251 15721 raft_consensus.cc:2804] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:32.916039 15721 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.916478 15721 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.917017 15564 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:32.917462 15563 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:32.917582 15564 ts_tablet_manager.cc:1403] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.005s
I20260709 13:21:32.917968 15563 ts_tablet_manager.cc:1403] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:21:32.918866 15568 tablet_bootstrap.cc:492] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:32.919479 15835 raft_consensus.cc:493] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.920009 15568 ts_tablet_manager.cc:1403] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.005s
I20260709 13:21:32.919977 15835 raft_consensus.cc:515] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.920758 15564 raft_consensus.cc:359] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.921447 15564 raft_consensus.cc:385] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.921758 15564 raft_consensus.cc:740] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.922535 15564 consensus_queue.cc:260] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:32.893309 15562 tablet_bootstrap.cc:654] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.922912 15568 raft_consensus.cc:359] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.923643 15568 raft_consensus.cc:385] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.923913 15568 raft_consensus.cc:740] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.924592 15568 consensus_queue.cc:260] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.924893 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:32.893469 15410 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 50247.2us
I20260709 13:21:32.925817 15291 raft_consensus.cc:2393] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:32.927194 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:32.927842 15443 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:32.926483 15563 raft_consensus.cc:359] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.893455 15189 catalog_manager.cc:5697] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:32.928849 15563 raft_consensus.cc:385] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.929085 15835 leader_election.cc:290] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:32.929199 15563 raft_consensus.cc:740] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.930112 15563 consensus_queue.cc:260] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.929967 15189 catalog_manager.cc:5697] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:32.930701 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f542e50c9876452b88a3f17ea981a62c" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:32.932309 15563 ts_tablet_manager.cc:1434] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:21:32.894959 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:32.895332 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:32.933557 15444 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:32.891835 15855 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:32.934166 15564 ts_tablet_manager.cc:1434] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:21:32.937856 15564 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:32.933169 15563 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:32.938264 15336 leader_election.cc:304] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:32.951241 15712 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.953327 15682 raft_consensus.cc:3060] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.953578 15712 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.958137 15563 tablet_bootstrap.cc:654] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.958910 15682 raft_consensus.cc:2749] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.959046 15712 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:32.960783 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:32.938045 15768 raft_consensus.cc:493] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.961915 15562 tablet_bootstrap.cc:492] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:32.961536 15768 raft_consensus.cc:515] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.962410 15562 ts_tablet_manager.cc:1403] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.005s
I20260709 13:21:32.963760 15768 leader_election.cc:290] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
W20260709 13:21:32.962489 15407 leader_election.cc:343] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.965617 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:32.966138 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:32.967072 15562 raft_consensus.cc:359] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.967257 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:32.967741 15562 raft_consensus.cc:385] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.968041 15562 raft_consensus.cc:740] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
W20260709 13:21:32.968969 15410 leader_election.cc:343] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.970643 15410 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:32.942735 15568 ts_tablet_manager.cc:1434] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:21:32.975063 15712 raft_consensus.cc:2749] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.975458 15721 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:32.976686 15289 raft_consensus.cc:2393] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:32.973984 15484 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:32.977497 15855 raft_consensus.cc:2804] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:32.977872 15855 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.978199 15855 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:32.968765 15562 consensus_queue.cc:260] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:32.981689 15562 ts_tablet_manager.cc:1434] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.001s
I20260709 13:21:32.982640 15568 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:32.972204 15336 leader_election.cc:304] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:32.986536 15721 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:32.976787 15563 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:32.987627 15563 ts_tablet_manager.cc:1403] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.001s
I20260709 13:21:32.988348 15562 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:32.892913 15848 consensus_queue.cc:1048] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:32.991811 15563 raft_consensus.cc:359] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.992673 15563 raft_consensus.cc:385] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.993018 15563 raft_consensus.cc:740] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.993767 15563 consensus_queue.cc:260] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:32.994143 15835 raft_consensus.cc:2749] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.994976 15568 tablet_bootstrap.cc:654] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.996201 15563 ts_tablet_manager.cc:1434] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:21:32.996721 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:32.997254 15563 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:32.997294 15444 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.002024 15562 tablet_bootstrap.cc:654] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.004608 15563 tablet_bootstrap.cc:654] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.024797 15848 consensus_queue.cc:1048] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:33.028309 15841 consensus_queue.cc:1048] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:33.027560 15848 consensus_queue.cc:1048] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:21:33.039158 15857 consensus_queue.cc:1048] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:33.047361 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:33.060758 15835 raft_consensus.cc:493] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.061463 15835 raft_consensus.cc:515] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.063903 15835 leader_election.cc:290] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:33.066165 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2588887022d8411bba3692826b2fe934" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.066982 15289 raft_consensus.cc:2393] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:33.068405 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2588887022d8411bba3692826b2fe934" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:33.072747 15835 raft_consensus.cc:493] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.073378 15835 raft_consensus.cc:515] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.076952 15835 leader_election.cc:290] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:33.082324 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:33.083081 15443 raft_consensus.cc:2468] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:33.091257 15851 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.091869 15851 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.094106 15851 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.095460 15841 raft_consensus.cc:493] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.096068 15841 raft_consensus.cc:515] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.098488 15841 leader_election.cc:290] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.100122 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.100309 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.100543 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
W20260709 13:21:33.102067 15410 leader_election.cc:343] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:33.103574 15481 leader_election.cc:343] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:33.105214 15407 leader_election.cc:343] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.105795 15407 leader_election.cc:304] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:33.109952 15861 raft_consensus.cc:2749] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.114652 15444 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:32.988377 15855 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.134689 15516 raft_consensus.cc:2393] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:33.137180 15855 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:33.119797 15841 consensus_queue.cc:1048] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:33.148957 15564 tablet_bootstrap.cc:654] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.152144 15364 raft_consensus.cc:3060] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.168269 15793 raft_consensus.cc:493] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.168830 15793 raft_consensus.cc:515] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.171175 15793 leader_election.cc:290] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:33.175024 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.175770 15366 raft_consensus.cc:2468] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:33.176682 15721 raft_consensus.cc:493] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.177215 15721 raft_consensus.cc:515] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.082213 15336 leader_election.cc:304] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:33.129155 15854 consensus_queue.cc:1048] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:33.180445 15336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 98300.6us
I20260709 13:21:33.181214 15721 leader_election.cc:290] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.184653 15261 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:33.187335 15721 raft_consensus.cc:2804] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.188074 15721 raft_consensus.cc:697] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:33.188970 15721 consensus_queue.cc:237] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.190575 15664 raft_consensus.cc:493] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.193660 15481 leader_election.cc:304] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:33.195148 15664 raft_consensus.cc:515] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.195520 15793 raft_consensus.cc:2804] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.196297 15793 raft_consensus.cc:493] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.196628 15793 raft_consensus.cc:3060] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.197227 15664 leader_election.cc:290] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:33.204396 15562 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:33.205042 15562 ts_tablet_manager.cc:1403] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.217s	user 0.020s	sys 0.018s
I20260709 13:21:33.207993 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.208709 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.209136 15562 raft_consensus.cc:359] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.209884 15562 raft_consensus.cc:385] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.210150 15562 raft_consensus.cc:740] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.210937 15562 consensus_queue.cc:260] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.212942 15562 ts_tablet_manager.cc:1434] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:21:33.213863 15562 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:33.217464 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.220005 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
W20260709 13:21:33.220216 15407 leader_election.cc:343] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.220777 15444 raft_consensus.cc:2393] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:33.222889 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:33.228623 15562 tablet_bootstrap.cc:654] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.229585 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.232383 15568 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:33.233001 15568 ts_tablet_manager.cc:1403] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.251s	user 0.017s	sys 0.034s
I20260709 13:21:33.237428 15568 raft_consensus.cc:359] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.238085 15568 raft_consensus.cc:385] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.238333 15568 raft_consensus.cc:740] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.239122 15568 consensus_queue.cc:260] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.243209 15563 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:33.243814 15563 ts_tablet_manager.cc:1403] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.247s	user 0.062s	sys 0.095s
I20260709 13:21:33.244515 15564 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:33.244989 15564 ts_tablet_manager.cc:1403] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.310s	user 0.011s	sys 0.000s
I20260709 13:21:33.244982 15841 consensus_queue.cc:1048] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:33.246539 15563 raft_consensus.cc:359] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.247241 15563 raft_consensus.cc:385] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.247493 15563 raft_consensus.cc:740] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.248538 15568 ts_tablet_manager.cc:1434] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:21:33.249470 15568 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:33.249509 15835 raft_consensus.cc:2749] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.253841 15841 consensus_queue.cc:1048] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:33.249810 15564 raft_consensus.cc:359] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.254945 15564 raft_consensus.cc:385] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.255277 15564 raft_consensus.cc:740] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.258059 15568 tablet_bootstrap.cc:654] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.249867 15563 consensus_queue.cc:260] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.262173 15563 ts_tablet_manager.cc:1434] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:21:33.262842 15564 consensus_queue.cc:260] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.263335 15563 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:33.265264 15564 ts_tablet_manager.cc:1434] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:21:33.266211 15564 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
W20260709 13:21:33.266314 15407 leader_election.cc:343] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.266908 15407 leader_election.cc:304] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:33.267769 15841 raft_consensus.cc:2749] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.269754 15563 tablet_bootstrap.cc:654] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.271870 15712 consensus_queue.cc:1048] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:33.272773 15564 tablet_bootstrap.cc:654] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.275280 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.276816 15336 leader_election.cc:304] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:33.277179 15861 consensus_queue.cc:1048] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:33.280592 15835 raft_consensus.cc:2804] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.280999 15835 raft_consensus.cc:493] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.281350 15835 raft_consensus.cc:3060] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.282387 15563 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:33.282918 15563 ts_tablet_manager.cc:1403] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.003s
I20260709 13:21:33.285651 15563 raft_consensus.cc:359] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.288456 15793 raft_consensus.cc:515] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.289884 15516 raft_consensus.cc:2393] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:33.286311 15563 raft_consensus.cc:385] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.290314 15563 raft_consensus.cc:740] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.291507 15793 leader_election.cc:290] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:33.291841 15563 consensus_queue.cc:260] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.293334 15564 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:33.293509 15563 ts_tablet_manager.cc:1434] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:21:33.293843 15564 ts_tablet_manager.cc:1403] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.002s
I20260709 13:21:33.296819 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:33.297473 15443 raft_consensus.cc:2468] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:33.297955 15564 raft_consensus.cc:359] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.298905 15564 raft_consensus.cc:385] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.299238 15564 raft_consensus.cc:740] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.300210 15564 consensus_queue.cc:260] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.302610 15563 tablet_bootstrap.cc:492] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:33.303133 15564 ts_tablet_manager.cc:1434] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:21:33.304029 15564 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:33.306072 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.308349 15563 tablet_bootstrap.cc:654] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.309662 15564 tablet_bootstrap.cc:654] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.314564 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
W20260709 13:21:33.318449 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:33.360987 15562 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:33.362977 15562 ts_tablet_manager.cc:1403] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.149s	user 0.016s	sys 0.002s
I20260709 13:21:33.374351 15563 tablet_bootstrap.cc:492] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:33.376292 15563 ts_tablet_manager.cc:1403] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.015s
W20260709 13:21:33.376920 15258 leader_election.cc:343] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.374756 15562 raft_consensus.cc:359] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.377496 15562 raft_consensus.cc:385] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.377918 15562 raft_consensus.cc:740] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.378069 15854 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.378577 15854 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.378825 15562 consensus_queue.cc:260] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.382196 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:33.382512 15564 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:33.383004 15443 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:33.383908 15562 ts_tablet_manager.cc:1434] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:21:33.384506 15261 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:33.384888 15562 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:33.385448 15721 raft_consensus.cc:2804] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.381498 15563 raft_consensus.cc:359] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.385862 15721 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.386866 15721 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.383273 15564 ts_tablet_manager.cc:1403] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.079s	user 0.037s	sys 0.020s
I20260709 13:21:33.380682 15854 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:33.395870 15564 raft_consensus.cc:359] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.386025 15563 raft_consensus.cc:385] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.396795 15563 raft_consensus.cc:740] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.396457 15721 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.402271 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:33.402863 15443 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.404150 15562 tablet_bootstrap.cc:654] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.397644 15563 consensus_queue.cc:260] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.413012 15861 raft_consensus.cc:493] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.413733 15861 raft_consensus.cc:515] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.415016 15443 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:33.396595 15564 raft_consensus.cc:385] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.420392 15564 raft_consensus.cc:740] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.421386 15564 consensus_queue.cc:260] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.424418 15564 ts_tablet_manager.cc:1434] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:21:33.417662 15563 ts_tablet_manager.cc:1434] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.003s
I20260709 13:21:33.418077 15721 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:33.425980 15261 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:33.426964 15563 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:33.428491 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:33.431682 15410 leader_election.cc:343] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.432438 15861 leader_election.cc:290] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.432890 15564 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:33.433288 15721 raft_consensus.cc:2804] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.434120 15721 raft_consensus.cc:697] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:33.435403 15721 consensus_queue.cc:237] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.441521 15564 tablet_bootstrap.cc:654] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.442286 15851 consensus_queue.cc:1048] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:33.446732 15563 tablet_bootstrap.cc:654] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.459100 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.459546 15563 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:33.460215 15563 ts_tablet_manager.cc:1403] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.002s
I20260709 13:21:33.465538 15367 raft_consensus.cc:3060] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.469430 15563 raft_consensus.cc:359] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.470397 15563 raft_consensus.cc:385] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.470734 15563 raft_consensus.cc:740] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.375347 15191 catalog_manager.cc:5697] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:33.473843 15564 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:33.474509 15564 ts_tablet_manager.cc:1403] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.000s
I20260709 13:21:33.479780 15564 raft_consensus.cc:359] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.480687 15564 raft_consensus.cc:385] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.481038 15564 raft_consensus.cc:740] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.471706 15563 consensus_queue.cc:260] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:33.483469 15484 leader_election.cc:343] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.484169 15568 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:33.484102 15484 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:33.484690 15568 ts_tablet_manager.cc:1403] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.235s	user 0.043s	sys 0.078s
I20260709 13:21:33.485186 15563 ts_tablet_manager.cc:1434] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:21:33.487090 15712 raft_consensus.cc:493] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.487498 15793 raft_consensus.cc:2749] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.487646 15712 raft_consensus.cc:515] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.491544 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.491442 15712 leader_election.cc:290] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:33.492576 15563 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
W20260709 13:21:33.493072 15407 leader_election.cc:343] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.493643 15407 leader_election.cc:304] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
W20260709 13:21:33.496172 15410 leader_election.cc:343] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.496968 15841 raft_consensus.cc:2749] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.494379 15568 raft_consensus.cc:359] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.497689 15568 raft_consensus.cc:385] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.497960 15568 raft_consensus.cc:740] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.500885 15515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.501734 15564 consensus_queue.cc:260] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:33.502410 15407 leader_election.cc:343] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.503005 15407 leader_election.cc:304] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:33.500002 15568 consensus_queue.cc:260] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.505051 15861 raft_consensus.cc:2749] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.507180 15568 ts_tablet_manager.cc:1434] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:21:33.508214 15568 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:33.510212 15564 ts_tablet_manager.cc:1434] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:21:33.511590 15564 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:33.517822 15563 tablet_bootstrap.cc:654] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.522002 15568 tablet_bootstrap.cc:654] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.523458 15564 tablet_bootstrap.cc:654] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.529269 15835 raft_consensus.cc:515] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.532934 15366 raft_consensus.cc:2393] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:33.534904 15841 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.535624 15841 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.536962 15563 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:33.537350 15568 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:33.537756 15563 ts_tablet_manager.cc:1403] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.000s
I20260709 13:21:33.540303 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:33.542243 15410 leader_election.cc:343] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.542913 15563 raft_consensus.cc:359] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.543555 15848 raft_consensus.cc:493] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.547293 15768 raft_consensus.cc:493] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.548797 15848 raft_consensus.cc:515] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.548892 15768 raft_consensus.cc:515] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.552527 15768 leader_election.cc:290] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:33.548215 15563 raft_consensus.cc:385] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.553821 15563 raft_consensus.cc:740] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.554020 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:33.563076 15443 raft_consensus.cc:2468] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:33.556077 15712 raft_consensus.cc:493] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.564075 15712 raft_consensus.cc:515] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.537848 15841 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:33.489352 15484 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:33.565796 15793 raft_consensus.cc:2749] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.564338 15563 consensus_queue.cc:260] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.570839 15261 leader_election.cc:304] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:33.537873 15568 ts_tablet_manager.cc:1403] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:21:33.570248 15562 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:33.572069 15562 ts_tablet_manager.cc:1403] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.187s	user 0.085s	sys 0.065s
I20260709 13:21:33.572773 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.573520 15563 ts_tablet_manager.cc:1434] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:21:33.566323 15712 leader_election.cc:290] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.566506 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.575234 15562 raft_consensus.cc:359] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.575799 15364 raft_consensus.cc:2468] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:33.567167 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:33.577152 15443 raft_consensus.cc:3060] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.568446 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed4922079864d94a4a771fb9466d63c" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.578260 15291 raft_consensus.cc:2468] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
W20260709 13:21:33.580574 15410 leader_election.cc:343] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.574472 15563 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:33.569229 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.575577 15623 raft_consensus.cc:2804] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.582099 15623 raft_consensus.cc:493] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:33.582230 15407 leader_election.cc:343] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.557937 15848 leader_election.cc:290] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:33.582698 15407 leader_election.cc:304] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:33.582494 15623 raft_consensus.cc:3060] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.584124 15712 raft_consensus.cc:2749] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.584467 15443 raft_consensus.cc:2468] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:33.566183 15835 leader_election.cc:290] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:33.574164 15568 raft_consensus.cc:359] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.575986 15562 raft_consensus.cc:385] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.586805 15562 raft_consensus.cc:740] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.587076 15568 raft_consensus.cc:385] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.587458 15568 raft_consensus.cc:740] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.587831 15562 consensus_queue.cc:260] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.590039 15562 ts_tablet_manager.cc:1434] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 13:21:33.591022 15562 tablet_bootstrap.cc:492] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
W20260709 13:21:33.566087 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 76813.6us
I20260709 13:21:33.560488 15515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.593130 15336 leader_election.cc:304] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:33.594095 15912 raft_consensus.cc:2804] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:21:33.594367 15407 leader_election.cc:343] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.594532 15912 raft_consensus.cc:493] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.594961 15912 raft_consensus.cc:3060] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.597355 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:33.597179 15568 consensus_queue.cc:260] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.598062 15443 raft_consensus.cc:2393] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:33.594901 15407 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:33.598840 15568 ts_tablet_manager.cc:1434] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:21:33.599907 15841 raft_consensus.cc:2749] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.602312 15336 leader_election.cc:304] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
W20260709 13:21:33.602548 15258 leader_election.cc:343] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.602993 15258 leader_election.cc:304] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:33.603570 15568 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:33.603681 15721 raft_consensus.cc:2749] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.607604 15835 raft_consensus.cc:2804] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.608131 15835 raft_consensus.cc:697] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:33.608934 15835 consensus_queue.cc:237] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.609901 15912 raft_consensus.cc:515] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.613214 15562 tablet_bootstrap.cc:654] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.615957 15564 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:33.616475 15564 ts_tablet_manager.cc:1403] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.003s
I20260709 13:21:33.617687 15912 leader_election.cc:290] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:33.619720 15568 tablet_bootstrap.cc:654] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.620393 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed4922079864d94a4a771fb9466d63c" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:33.621274 15291 raft_consensus.cc:3060] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.625645 15793 raft_consensus.cc:493] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.626163 15793 raft_consensus.cc:515] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.627867 15564 raft_consensus.cc:359] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.628613 15564 raft_consensus.cc:385] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.631700 15564 raft_consensus.cc:740] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.629915 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed4922079864d94a4a771fb9466d63c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.639478 15367 raft_consensus.cc:2393] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:33.629330 15793 leader_election.cc:290] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.642004 15564 consensus_queue.cc:260] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.634531 15291 raft_consensus.cc:2468] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:33.638429 15563 tablet_bootstrap.cc:654] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.648099 15336 leader_election.cc:304] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:33.648664 15564 ts_tablet_manager.cc:1434] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:21:33.649055 15912 raft_consensus.cc:2804] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.649595 15912 raft_consensus.cc:697] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:33.650250 15564 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:33.650414 15912 consensus_queue.cc:237] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:33.661669 15261 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 90912.8us
I20260709 13:21:33.665022 15848 consensus_queue.cc:1048] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:33.666917 15515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:33.667366 15568 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:33.667757 15515 raft_consensus.cc:2393] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:33.667874 15568 ts_tablet_manager.cc:1403] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.013s
I20260709 13:21:33.669466 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed4922079864d94a4a771fb9466d63c" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.670042 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed4922079864d94a4a771fb9466d63c" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:33.670861 15568 raft_consensus.cc:359] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.670070 15516 raft_consensus.cc:2468] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:33.671543 15568 raft_consensus.cc:385] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.671864 15568 raft_consensus.cc:740] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.672539 15568 consensus_queue.cc:260] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.673883 15564 tablet_bootstrap.cc:654] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.674013 15623 raft_consensus.cc:515] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.675278 15563 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:33.675781 15563 ts_tablet_manager.cc:1403] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.101s	user 0.008s	sys 0.004s
I20260709 13:21:33.678576 15563 raft_consensus.cc:359] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.679268 15563 raft_consensus.cc:385] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.679129 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:33.679596 15563 raft_consensus.cc:740] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.679774 15516 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.680477 15563 consensus_queue.cc:260] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.682601 15563 ts_tablet_manager.cc:1434] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:21:33.683544 15563 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:33.685869 15568 ts_tablet_manager.cc:1434] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:21:33.688022 15564 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:33.688341 15568 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:33.688117 15516 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:33.688634 15564 ts_tablet_manager.cc:1403] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.007s
I20260709 13:21:33.689667 15515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.690384 15515 raft_consensus.cc:2376] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1. Re-sending same reply.
I20260709 13:21:33.691479 15564 raft_consensus.cc:359] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.692276 15564 raft_consensus.cc:385] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.692457 15563 tablet_bootstrap.cc:654] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.692536 15564 raft_consensus.cc:740] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.693554 15564 consensus_queue.cc:260] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.695058 15568 tablet_bootstrap.cc:654] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.696079 15564 ts_tablet_manager.cc:1434] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:33.697111 15564 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:33.704411 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:33.704811 15848 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.705336 15367 raft_consensus.cc:3060] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.705513 15848 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.708110 15848 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.709589 15564 tablet_bootstrap.cc:654] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.719193 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.720185 15368 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:33.721662 15258 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:33.722684 15848 raft_consensus.cc:2804] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.723155 15848 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.723510 15848 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.728875 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:33.729764 15444 raft_consensus.cc:3060] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.734169 15848 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.739464 15848 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.743062 15368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:33.744287 15368 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.752152 15444 raft_consensus.cc:2468] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:33.753391 15261 leader_election.cc:304] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:33.754211 15848 raft_consensus.cc:2804] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.754884 15848 raft_consensus.cc:697] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:33.755980 15848 consensus_queue.cc:237] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.759136 15367 raft_consensus.cc:2468] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:33.762209 15623 leader_election.cc:290] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:33.762603 15562 tablet_bootstrap.cc:492] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:33.774734 15562 ts_tablet_manager.cc:1403] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.184s	user 0.046s	sys 0.027s
I20260709 13:21:33.777860 15562 raft_consensus.cc:359] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.779168 15562 raft_consensus.cc:385] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.779510 15562 raft_consensus.cc:740] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.780480 15562 consensus_queue.cc:260] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
W20260709 13:21:33.756049 15919 log.cc:927] Time spent T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:21:33.769516 15368 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:33.770993 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.786990 15518 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:33.772354 15515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:33.784787 15562 ts_tablet_manager.cc:1434] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:21:33.788825 15562 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:33.794909 15515 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.799734 15258 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:33.800585 15848 raft_consensus.cc:2804] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.801162 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed4922079864d94a4a771fb9466d63c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.801911 15623 raft_consensus.cc:493] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.802148 15291 raft_consensus.cc:2393] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:33.802563 15623 raft_consensus.cc:515] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:33.801250 15848 raft_consensus.cc:697] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:33.805914 15848 consensus_queue.cc:237] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.837857 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:33.838963 15443 raft_consensus.cc:2468] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:33.840503 15261 leader_election.cc:304] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:33.842870 15623 leader_election.cc:290] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:33.845343 15793 raft_consensus.cc:493] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.846107 15793 raft_consensus.cc:515] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.848974 15918 consensus_queue.cc:1048] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:33.852800 15793 leader_election.cc:290] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.853937 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.854888 15367 raft_consensus.cc:2468] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:33.856583 15481 leader_election.cc:304] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:33.858012 15793 raft_consensus.cc:2804] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.858523 15793 raft_consensus.cc:493] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.859018 15793 raft_consensus.cc:3060] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.873405 15562 tablet_bootstrap.cc:654] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.880201 15712 raft_consensus.cc:493] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.881675 15484 leader_election.cc:304] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:33.887902 15858 raft_consensus.cc:493] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.892910 15858 raft_consensus.cc:515] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.870733 15623 raft_consensus.cc:2804] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.893563 15854 raft_consensus.cc:493] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.903632 15659 raft_consensus.cc:3060] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.918062 15858 leader_election.cc:290] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:33.919121 15564 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:33.920015 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "500dbd579fd848328ce9984276eb0e04" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:33.905939 15623 raft_consensus.cc:493] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.927402 15564 ts_tablet_manager.cc:1403] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.231s	user 0.056s	sys 0.072s
I20260709 13:21:33.907704 15857 raft_consensus.cc:493] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.883159 15712 raft_consensus.cc:515] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:33.928334 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 45597.9us
W20260709 13:21:33.935740 15407 leader_election.cc:343] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.916162 15854 raft_consensus.cc:515] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.937383 15793 raft_consensus.cc:515] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:33.930294 15623 raft_consensus.cc:3060] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.939108 15854 leader_election.cc:290] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.947070 15659 raft_consensus.cc:2749] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.951144 15564 raft_consensus.cc:359] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.959203 15564 raft_consensus.cc:385] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.954300 15854 consensus_queue.cc:1048] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:33.957567 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.932646 15857 raft_consensus.cc:515] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:33.959869 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:33.966917 15515 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:33.969709 15857 leader_election.cc:290] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:33.973291 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:33.974262 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
W20260709 13:21:33.983673 15407 leader_election.cc:343] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.961202 15564 raft_consensus.cc:740] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.951853 15712 leader_election.cc:290] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:33.960628 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "500dbd579fd848328ce9984276eb0e04" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
W20260709 13:21:33.996460 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 89436.7us
I20260709 13:21:33.988350 15564 consensus_queue.cc:260] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:33.998950 15484 leader_election.cc:304] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
W20260709 13:21:34.021713 15258 leader_election.cc:343] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.022791 15564 ts_tablet_manager.cc:1434] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.093s	user 0.010s	sys 0.000s
I20260709 13:21:34.023829 15564 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:34.030802 15568 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:34.031685 15568 ts_tablet_manager.cc:1403] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.344s	user 0.017s	sys 0.059s
W20260709 13:21:34.044214 15407 leader_election.cc:343] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.044739 15407 leader_election.cc:304] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:34.030972 15659 raft_consensus.cc:2749] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.051467 15712 raft_consensus.cc:2749] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.053256 15564 tablet_bootstrap.cc:654] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.069017 15563 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:34.069602 15563 ts_tablet_manager.cc:1403] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.386s	user 0.040s	sys 0.073s
I20260709 13:21:34.069218 15793 leader_election.cc:290] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
W20260709 13:21:34.076161 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 48212us
I20260709 13:21:34.082216 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:34.082921 15365 raft_consensus.cc:3060] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.083573 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:34.084244 15291 raft_consensus.cc:2468] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:34.088219 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:34.088851 15293 raft_consensus.cc:3060] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.089166 15568 raft_consensus.cc:359] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:34.089694 15410 leader_election.cc:343] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.090335 15568 raft_consensus.cc:385] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.090710 15568 raft_consensus.cc:740] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.094359 15562 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:34.093971 15568 consensus_queue.cc:260] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:34.094928 15562 ts_tablet_manager.cc:1403] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.306s	user 0.023s	sys 0.011s
I20260709 13:21:34.095840 15623 raft_consensus.cc:515] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.096678 15568 ts_tablet_manager.cc:1434] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 13:21:34.097721 15568 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:34.098227 15623 leader_election.cc:290] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:34.098955 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:34.099663 15443 raft_consensus.cc:2468] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:34.104331 15912 consensus_queue.cc:1048] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:34.106703 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:34.109535 15481 leader_election.cc:343] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.113509 15568 tablet_bootstrap.cc:654] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.090463 15410 leader_election.cc:304] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:34.116588 15941 raft_consensus.cc:2749] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.119055 15562 raft_consensus.cc:359] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.119781 15562 raft_consensus.cc:385] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.120038 15562 raft_consensus.cc:740] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.120879 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:34.122514 15365 raft_consensus.cc:2468] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
W20260709 13:21:34.127765 15258 leader_election.cc:343] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.128854 15562 consensus_queue.cc:260] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.138512 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:34.114367 15293 raft_consensus.cc:2468] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:34.177091 15484 leader_election.cc:304] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
W20260709 13:21:34.184518 15407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 47414.9us
W20260709 13:21:34.179793 15258 leader_election.cc:343] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.181051 15912 consensus_queue.cc:1048] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:34.181643 15333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 43651.1us
I20260709 13:21:34.186645 15258 leader_election.cc:304] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:34.176347 15918 consensus_queue.cc:1048] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:21:34.180083 15513 raft_consensus.cc:3060] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.197309 15562 ts_tablet_manager.cc:1434] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.000s
I20260709 13:21:34.199455 15444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:34.202283 15444 raft_consensus.cc:3060] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.197731 15854 raft_consensus.cc:2749] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.205690 15562 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:34.211902 15444 raft_consensus.cc:2468] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:34.201555 15943 raft_consensus.cc:2804] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.216389 15563 raft_consensus.cc:359] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.221277 15563 raft_consensus.cc:385] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.221719 15563 raft_consensus.cc:740] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.222561 15563 consensus_queue.cc:260] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
W20260709 13:21:34.248301 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 54761us
I20260709 13:21:34.202337 15941 raft_consensus.cc:493] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.253643 15563 ts_tablet_manager.cc:1434] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.184s	user 0.008s	sys 0.000s
I20260709 13:21:34.255285 15563 tablet_bootstrap.cc:492] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3: Bootstrap starting.
I20260709 13:21:34.221177 15943 raft_consensus.cc:493] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.217098 15481 leader_election.cc:304] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:34.283311 15564 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:34.283807 15564 ts_tablet_manager.cc:1403] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.260s	user 0.028s	sys 0.042s
I20260709 13:21:34.273285 15943 raft_consensus.cc:3060] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.259614 15941 raft_consensus.cc:515] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
W20260709 13:21:34.281932 15333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 83596.3us
I20260709 13:21:34.286919 15935 raft_consensus.cc:493] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.306746 15935 raft_consensus.cc:515] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:34.308835 15935 leader_election.cc:290] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:34.312633 15568 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:34.313390 15568 ts_tablet_manager.cc:1403] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.216s	user 0.010s	sys 0.018s
I20260709 13:21:34.292527 15793 raft_consensus.cc:2804] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:21:34.291913 15481 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 89825.6us
I20260709 13:21:34.323542 15912 raft_consensus.cc:493] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.324157 15912 raft_consensus.cc:515] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:34.326402 15912 leader_election.cc:290] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:34.331902 15793 raft_consensus.cc:697] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:34.336747 15793 consensus_queue.cc:237] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:34.338641 15564 raft_consensus.cc:359] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.340206 15564 raft_consensus.cc:385] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.340745 15564 raft_consensus.cc:740] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.344115 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:34.344805 15441 raft_consensus.cc:2468] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:34.346068 15336 leader_election.cc:304] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:34.348569 15513 raft_consensus.cc:2468] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:34.351827 15261 leader_election.cc:304] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:34.352741 15935 raft_consensus.cc:2804] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.353222 15935 raft_consensus.cc:697] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
W20260709 13:21:34.371876 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 117635us
I20260709 13:21:34.369545 15952 consensus_queue.cc:1048] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
W20260709 13:21:34.381889 15947 log.cc:927] Time spent T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 13:21:34.376802 15564 consensus_queue.cc:260] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.389078 15568 raft_consensus.cc:359] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:34.394809 15568 raft_consensus.cc:385] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.395336 15568 raft_consensus.cc:740] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.400707 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:34.409384 15563 tablet_bootstrap.cc:654] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.421754 15193 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42716 (request call id 60) took 1009 ms. Trace:
I20260709 13:21:34.418522 15562 tablet_bootstrap.cc:654] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.396411 15568 consensus_queue.cc:260] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:34.422233 15193 rpcz_store.cc:276] 0709 13:21:33.412708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.416089 (+  3381us) service_pool.cc:224] Handling call
0709 13:21:34.421694 (+1005605us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15040,"reported_tablets":6}
W20260709 13:21:34.433590 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 45350.4us
W20260709 13:21:34.437626 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 66944us
I20260709 13:21:34.440809 15568 ts_tablet_manager.cc:1434] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.127s	user 0.008s	sys 0.003s
I20260709 13:21:34.449234 15950 raft_consensus.cc:2804] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.449716 15950 raft_consensus.cc:493] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.450106 15950 raft_consensus.cc:3060] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.452714 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:34.453800 15295 raft_consensus.cc:2468] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:34.456018 15941 leader_election.cc:290] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:34.452767 15948 consensus_queue.cc:1048] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:34.457798 15935 consensus_queue.cc:237] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.460412 15190 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42728 (request call id 67) took 1067 ms. Trace:
I20260709 13:21:34.460299 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69858931805f420db0a026c73d7b0f06" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:34.461061 15190 rpcz_store.cc:276] 0709 13:21:33.393102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.397493 (+  4391us) service_pool.cc:224] Handling call
0709 13:21:34.460350 (+1062857us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32842,"reported_tablets":5}
W20260709 13:21:34.462463 15407 leader_election.cc:343] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.439764 15943 raft_consensus.cc:515] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:34.468896 15943 leader_election.cc:290] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:34.470412 15564 ts_tablet_manager.cc:1434] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.186s	user 0.009s	sys 0.000s
I20260709 13:21:34.470963 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:34.471421 15564 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:34.471627 15441 raft_consensus.cc:2468] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:34.474012 15568 tablet_bootstrap.cc:492] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:34.476974 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
W20260709 13:21:34.478659 15481 leader_election.cc:343] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:34.513792 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:21:34.528870 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 83215.5us
I20260709 13:21:34.517411 15623 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.571848 15956 consensus_queue.cc:1048] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.082s
I20260709 13:21:34.580214 15568 tablet_bootstrap.cc:654] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.548704 15623 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.586663 15623 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:34.588006 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:34.588804 15443 raft_consensus.cc:3060] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:34.547667 15261 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 45573.4us
W20260709 13:21:34.610915 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 105712us
I20260709 13:21:34.615448 15562 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:34.616037 15562 ts_tablet_manager.cc:1403] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.411s	user 0.019s	sys 0.013s
I20260709 13:21:34.590256 15950 raft_consensus.cc:515] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:34.636041 15564 tablet_bootstrap.cc:654] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.638823 15967 consensus_queue.cc:1048] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:21:34.640039 15369 raft_consensus.cc:2393] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:34.649361 15261 leader_election.cc:304] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:34.652139 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:34.652809 15440 raft_consensus.cc:2468] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:34.656481 15967 raft_consensus.cc:2804] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.656946 15967 raft_consensus.cc:493] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.657280 15967 raft_consensus.cc:3060] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.663398 15261 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:34.664966 15918 raft_consensus.cc:2804] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.665306 15950 leader_election.cc:290] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:34.665320 15918 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.666810 15918 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.669399 15562 raft_consensus.cc:359] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:34.670364 15562 raft_consensus.cc:385] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.666456 15438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:34.670794 15562 raft_consensus.cc:740] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.671307 15438 raft_consensus.cc:3060] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.671981 15562 consensus_queue.cc:260] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:34.677155 15562 ts_tablet_manager.cc:1434] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.061s	user 0.010s	sys 0.000s
I20260709 13:21:34.679982 15562 tablet_bootstrap.cc:492] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:34.715281 15563 tablet_bootstrap.cc:492] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3: No bootstrap required, opened a new log
I20260709 13:21:34.716027 15563 ts_tablet_manager.cc:1403] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3: Time spent bootstrapping tablet: real 0.461s	user 0.021s	sys 0.012s
I20260709 13:21:34.731333 15292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69858931805f420db0a026c73d7b0f06" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:34.717003 15956 raft_consensus.cc:493] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.721653 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:34.734524 15563 raft_consensus.cc:359] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:34.755849 15563 raft_consensus.cc:385] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.756219 15563 raft_consensus.cc:740] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Initialized, Role: FOLLOWER
W20260709 13:21:34.750943 15410 leader_election.cc:343] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.757660 15410 leader_election.cc:304] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:34.763463 15563 consensus_queue.cc:260] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:34.787451 15563 ts_tablet_manager.cc:1434] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.004s
I20260709 13:21:34.772430 15712 raft_consensus.cc:493] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.770059 15967 raft_consensus.cc:515] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:34.799162 15967 leader_election.cc:290] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:34.803347 15712 raft_consensus.cc:515] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.805384 15712 leader_election.cc:290] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:34.771983 15841 raft_consensus.cc:2749] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.806542 15293 raft_consensus.cc:2393] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:34.748045 15438 raft_consensus.cc:2468] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:34.741747 15443 raft_consensus.cc:2468] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:34.823361 15855 raft_consensus.cc:493] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.823997 15855 raft_consensus.cc:515] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:34.824862 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:34.825747 15366 raft_consensus.cc:2393] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:34.745020 15918 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.738978 15956 raft_consensus.cc:515] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:34.850353 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:34.853792 15973 log.cc:927] Time spent T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:21:34.856853 15511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:34.858385 15407 leader_election.cc:343] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.866382 15336 leader_election.cc:304] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:34.871197 15562 tablet_bootstrap.cc:654] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.880537 15952 raft_consensus.cc:2804] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.881100 15952 raft_consensus.cc:697] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:34.881937 15952 consensus_queue.cc:237] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:34.891474 15918 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:34.891752 15956 leader_election.cc:290] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:34.905815 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3593559594274cc7aef7c38772401110" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:34.922549 15436 raft_consensus.cc:2393] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:34.925439 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:34.906980 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:34.926111 15442 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.911135 15568 tablet_bootstrap.cc:492] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:34.927393 15568 ts_tablet_manager.cc:1403] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.454s	user 0.022s	sys 0.020s
I20260709 13:21:34.913062 15855 leader_election.cc:290] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:34.913780 15193 catalog_manager.cc:5697] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.932171 15193 catalog_manager.cc:5697] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.933812 15951 raft_consensus.cc:493] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.935537 15951 raft_consensus.cc:515] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:34.934831 15568 raft_consensus.cc:359] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:34.938381 15564 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:34.938691 15568 raft_consensus.cc:385] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.938910 15564 ts_tablet_manager.cc:1403] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.468s	user 0.026s	sys 0.027s
I20260709 13:21:34.939052 15568 raft_consensus.cc:740] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.939985 15568 consensus_queue.cc:260] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:34.933727 15193 catalog_manager.cc:5697] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:34.942116 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73732c746e334164b459508a1eeda5ec" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:34.942386 15261 leader_election.cc:304] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:34.943028 15951 leader_election.cc:290] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
W20260709 13:21:34.943679 15407 leader_election.cc:343] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.944928 15193 catalog_manager.cc:5697] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:34.946902 15336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 80607.9us
I20260709 13:21:34.945299 15568 ts_tablet_manager.cc:1434] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:21:34.944327 15918 raft_consensus.cc:2749] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:21:34.949446 15410 leader_election.cc:343] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.949730 15568 tablet_bootstrap.cc:492] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:34.949849 15410 leader_election.cc:304] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:34.951797 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:34.952441 15290 raft_consensus.cc:2468] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:34.952997 15934 consensus_queue.cc:1048] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.956470 15854 consensus_queue.cc:1048] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:34.956801 15951 raft_consensus.cc:2749] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.960630 15564 raft_consensus.cc:359] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.962772 15564 raft_consensus.cc:385] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.964027 15564 raft_consensus.cc:740] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.965380 15564 consensus_queue.cc:260] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:34.965910 15336 leader_election.cc:304] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:34.969175 15956 raft_consensus.cc:2804] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.969646 15956 raft_consensus.cc:493] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.970045 15956 raft_consensus.cc:3060] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:34.969844 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 190654us
I20260709 13:21:34.978816 15562 tablet_bootstrap.cc:492] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:34.979353 15562 ts_tablet_manager.cc:1403] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.300s	user 0.012s	sys 0.004s
I20260709 13:21:34.981513 15442 raft_consensus.cc:2468] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:34.981305 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73732c746e334164b459508a1eeda5ec" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:34.967674 15564 ts_tablet_manager.cc:1434] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
W20260709 13:21:34.987179 15407 leader_election.cc:343] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.987594 15407 leader_election.cc:304] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:34.988215 15951 raft_consensus.cc:2749] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.990556 15968 consensus_queue.cc:1048] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.993599 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:34.994318 15290 raft_consensus.cc:2468] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:34.995241 15562 raft_consensus.cc:359] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:34.996752 15562 raft_consensus.cc:385] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.996806 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:34.997521 15562 raft_consensus.cc:740] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.999154 15562 consensus_queue.cc:260] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.000260 15564 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:35.006121 15261 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:35.007190 15968 raft_consensus.cc:2804] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.008255 15562 ts_tablet_manager.cc:1434] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:21:35.009553 15562 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:35.007838 15968 raft_consensus.cc:697] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:34.961971 15568 tablet_bootstrap.cc:654] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:35.020146 15258 leader_election.cc:343] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.023005 15956 raft_consensus.cc:515] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.023406 15515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:35.024206 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:35.024858 15513 raft_consensus.cc:2468] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:35.025197 15956 leader_election.cc:290] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:35.027328 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:35.027882 15366 raft_consensus.cc:2393] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:35.029322 15484 leader_election.cc:304] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:35.036518 15967 raft_consensus.cc:493] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.037056 15967 raft_consensus.cc:515] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.027913 15290 raft_consensus.cc:3060] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.045019 15967 leader_election.cc:290] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:35.045624 15978 raft_consensus.cc:2804] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.049072 15978 raft_consensus.cc:493] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.049431 15978 raft_consensus.cc:3060] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.045379 15979 consensus_queue.cc:1048] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.064010 15956 raft_consensus.cc:493] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.064574 15956 raft_consensus.cc:515] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.059659 15484 leader_election.cc:304] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:35.065987 15978 raft_consensus.cc:515] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.071439 15979 raft_consensus.cc:2804] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:21:35.073642 15258 leader_election.cc:343] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.077167 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.078078 15440 raft_consensus.cc:2468] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:35.079413 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:35.080420 15979 raft_consensus.cc:697] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:35.081664 15290 raft_consensus.cc:2468] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:35.081382 15979 consensus_queue.cc:237] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.082566 15956 leader_election.cc:290] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:35.082309 15958 consensus_queue.cc:1048] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.087554 15562 tablet_bootstrap.cc:654] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.088285 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:35.089788 15336 leader_election.cc:304] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:35.091610 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:35.091813 15258 leader_election.cc:343] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.089959 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd5f83d74ae74d00b588a9fd88c8b0b0" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.103521 15442 raft_consensus.cc:2393] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:35.102242 15261 leader_election.cc:304] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:35.092908 15956 raft_consensus.cc:2804] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.104660 15956 raft_consensus.cc:697] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:35.105856 15935 raft_consensus.cc:2804] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.106308 15935 raft_consensus.cc:493] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.106743 15935 raft_consensus.cc:3060] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.106118 15956 consensus_queue.cc:237] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.105895 15336 leader_election.cc:304] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:35.111503 15952 raft_consensus.cc:3060] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.102047 15564 tablet_bootstrap.cc:654] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.104101 15518 raft_consensus.cc:2393] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:35.084553 15968 consensus_queue.cc:237] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.146735 15978 leader_election.cc:290] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:35.156597 15935 raft_consensus.cc:515] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.157398 15952 raft_consensus.cc:2749] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:35.159727 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:35.160364 15442 raft_consensus.cc:3060] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.161202 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:35.161979 15290 raft_consensus.cc:2393] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:35.169808 15978 consensus_queue.cc:1048] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:35.171769 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d95073a3b15444a4880921aaad3f82a1" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:35.178469 15189 catalog_manager.cc:5697] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:35.180600 15568 tablet_bootstrap.cc:492] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:35.180267 15189 catalog_manager.cc:5697] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:35.182597 15562 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:35.181123 15568 ts_tablet_manager.cc:1403] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.232s	user 0.014s	sys 0.009s
I20260709 13:21:35.183187 15562 ts_tablet_manager.cc:1403] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.174s	user 0.011s	sys 0.004s
I20260709 13:21:35.185580 15511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:35.179786 15935 leader_election.cc:290] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:35.191639 15564 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:35.192116 15564 ts_tablet_manager.cc:1403] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.192s	user 0.011s	sys 0.005s
I20260709 13:21:35.192817 15481 leader_election.cc:304] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:35.193675 15979 raft_consensus.cc:2749] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:21:35.200213 15258 leader_election.cc:343] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.203323 15442 raft_consensus.cc:2468] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 1.
I20260709 13:21:35.204509 15261 leader_election.cc:304] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:35.205283 15935 raft_consensus.cc:2804] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.205766 15935 raft_consensus.cc:697] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:35.205586 15564 raft_consensus.cc:359] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.206691 15564 raft_consensus.cc:385] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.206664 15935 consensus_queue.cc:237] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.207257 15564 raft_consensus.cc:740] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.214960 15564 consensus_queue.cc:260] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.215564 15568 raft_consensus.cc:359] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.216295 15568 raft_consensus.cc:385] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.216589 15568 raft_consensus.cc:740] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.210438 15562 raft_consensus.cc:359] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.218964 15562 raft_consensus.cc:385] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.219277 15562 raft_consensus.cc:740] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.220000 15562 consensus_queue.cc:260] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.222167 15562 ts_tablet_manager.cc:1434] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.038s	user 0.003s	sys 0.000s
I20260709 13:21:35.223196 15562 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:35.227561 15568 consensus_queue.cc:260] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.228689 15564 ts_tablet_manager.cc:1434] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 13:21:35.229664 15564 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:35.231419 15568 ts_tablet_manager.cc:1434] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 13:21:35.235491 15568 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:35.238286 15193 catalog_manager.cc:5697] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:35.239713 15193 catalog_manager.cc:5697] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } }
I20260709 13:21:35.248654 15562 tablet_bootstrap.cc:654] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.253652 15568 tablet_bootstrap.cc:654] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.259601 15562 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:35.260175 15562 ts_tablet_manager.cc:1403] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.037s	user 0.002s	sys 0.008s
I20260709 13:21:35.263128 15562 raft_consensus.cc:359] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.265229 15659 raft_consensus.cc:493] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.265766 15659 raft_consensus.cc:515] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.267851 15659 leader_election.cc:290] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.268041 15568 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:35.268541 15568 ts_tablet_manager.cc:1403] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 13:21:35.271164 15568 raft_consensus.cc:359] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.271790 15568 raft_consensus.cc:385] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.272051 15568 raft_consensus.cc:740] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.272158 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "500dbd579fd848328ce9984276eb0e04" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:35.272722 15562 raft_consensus.cc:385] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.272836 15367 raft_consensus.cc:2468] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:35.273514 15568 consensus_queue.cc:260] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.273041 15562 raft_consensus.cc:740] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.275537 15481 leader_election.cc:304] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:35.276397 15659 raft_consensus.cc:2804] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.276484 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "500dbd579fd848328ce9984276eb0e04" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.276858 15659 raft_consensus.cc:493] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.277439 15659 raft_consensus.cc:3060] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.282542 15442 raft_consensus.cc:2468] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:35.274896 15562 consensus_queue.cc:260] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.285802 15562 ts_tablet_manager.cc:1434] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:21:35.293008 15564 tablet_bootstrap.cc:654] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.293067 15562 tablet_bootstrap.cc:492] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:35.291493 15659 raft_consensus.cc:515] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.290237 15568 ts_tablet_manager.cc:1434] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:21:35.297240 15949 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.297776 15949 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.307091 15949 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:35.308032 15568 tablet_bootstrap.cc:492] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:35.316808 15562 tablet_bootstrap.cc:654] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.318816 15659 leader_election.cc:290] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.320868 15191 catalog_manager.cc:5697] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:35.324446 15564 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:35.324615 15956 consensus_queue.cc:1048] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:35.325711 15949 raft_consensus.cc:493] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.328642 15949 raft_consensus.cc:515] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.330770 15949 leader_election.cc:290] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:35.331617 15564 ts_tablet_manager.cc:1403] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.102s	user 0.007s	sys 0.004s
I20260709 13:21:35.332093 15568 tablet_bootstrap.cc:654] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.334478 15564 raft_consensus.cc:359] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.335577 15564 raft_consensus.cc:385] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.335843 15564 raft_consensus.cc:740] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.336584 15564 consensus_queue.cc:260] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.346092 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.346975 15436 raft_consensus.cc:2393] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:35.347846 15562 tablet_bootstrap.cc:492] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:35.350095 15562 ts_tablet_manager.cc:1403] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.003s
I20260709 13:21:35.351671 15193 catalog_manager.cc:5697] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:35.361085 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "500dbd579fd848328ce9984276eb0e04" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:35.361703 15436 raft_consensus.cc:3060] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.363027 15564 ts_tablet_manager.cc:1434] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:21:35.364109 15564 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:35.367969 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.368763 15440 raft_consensus.cc:2393] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:35.369099 15562 raft_consensus.cc:359] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.369982 15562 raft_consensus.cc:385] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.370329 15562 raft_consensus.cc:740] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.373445 15568 tablet_bootstrap.cc:492] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:35.374116 15568 ts_tablet_manager.cc:1403] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.001s
I20260709 13:21:35.377921 15568 raft_consensus.cc:359] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.378805 15568 raft_consensus.cc:385] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.379140 15568 raft_consensus.cc:740] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.380404 15564 tablet_bootstrap.cc:654] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.379902 15568 consensus_queue.cc:260] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.371450 15562 consensus_queue.cc:260] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.384192 15562 ts_tablet_manager.cc:1434] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:21:35.391325 15568 ts_tablet_manager.cc:1434] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:21:35.391086 15293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:35.392675 15568 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:35.393698 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "500dbd579fd848328ce9984276eb0e04" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:35.394428 15365 raft_consensus.cc:3060] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.395928 15564 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:35.396421 15564 ts_tablet_manager.cc:1403] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.007s
I20260709 13:21:35.397823 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:35.399633 15562 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:35.399564 15564 raft_consensus.cc:359] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.400353 15564 raft_consensus.cc:385] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.400651 15564 raft_consensus.cc:740] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.401365 15564 consensus_queue.cc:260] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.403419 15564 ts_tablet_manager.cc:1434] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:21:35.404824 15564 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:35.410868 15564 tablet_bootstrap.cc:654] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.419581 15436 raft_consensus.cc:2468] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:35.424803 15562 tablet_bootstrap.cc:654] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.430032 15568 tablet_bootstrap.cc:654] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.437678 15562 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:35.443706 15562 ts_tablet_manager.cc:1403] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.001s
I20260709 13:21:35.448088 15562 raft_consensus.cc:359] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.448763 15562 raft_consensus.cc:385] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.449008 15562 raft_consensus.cc:740] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.451512 15568 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:35.452235 15568 ts_tablet_manager.cc:1403] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.005s
I20260709 13:21:35.455842 15854 raft_consensus.cc:493] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.456140 15568 raft_consensus.cc:359] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.457139 15568 raft_consensus.cc:385] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.456634 15854 raft_consensus.cc:515] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.457489 15568 raft_consensus.cc:740] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.458942 15562 consensus_queue.cc:260] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.463424 15562 ts_tablet_manager.cc:1434] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 13:21:35.464711 15562 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:35.472522 15562 tablet_bootstrap.cc:654] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.458469 15568 consensus_queue.cc:260] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.478554 15568 ts_tablet_manager.cc:1434] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:21:35.479818 15568 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:35.478878 15967 raft_consensus.cc:493] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.482513 15967 raft_consensus.cc:515] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.485559 15967 leader_election.cc:290] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.488343 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.489527 15441 raft_consensus.cc:2468] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:35.493170 15854 leader_election.cc:290] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:35.494256 15949 raft_consensus.cc:493] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.494998 15949 raft_consensus.cc:515] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.497475 15949 leader_election.cc:290] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.498238 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73732c746e334164b459508a1eeda5ec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:35.498903 15367 raft_consensus.cc:2468] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:35.500046 15481 leader_election.cc:304] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:35.500926 15949 raft_consensus.cc:2804] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.501371 15949 raft_consensus.cc:493] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.501828 15949 raft_consensus.cc:3060] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.502691 15568 tablet_bootstrap.cc:654] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.515879 15949 raft_consensus.cc:515] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.519268 15562 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:35.519378 15949 leader_election.cc:290] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.519863 15562 ts_tablet_manager.cc:1403] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.008s
I20260709 13:21:35.523518 15562 raft_consensus.cc:359] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.524277 15562 raft_consensus.cc:385] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.524677 15562 raft_consensus.cc:740] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.525518 15562 consensus_queue.cc:260] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.526360 15568 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:35.524302 15484 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:35.526997 15568 ts_tablet_manager.cc:1403] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.000s
I20260709 13:21:35.527444 15949 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.528053 15562 ts_tablet_manager.cc:1434] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:21:35.528707 15484 leader_election.cc:304] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:35.529479 15659 raft_consensus.cc:3060] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.529800 15484 leader_election.cc:304] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:35.530519 15979 raft_consensus.cc:2804] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.530998 15979 raft_consensus.cc:697] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:35.531277 15568 raft_consensus.cc:359] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.532245 15568 raft_consensus.cc:385] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.531816 15979 consensus_queue.cc:237] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:35.532545 15568 raft_consensus.cc:740] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.533365 15568 consensus_queue.cc:260] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.526413 15564 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:35.535401 15564 ts_tablet_manager.cc:1403] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.131s	user 0.032s	sys 0.079s
I20260709 13:21:35.535727 15949 raft_consensus.cc:2749] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:35.536974 15659 raft_consensus.cc:2749] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:35.537760 15562 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b: Bootstrap starting.
I20260709 13:21:35.538314 15564 raft_consensus.cc:359] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.539408 15564 raft_consensus.cc:385] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.538698 15568 ts_tablet_manager.cc:1434] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:21:35.540776 15568 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b: Bootstrap starting.
I20260709 13:21:35.539681 15564 raft_consensus.cc:740] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.659049 15564 consensus_queue.cc:260] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.679486 15956 raft_consensus.cc:493] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.680687 15956 raft_consensus.cc:515] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:35.685092 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 155173us
I20260709 13:21:35.697190 15956 leader_election.cc:290] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.700752 15855 raft_consensus.cc:493] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.696935 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:35.703580 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.704816 15436 raft_consensus.cc:2468] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:35.703759 15367 raft_consensus.cc:2468] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:35.707470 15190 catalog_manager.cc:5697] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:35.706180 15855 raft_consensus.cc:515] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:35.696875 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:35.721117 15366 raft_consensus.cc:2468] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:35.729293 15190 catalog_manager.cc:5697] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:35.739333 15918 raft_consensus.cc:493] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.740304 15918 raft_consensus.cc:515] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.715890 15564 ts_tablet_manager.cc:1434] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.180s	user 0.006s	sys 0.006s
I20260709 13:21:35.753412 15961 raft_consensus.cc:493] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.770190 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:35.774117 15961 raft_consensus.cc:515] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.776870 15961 leader_election.cc:290] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:35.779083 15291 raft_consensus.cc:2468] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:35.792572 15855 leader_election.cc:290] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:35.788556 15562 tablet_bootstrap.cc:654] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.799335 15336 leader_election.cc:304] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:35.804419 15950 raft_consensus.cc:2804] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.805034 15950 raft_consensus.cc:493] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.805449 15950 raft_consensus.cc:3060] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.807842 15953 raft_consensus.cc:493] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.808820 15568 tablet_bootstrap.cc:654] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.808759 15953 raft_consensus.cc:515] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.813812 15564 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:35.814653 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:35.815490 15290 raft_consensus.cc:2468] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:35.817803 15410 leader_election.cc:304] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:35.812610 15953 leader_election.cc:290] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:35.819185 15841 raft_consensus.cc:2804] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.819537 15918 leader_election.cc:290] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.819646 15841 raft_consensus.cc:493] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.820441 15841 raft_consensus.cc:3060] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.829427 15518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:35.830256 15518 raft_consensus.cc:2468] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:35.835168 15258 leader_election.cc:304] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
W20260709 13:21:35.836915 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 64658.5us
I20260709 13:21:35.858768 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:35.860332 15950 raft_consensus.cc:515] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:35.877671 15982 consensus_queue.cc:1048] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:21:35.885592 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:35.886480 15436 raft_consensus.cc:3060] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.887202 15950 leader_election.cc:290] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:35.893642 15258 leader_election.cc:304] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
W20260709 13:21:35.904412 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 66879.6us
I20260709 13:21:35.907917 15918 raft_consensus.cc:2804] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.908413 15918 raft_consensus.cc:493] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.908821 15918 raft_consensus.cc:3060] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:35.910969 15261 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 115885us
I20260709 13:21:35.917872 15562 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b: No bootstrap required, opened a new log
I20260709 13:21:35.920207 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73732c746e334164b459508a1eeda5ec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:35.921082 15440 raft_consensus.cc:2468] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:35.923156 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73732c746e334164b459508a1eeda5ec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:35.923760 15440 raft_consensus.cc:3060] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.925209 15562 ts_tablet_manager.cc:1403] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent bootstrapping tablet: real 0.396s	user 0.026s	sys 0.009s
I20260709 13:21:35.902201 16017 raft_consensus.cc:2804] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.926898 16017 raft_consensus.cc:493] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.927337 16017 raft_consensus.cc:3060] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.928915 15562 raft_consensus.cc:359] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.929718 15562 raft_consensus.cc:385] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.930902 15562 raft_consensus.cc:740] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.941310 15967 consensus_queue.cc:1048] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:21:35.941547 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:35.944106 15568 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b: No bootstrap required, opened a new log
I20260709 13:21:35.944881 15568 ts_tablet_manager.cc:1403] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent bootstrapping tablet: real 0.404s	user 0.016s	sys 0.017s
I20260709 13:21:35.947834 15564 tablet_bootstrap.cc:654] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.941390 15959 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:35.952054 15258 leader_election.cc:343] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.934258 15950 consensus_queue.cc:1048] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:21:35.930567 15193 catalog_manager.cc:5697] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:35.963203 15959 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.968734 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73732c746e334164b459508a1eeda5ec" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:35.969679 15369 raft_consensus.cc:3060] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.966398 15959 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:35.977031 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:35.977789 15367 raft_consensus.cc:2468] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:35.995695 15956 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.996245 15956 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:35.998679 15956 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:36.004295 15568 raft_consensus.cc:359] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:36.005395 15568 raft_consensus.cc:385] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.005836 15568 raft_consensus.cc:740] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.006959 15568 consensus_queue.cc:260] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:36.007671 15363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:36.008620 15363 raft_consensus.cc:2468] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 0.
I20260709 13:21:36.009157 15568 ts_tablet_manager.cc:1434] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 13:21:36.012056 15436 raft_consensus.cc:2468] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
W20260709 13:21:36.014276 15365 consensus_meta.cc:287] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b: Time spent flushing consensus metadata: real 0.619s	user 0.005s	sys 0.019s
I20260709 13:21:36.014930 15365 raft_consensus.cc:2468] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:36.015658 15336 leader_election.cc:304] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
W20260709 13:21:36.016649 15261 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 57236.2us
I20260709 13:21:36.016920 15961 consensus_queue.cc:1048] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.019263 15967 consensus_queue.cc:1048] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:36.023232 15443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:36.023494 15956 raft_consensus.cc:2804] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.024010 15956 raft_consensus.cc:697] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:36.025164 15956 consensus_queue.cc:237] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:36.026912 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 66155.5us
I20260709 13:21:36.028777 15481 leader_election.cc:304] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:36.029726 15961 raft_consensus.cc:2804] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.028551 15562 consensus_queue.cc:260] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:36.031631 15961 raft_consensus.cc:493] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.958943 15841 raft_consensus.cc:515] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:36.046355 15443 raft_consensus.cc:2393] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:36.050374 15961 raft_consensus.cc:3060] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.026567 15918 raft_consensus.cc:515] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:36.039026 15440 raft_consensus.cc:2468] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:36.071233 15841 leader_election.cc:290] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:36.082911 15934 consensus_queue.cc:1048] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.087203 15918 leader_election.cc:290] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:36.091136 15369 raft_consensus.cc:2468] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:36.057998 16017 raft_consensus.cc:515] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:36.091797 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:36.097041 15979 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.097556 15291 raft_consensus.cc:2393] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:36.097584 15979 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:36.104045 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:36.104797 15289 raft_consensus.cc:3060] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.105865 15979 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:36.101524 15365 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:36.124257 16017 leader_election.cc:290] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:36.129987 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69858931805f420db0a026c73d7b0f06" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:36.130764 15562 ts_tablet_manager.cc:1434] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b: Time spent starting tablet: real 0.205s	user 0.007s	sys 0.002s
I20260709 13:21:36.130849 15442 raft_consensus.cc:2468] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:36.131950 15961 raft_consensus.cc:515] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:36.132714 15261 leader_election.cc:304] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
W20260709 13:21:36.133157 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 59053.4us
W20260709 13:21:36.137193 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 55542us
I20260709 13:21:36.150151 15484 leader_election.cc:304] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:36.160025 15193 catalog_manager.cc:5697] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 1 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.167876 15484 leader_election.cc:304] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:36.169842 16017 consensus_queue.cc:1048] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:21:36.142560 15289 raft_consensus.cc:2468] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:36.173033 15721 raft_consensus.cc:2749] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.148542 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:36.173724 15855 raft_consensus.cc:2804] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.174117 15436 raft_consensus.cc:2393] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:36.155658 15961 leader_election.cc:290] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:36.180406 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69858931805f420db0a026c73d7b0f06" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:36.174177 15855 raft_consensus.cc:493] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.177014 15979 raft_consensus.cc:2804] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.199180 15979 raft_consensus.cc:697] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:36.199306 15291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:36.200079 15291 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:36.200340 15979 consensus_queue.cc:237] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:36.184510 15949 consensus_queue.cc:1048] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:21:36.202951 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:36.203848 15290 raft_consensus.cc:2393] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:36.204465 15855 raft_consensus.cc:3060] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.204635 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:36.205304 15364 raft_consensus.cc:3060] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:36.211973 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 44193.6us
W20260709 13:21:36.215374 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:36.217137 15410 leader_election.cc:304] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:36.221084 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:36.222295 15363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "979c935f63114ce6b2c31919b88a250f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:36.210577 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:36.236874 15366 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 0.
I20260709 13:21:36.246876 15659 consensus_queue.cc:1048] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.247675 16019 raft_consensus.cc:2804] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.248303 16019 raft_consensus.cc:697] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:36.249380 16019 consensus_queue.cc:237] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:36.255425 15258 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:36.256723 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06ec5b047f9d4549ac7e3bf2d5f71bb2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:36.257552 15367 raft_consensus.cc:3060] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.266026 15258 leader_election.cc:304] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:36.269239 15935 raft_consensus.cc:2749] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.269155 15854 raft_consensus.cc:2804] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.270738 15854 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.271238 15854 raft_consensus.cc:3060] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.276916 15564 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: No bootstrap required, opened a new log
I20260709 13:21:36.277669 15564 ts_tablet_manager.cc:1403] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.464s	user 0.013s	sys 0.013s
I20260709 13:21:36.282204 15336 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:36.283453 15982 raft_consensus.cc:2804] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.297127 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:36.297763 15517 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 0.
I20260709 13:21:36.306970 15364 raft_consensus.cc:2468] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:36.312479 15855 raft_consensus.cc:515] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
W20260709 13:21:36.290984 15481 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 44651.8us
I20260709 13:21:36.295603 15982 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.328069 15982 raft_consensus.cc:3060] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.328475 15854 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:36.331198 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:36.331913 15516 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 0.
I20260709 13:21:36.344480 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:36.345368 15517 raft_consensus.cc:2393] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:36.347126 15564 raft_consensus.cc:359] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:36.350418 15564 raft_consensus.cc:385] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.350768 15564 raft_consensus.cc:740] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.368036 15367 raft_consensus.cc:2468] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
W20260709 13:21:36.373570 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 54167us
I20260709 13:21:36.370218 15564 consensus_queue.cc:260] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:36.318748 15481 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:36.363914 15855 leader_election.cc:290] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:36.374720 15854 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
W20260709 13:21:36.384594 15481 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 67031.2us
I20260709 13:21:36.385089 16024 consensus_queue.cc:1048] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.385350 15481 leader_election.cc:304] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:36.420733 15564 ts_tablet_manager.cc:1434] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.143s	user 0.010s	sys 0.000s
I20260709 13:21:36.393110 15659 raft_consensus.cc:2804] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.410141 15855 raft_consensus.cc:2804] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:21:36.422571 15484 leader_election.cc:343] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.431502 15659 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.442211 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69858931805f420db0a026c73d7b0f06" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
W20260709 13:21:36.459123 15336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:35805, user_credentials={real_user=slave}} blocked reactor thread for 60603.1us
I20260709 13:21:36.445672 15855 raft_consensus.cc:697] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
W20260709 13:21:36.463415 16038 log.cc:927] Time spent T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:21:36.461836 15333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 61475.3us
I20260709 13:21:36.448381 15659 raft_consensus.cc:3060] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.485682 15440 raft_consensus.cc:3060] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.504874 15855 consensus_queue.cc:237] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:36.524792 15982 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:36.539319 15258 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 115925us
W20260709 13:21:36.446805 15481 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 61540.9us
I20260709 13:21:36.542178 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:36.546005 15369 raft_consensus.cc:2393] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:36.545823 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:36.553872 15191 catalog_manager.cc:5697] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:36.557924 15659 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:36.559355 15513 raft_consensus.cc:2393] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88eead6a17f54dd88c723744e0aa5b50 in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:36.554838 15982 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:36.574477 15659 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:36.574795 15258 leader_election.cc:304] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:36.576512 16037 consensus_queue.cc:1048] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:36.577653 16050 raft_consensus.cc:2749] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.579854 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:36.580727 15513 raft_consensus.cc:2393] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:36.587266 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:36.582247 15190 catalog_manager.cc:5697] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:36.588153 15364 raft_consensus.cc:2393] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:36.591066 15288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:36.591900 15288 raft_consensus.cc:2393] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:36.594322 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4413c641a248a9a0cf781768951edc" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:36.595356 15290 raft_consensus.cc:2393] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:36.596580 15290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69858931805f420db0a026c73d7b0f06" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:36.597170 15290 raft_consensus.cc:3060] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.600476 15440 raft_consensus.cc:2468] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:36.602937 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:36.619117 15289 raft_consensus.cc:2393] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:36.628985 15290 raft_consensus.cc:2468] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:36.646675 15258 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:36.650696 15982 consensus_queue.cc:1048] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.659597 16050 raft_consensus.cc:2749] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.680954 15289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:36.681974 15289 raft_consensus.cc:2393] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 1: Already voted for candidate 88eead6a17f54dd88c723744e0aa5b50 in this term.
I20260709 13:21:36.677968 15484 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:36.683988 15659 raft_consensus.cc:2749] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.684417 15484 leader_election.cc:304] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:36.691557 15659 raft_consensus.cc:2804] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.692086 15659 raft_consensus.cc:697] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:36.692946 15659 consensus_queue.cc:237] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:36.711695 15659 consensus_queue.cc:1048] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:36.714252 16051 consensus_queue.cc:1048] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:36.734122 15336 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:36.743165 16044 raft_consensus.cc:2749] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.744234 16048 consensus_queue.cc:1048] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.768652 15190 catalog_manager.cc:5697] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.796802 15953 consensus_queue.cc:1048] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.771658 15190 catalog_manager.cc:5697] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 1 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.832937 15841 consensus_queue.cc:1048] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:36.914291 16051 consensus_queue.cc:1048] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:36.933948 15659 consensus_queue.cc:1048] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:37.141645 15659 consensus_queue.cc:1048] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:37.156617 15949 consensus_queue.cc:1048] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:37.251986 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:21:38.285691 15240 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:38.390319 16044 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.391054 16044 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:38.393479 16044 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:38.394428 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:38.395119 15516 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:38.396133 15333 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:38.395438 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:38.397073 15295 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:38.397254 16044 raft_consensus.cc:2804] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:38.398348 16044 raft_consensus.cc:493] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.398763 16044 raft_consensus.cc:3060] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:38.405342 16044 raft_consensus.cc:515] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:38.407019 16044 leader_election.cc:290] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:38.407812 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:38.408371 15516 raft_consensus.cc:3060] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:38.410422 15295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d52192ec7445f4a801452a645af6ba" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:38.411089 15295 raft_consensus.cc:3060] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:38.416136 15516 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:38.417153 15333 leader_election.cc:304] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:38.417893 16044 raft_consensus.cc:2804] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:38.418080 15295 raft_consensus.cc:2468] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:38.418391 16044 raft_consensus.cc:697] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:38.419418 16044 consensus_queue.cc:237] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:38.428145 15190 catalog_manager.cc:5697] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 0 to 2, leader changed from <none> to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 2 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:38.939682  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
W20260709 13:21:38.942224 15484 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:38.947244 15484 consensus_peers.cc:597] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:38.963804 15336 consensus_peers.cc:597] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:38.973920 15484 consensus_peers.cc:597] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:38.989274 15516 raft_consensus.cc:1275] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:38.990906 16082 consensus_queue.cc:1048] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:38.992868  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:38.993988  5410 tablet_replica.cc:333] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
W20260709 13:21:38.994498 15410 consensus_peers.cc:597] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:38.995038  5410 raft_consensus.cc:2243] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.995618  5410 raft_consensus.cc:2272] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.998180  5410 tablet_replica.cc:333] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:38.999351  5410 raft_consensus.cc:2243] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.999826  5410 raft_consensus.cc:2272] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:38.999946 15336 consensus_peers.cc:597] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.002802  5410 tablet_replica.cc:333] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.003976  5410 raft_consensus.cc:2243] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.004652  5410 raft_consensus.cc:2272] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.007635  5410 tablet_replica.cc:333] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.008518  5410 raft_consensus.cc:2243] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.008989  5410 raft_consensus.cc:2272] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.011523  5410 tablet_replica.cc:333] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.012194  5410 raft_consensus.cc:2243] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.012895  5410 raft_consensus.cc:2272] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.015730  5410 tablet_replica.cc:333] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.016533  5410 raft_consensus.cc:2243] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.017448  5410 raft_consensus.cc:2272] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.020499  5410 tablet_replica.cc:333] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.021312  5410 raft_consensus.cc:2243] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.022399  5410 raft_consensus.cc:2272] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.025213  5410 tablet_replica.cc:333] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.025992  5410 raft_consensus.cc:2243] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.026979  5410 raft_consensus.cc:2272] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.028673 15336 consensus_peers.cc:597] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.029840  5410 tablet_replica.cc:333] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.030774  5410 raft_consensus.cc:2243] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.031342  5410 raft_consensus.cc:2272] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.032051 15484 consensus_peers.cc:597] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.034080  5410 tablet_replica.cc:333] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.035022  5410 raft_consensus.cc:2243] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.036022  5410 raft_consensus.cc:2272] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.039213  5410 tablet_replica.cc:333] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.039990  5410 raft_consensus.cc:2243] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.041193  5410 raft_consensus.cc:2272] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.043329 15336 consensus_peers.cc:597] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.044231  5410 tablet_replica.cc:333] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.045080  5410 raft_consensus.cc:2243] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.045538  5410 raft_consensus.cc:2272] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.046453 15484 consensus_peers.cc:597] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.047900  5410 tablet_replica.cc:333] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.048610  5410 raft_consensus.cc:2243] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.049454  5410 raft_consensus.cc:2272] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.051560  5410 tablet_replica.cc:333] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.052347  5410 raft_consensus.cc:2243] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.052858  5410 raft_consensus.cc:2272] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.055946  5410 tablet_replica.cc:333] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.056533  5410 raft_consensus.cc:2243] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.057570  5410 raft_consensus.cc:2272] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.060014  5410 tablet_replica.cc:333] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.060541  5410 raft_consensus.cc:2243] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.061285  5410 raft_consensus.cc:2272] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.063414  5410 tablet_replica.cc:333] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.063934  5410 raft_consensus.cc:2243] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.064400  5410 raft_consensus.cc:2272] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.068579  5410 tablet_replica.cc:333] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.069416  5410 raft_consensus.cc:2243] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.069932  5410 raft_consensus.cc:2272] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.072651  5410 tablet_replica.cc:333] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.074162  5410 raft_consensus.cc:2243] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.074687  5410 raft_consensus.cc:2272] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.076933 15336 consensus_peers.cc:597] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.077333  5410 tablet_replica.cc:333] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.078177  5410 raft_consensus.cc:2243] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.079197  5410 raft_consensus.cc:2272] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.081938  5410 tablet_replica.cc:333] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
W20260709 13:21:39.082187 15484 consensus_peers.cc:597] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.082587  5410 raft_consensus.cc:2243] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.083457  5410 raft_consensus.cc:2272] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.085397  5410 tablet_replica.cc:333] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.085965  5410 raft_consensus.cc:2243] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.086768  5410 raft_consensus.cc:2272] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.088645  5410 tablet_replica.cc:333] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.089179  5410 raft_consensus.cc:2243] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.089576  5410 raft_consensus.cc:2272] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.092110  5410 tablet_replica.cc:333] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.092831  5410 raft_consensus.cc:2243] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.093359  5410 raft_consensus.cc:2272] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.095788  5410 tablet_replica.cc:333] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.096396  5410 raft_consensus.cc:2243] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.096980  5410 raft_consensus.cc:2272] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.099797  5410 tablet_replica.cc:333] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.100517  5410 raft_consensus.cc:2243] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.101025  5410 raft_consensus.cc:2272] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.102842 15410 consensus_peers.cc:597] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.103540  5410 tablet_replica.cc:333] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.104382  5410 raft_consensus.cc:2243] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.105372  5410 raft_consensus.cc:2272] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.108013  5410 tablet_replica.cc:333] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.108702  5410 raft_consensus.cc:2243] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.109632  5410 raft_consensus.cc:2272] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.112100  5410 tablet_replica.cc:333] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.112557  5410 raft_consensus.cc:2243] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.112919  5410 raft_consensus.cc:2272] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.114923  5410 tablet_replica.cc:333] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.115401  5410 raft_consensus.cc:2243] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.115772  5410 raft_consensus.cc:2272] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.117477  5410 tablet_replica.cc:333] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.118156  5410 raft_consensus.cc:2243] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.119024  5410 raft_consensus.cc:2272] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.120716  5410 tablet_replica.cc:333] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.121143  5410 raft_consensus.cc:2243] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.121474  5410 raft_consensus.cc:2272] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.123363  5410 tablet_replica.cc:333] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.123822  5410 raft_consensus.cc:2243] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.124148  5410 raft_consensus.cc:2272] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.126009  5410 tablet_replica.cc:333] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.126508  5410 raft_consensus.cc:2243] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.126986  5410 raft_consensus.cc:2272] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.129158  5410 tablet_replica.cc:333] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
W20260709 13:21:39.129294 15410 consensus_peers.cc:597] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.129694  5410 raft_consensus.cc:2243] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.130156  5410 raft_consensus.cc:2272] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.132416  5410 tablet_replica.cc:333] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.133097  5410 raft_consensus.cc:2243] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.133987  5410 raft_consensus.cc:2272] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.135987  5410 tablet_replica.cc:333] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.136502  5410 raft_consensus.cc:2243] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.136893  5410 raft_consensus.cc:2272] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.139132  5410 tablet_replica.cc:333] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.139669  5410 raft_consensus.cc:2243] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.140079  5410 raft_consensus.cc:2272] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.142011  5410 tablet_replica.cc:333] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.142517  5410 raft_consensus.cc:2243] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.143414  5410 raft_consensus.cc:2272] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.145457  5410 tablet_replica.cc:333] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.146023  5410 raft_consensus.cc:2243] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.146443  5410 raft_consensus.cc:2272] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.148324  5410 tablet_replica.cc:333] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.148838  5410 raft_consensus.cc:2243] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.149641  5410 raft_consensus.cc:2272] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.152073  5410 tablet_replica.cc:333] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.152786  5410 raft_consensus.cc:2243] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:39.153213 15484 consensus_peers.cc:597] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.153393  5410 raft_consensus.cc:2272] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.156713  5410 tablet_replica.cc:333] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.157441  5410 raft_consensus.cc:2243] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:39.157800 15336 consensus_peers.cc:597] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.157948  5410 raft_consensus.cc:2272] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.160593  5410 tablet_replica.cc:333] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.161293  5410 raft_consensus.cc:2243] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.161806  5410 raft_consensus.cc:2272] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.162854 15336 consensus_peers.cc:597] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.164312  5410 tablet_replica.cc:333] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: stopping tablet replica
I20260709 13:21:39.164840  5410 raft_consensus.cc:2243] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.165601  5410 raft_consensus.cc:2272] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.171034 15410 consensus_peers.cc:597] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:39.178488 15410 consensus_peers.cc:597] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:39.183156 15484 consensus_peers.cc:597] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:39.220212 15410 consensus_peers.cc:597] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.241222  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
W20260709 13:21:39.315308 15484 consensus_peers.cc:597] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.329677  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:39.340000 16088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:39.345350 16089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:39.379300 16092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:39.415179 15336 consensus_peers.cc:597] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.417076  5410 server_base.cc:1061] running on GCE node
I20260709 13:21:39.421607  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:39.421977  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:39.422454  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603299422432 us; error 0 us; skew 500 ppm
W20260709 13:21:39.421947 15336 consensus_peers.cc:597] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.423465  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260709 13:21:39.424144 15410 consensus_peers.cc:597] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.426484  5410 webserver.cc:533] Webserver started at http://127.5.72.129:46637/ using document root <none> and password file <none>
I20260709 13:21:39.427104  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:39.427294  5410 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:21:39.430802  5410 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:21:39.439168 16101 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:39.440060  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:21:39.440407  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "88eead6a17f54dd88c723744e0aa5b50"
format_stamp: "Formatted at 2026-07-09 13:21:25 on dist-test-slave-n5kz"
I20260709 13:21:39.440905  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:39.463583  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:39.464799  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:39.466306  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:39.470966 16108 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:21:39.487135 15410 consensus_peers.cc:597] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:39.496215 15410 consensus_peers.cc:597] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.744549  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:21:39.745012  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.275s	user 0.005s	sys 0.004s
I20260709 13:21:39.745370  5410 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:21:39.752238 16108 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.767508 16108 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.768617 16108 tablet_bootstrap.cc:492] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.769312 16108 ts_tablet_manager.cc:1403] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:21:39.772140 16108 raft_consensus.cc:359] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.772825 16108 raft_consensus.cc:740] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.773538 16108 consensus_queue.cc:260] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.775916 16108 ts_tablet_manager.cc:1434] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:39.776690 16108 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.790951 16108 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.791903 16108 tablet_bootstrap.cc:492] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.792546 16108 ts_tablet_manager.cc:1403] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 13:21:39.794957 16108 raft_consensus.cc:359] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:39.795610 16108 raft_consensus.cc:740] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.796206 16108 consensus_queue.cc:260] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:39.797876 16108 ts_tablet_manager.cc:1434] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 13:21:39.798609 16108 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.818115 16108 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.819072 16108 tablet_bootstrap.cc:492] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.819721 16108 ts_tablet_manager.cc:1403] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.004s
I20260709 13:21:39.822088 16108 raft_consensus.cc:359] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.822746 16108 raft_consensus.cc:740] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.823422 16108 consensus_queue.cc:260] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.825177 16108 ts_tablet_manager.cc:1434] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:21:39.825929 16108 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.840608 16108 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.841562 16108 tablet_bootstrap.cc:492] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.842142 16108 ts_tablet_manager.cc:1403] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:21:39.844182 16108 raft_consensus.cc:359] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.844641 16108 raft_consensus.cc:740] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.845144 16108 consensus_queue.cc:260] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.846681 16108 ts_tablet_manager.cc:1434] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:21:39.847364 16108 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.863607 16108 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.864550 16108 tablet_bootstrap.cc:492] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.865181 16108 ts_tablet_manager.cc:1403] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.005s
I20260709 13:21:39.867241 16108 raft_consensus.cc:359] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.867726 16108 raft_consensus.cc:740] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.868466 16108 consensus_queue.cc:260] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.869899 16108 ts_tablet_manager.cc:1434] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:21:39.870545 16108 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.887490 16108 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.888463 16108 tablet_bootstrap.cc:492] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.889109 16108 ts_tablet_manager.cc:1403] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:21:39.891183 16108 raft_consensus.cc:359] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:39.891785 16108 raft_consensus.cc:740] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.892467 16108 consensus_queue.cc:260] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:39.894250 16108 ts_tablet_manager.cc:1434] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.000s
I20260709 13:21:39.894981 16108 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.907231 16108 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.907999 16108 tablet_bootstrap.cc:492] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.908681 16108 ts_tablet_manager.cc:1403] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:21:39.910904 16108 raft_consensus.cc:359] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:39.911480 16108 raft_consensus.cc:740] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.912509 16108 consensus_queue.cc:260] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:39.914536 16108 ts_tablet_manager.cc:1434] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:21:39.917547 16108 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.928682 16108 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.929502 16108 tablet_bootstrap.cc:492] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.930123 16108 ts_tablet_manager.cc:1403] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:21:39.932495 16108 raft_consensus.cc:359] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:39.933174 16108 raft_consensus.cc:740] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.933830 16108 consensus_queue.cc:260] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:39.935592 16108 ts_tablet_manager.cc:1434] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:21:39.936317 16108 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.973392 16108 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:39.974411 16108 tablet_bootstrap.cc:492] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:39.975059 16108 ts_tablet_manager.cc:1403] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.039s	user 0.028s	sys 0.008s
I20260709 13:21:39.978830 16108 raft_consensus.cc:359] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.979514 16108 raft_consensus.cc:740] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.980335 16108 consensus_queue.cc:260] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:39.981001 16082 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:39.981564 16082 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:39.983721 16082 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:39.984504 16108 ts_tablet_manager.cc:1434] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:21:39.985430 16108 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:39.993141 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:39.999311  5410 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:21:39.999743  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.254s	user 0.248s	sys 0.000s
W20260709 13:21:40.006798 15336 leader_election.cc:336] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111)
I20260709 13:21:40.007344 15336 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.008121 16082 raft_consensus.cc:2749] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.013728 16108 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:40.014791 16108 tablet_bootstrap.cc:492] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:40.015522 16108 ts_tablet_manager.cc:1403] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.008s
I20260709 13:21:40.017940 16108 raft_consensus.cc:359] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.018458 16108 raft_consensus.cc:740] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.019157 16108 consensus_queue.cc:260] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.022029 16108 ts_tablet_manager.cc:1434] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:40.022747 16108 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:40.042405 16108 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:40.045221 16108 tablet_bootstrap.cc:492] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:40.047196 16108 ts_tablet_manager.cc:1403] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.007s
I20260709 13:21:40.053066 16108 raft_consensus.cc:359] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.055054 16108 raft_consensus.cc:740] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.057394 16108 consensus_queue.cc:260] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.061966 16108 ts_tablet_manager.cc:1434] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:21:40.064059 16108 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:40.065081 16039 raft_consensus.cc:493] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.066789 16039 raft_consensus.cc:515] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.073123 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:40.087201 15410 leader_election.cc:336] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111)
I20260709 13:21:40.087802 15410 leader_election.cc:304] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.088850 16039 leader_election.cc:290] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.089004 16029 raft_consensus.cc:2749] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.093230 16108 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:40.094321 16108 tablet_bootstrap.cc:492] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:40.095037 16108 ts_tablet_manager.cc:1403] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.032s	user 0.023s	sys 0.003s
I20260709 13:21:40.097344 16108 raft_consensus.cc:359] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.097892 16108 raft_consensus.cc:740] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.098657 16108 consensus_queue.cc:260] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.099543 16039 raft_consensus.cc:493] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.100104 16039 raft_consensus.cc:515] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.100942 16108 ts_tablet_manager.cc:1434] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:21:40.101786 16108 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:40.102213 16039 leader_election.cc:290] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.103390 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:40.112318 15410 leader_election.cc:336] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111)
I20260709 13:21:40.112879 15410 leader_election.cc:304] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.114667 16039 raft_consensus.cc:2749] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.128082 16108 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:40.129159 16108 tablet_bootstrap.cc:492] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:40.129710 16108 ts_tablet_manager.cc:1403] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.004s
I20260709 13:21:40.132315 16108 raft_consensus.cc:359] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:40.132783 16108 raft_consensus.cc:740] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.133307 16108 consensus_queue.cc:260] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:40.134838 16108 ts_tablet_manager.cc:1434] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:21:40.135397 16108 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:40.152825 16108 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:40.153852 16179 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:43313 every 8 connection(s)
I20260709 13:21:40.153697  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:43313
I20260709 13:21:40.155071 16108 tablet_bootstrap.cc:492] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:40.155872 16108 ts_tablet_manager.cc:1403] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:21:40.158521 16082 raft_consensus.cc:493] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.159739 16082 raft_consensus.cc:515] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.159881 16096 raft_consensus.cc:493] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.158836 16108 raft_consensus.cc:359] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.160476 16096 raft_consensus.cc:515] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.161104 16108 raft_consensus.cc:740] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.161687 16082 leader_election.cc:290] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:40.163982 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.161974 16108 consensus_queue.cc:260] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.163213 16096 leader_election.cc:290] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.165192 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.166471 16108 ts_tablet_manager.cc:1434] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:21:40.167645 16108 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:40.172398 16044 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.173090 16082 raft_consensus.cc:493] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.173655 16082 raft_consensus.cc:515] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.174885 16096 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.175431 16096 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.173105 16044 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.176494 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.177716 16096 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.177865 16044 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:40.178937 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.180047 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.189247 15982 raft_consensus.cc:493] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.190141 15982 raft_consensus.cc:515] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.193037 15982 leader_election.cc:290] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:40.193795 15659 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.194058 16095 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.194424 15659 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.197082 15659 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.194634 16095 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.197962 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.198894 15436 raft_consensus.cc:2468] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.200067 15484 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.205473 16095 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.207176 16096 raft_consensus.cc:2804] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.207631 16096 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.207751 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.208046 16096 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.209594 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.217180 16096 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.218385 15516 raft_consensus.cc:2393] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:40.220372 16082 leader_election.cc:290] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.223124 16096 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.224121 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:40.225281 15436 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.232672 15659 raft_consensus.cc:493] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.233426 15659 raft_consensus.cc:515] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.235769 15659 leader_election.cc:290] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.236959 15436 raft_consensus.cc:2468] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:40.237207 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.238050 15440 raft_consensus.cc:2468] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.239213 15484 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.240588 15484 leader_election.cc:304] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.241267 16096 raft_consensus.cc:2804] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.242002 16079 raft_consensus.cc:2804] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.242455 16079 raft_consensus.cc:493] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.242920 16079 raft_consensus.cc:3060] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.251385 16079 raft_consensus.cc:515] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.253862 16079 leader_election.cc:290] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.254413 16096 raft_consensus.cc:697] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:40.255460 16096 consensus_queue.cc:237] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.257778 16188 raft_consensus.cc:493] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.258363 16188 raft_consensus.cc:515] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.260888 16108 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:40.262549 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:40.263302 16082 raft_consensus.cc:493] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.263934 15440 raft_consensus.cc:3060] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.264055 16082 raft_consensus.cc:515] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.266255 16082 leader_election.cc:290] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.267998 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.267596 16188 leader_election.cc:290] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.274891 16108 tablet_bootstrap.cc:492] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:40.275650 16108 ts_tablet_manager.cc:1403] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.109s	user 0.040s	sys 0.026s
I20260709 13:21:40.288226 15190 catalog_manager.cc:5697] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:40.291028 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.292879 15440 raft_consensus.cc:2468] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:40.298287 16188 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.298869 16188 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.301348 16188 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.302116 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.302654 16108 raft_consensus.cc:359] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.303395 16108 raft_consensus.cc:740] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.302800 16082 raft_consensus.cc:493] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.303996 16082 raft_consensus.cc:515] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.304080 16108 consensus_queue.cc:260] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.305950 16108 ts_tablet_manager.cc:1434] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:21:40.306154 16082 leader_election.cc:290] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.307335 16082 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
W20260709 13:21:40.308027 15410 leader_election.cc:343] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.308508 15410 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.307812 16082 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.309257 15484 leader_election.cc:304] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.309392 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.310045 16079 raft_consensus.cc:2804] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.310153 16039 raft_consensus.cc:2749] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.310592 16079 raft_consensus.cc:697] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:40.310847 16108 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:40.311204 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.311812 15516 raft_consensus.cc:2468] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.311699 16079 consensus_queue.cc:237] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.308458 16188 raft_consensus.cc:493] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.319165 16188 raft_consensus.cc:515] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.320013 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.320717 16152 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.321132 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.321703 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.315128 15333 leader_election.cc:304] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.333580 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.328507 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99ac0daff22426fb7dfa484f28c548d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.335240 16188 leader_election.cc:290] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.336100 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.328569 16044 raft_consensus.cc:2804] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.337411 16044 raft_consensus.cc:493] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.337810 16188 raft_consensus.cc:493] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.341086 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.341740 15369 raft_consensus.cc:2468] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.343616 15481 leader_election.cc:304] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.316570 16187 raft_consensus.cc:493] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.348268 16187 raft_consensus.cc:515] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.327778 16096 raft_consensus.cc:493] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.347514 16082 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.337832 16044 raft_consensus.cc:3060] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.337824 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.350076 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.338388 16188 raft_consensus.cc:515] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.335260 16149 raft_consensus.cc:2468] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.355800 15659 raft_consensus.cc:2804] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.369588 15659 raft_consensus.cc:493] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.358770 16148 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.352839 16096 raft_consensus.cc:515] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.371714 15659 raft_consensus.cc:3060] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.372757 15484 leader_election.cc:304] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:40.373000 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:40.373793 15484 leader_election.cc:343] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:40.374636 15484 leader_election.cc:343] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:40.376112 15484 leader_election.cc:343] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.376477 15484 leader_election.cc:304] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:40.358943 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.382665 15513 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.354421 16187 leader_election.cc:290] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.383533 16194 raft_consensus.cc:2804] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.388491 16194 raft_consensus.cc:493] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.360524 16029 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.394647 15516 raft_consensus.cc:2468] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.398284 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.358937 16150 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.390079 16194 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.401172 16079 raft_consensus.cc:2749] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.401373 16039 raft_consensus.cc:493] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.394552 16187 raft_consensus.cc:493] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
W20260709 13:21:40.464440 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 87044.1us
I20260709 13:21:40.428828 16095 raft_consensus.cc:493] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.466166 16187 raft_consensus.cc:515] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.471590 16201 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.472576 16201 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.473469 16187 leader_election.cc:290] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:40.468999 16202 raft_consensus.cc:493] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.434657 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.400836 16029 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.476178 15982 raft_consensus.cc:493] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.475660 16095 raft_consensus.cc:515] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.478045 15982 raft_consensus.cc:515] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.478546 16096 leader_election.cc:290] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.479183 16096 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.479651 16096 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.480120 15982 leader_election.cc:290] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:40.480670 16201 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.447065 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.483536 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.484184 15369 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.437980 16188 leader_election.cc:290] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
W20260709 13:21:40.518383 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 53169.2us
W20260709 13:21:40.492877 15336 leader_election.cc:343] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.474553 16187 raft_consensus.cc:493] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.520648 15336 leader_election.cc:304] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:40.521938 15484 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:40.463842 16039 raft_consensus.cc:515] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.479666 16095 leader_election.cc:290] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.524454 15484 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
W20260709 13:21:40.527336 15484 leader_election.cc:343] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.529289 16082 raft_consensus.cc:2749] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.528687 15333 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.530088 16044 raft_consensus.cc:515] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.532658 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.533833 16044 leader_election.cc:290] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.533763 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.535198 16204 raft_consensus.cc:2804] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.535645 16204 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.536001 16204 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.536808 16039 leader_election.cc:290] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.531673 16187 raft_consensus.cc:515] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.540092 16029 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.541886 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.542603 16152 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.491082 16202 raft_consensus.cc:515] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.542632 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.545360 16202 leader_election.cc:290] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.546092 16205 raft_consensus.cc:2804] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.546515 16205 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.546877 16205 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.548321 16108 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:40.548629 16192 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.549345 16108 tablet_bootstrap.cc:492] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:40.550124 16108 ts_tablet_manager.cc:1403] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.239s	user 0.041s	sys 0.007s
I20260709 13:21:40.551769 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.552663 15516 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.521525 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.531059 15333 leader_election.cc:304] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.525334 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:40.568365 15336 leader_election.cc:343] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.569173 15336 leader_election.cc:304] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:40.569597 15659 raft_consensus.cc:515] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:40.572227 15484 leader_election.cc:343] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:40.574867 15333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 43884.1us
I20260709 13:21:40.575532 16044 raft_consensus.cc:2804] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.575929 16044 raft_consensus.cc:493] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.576325 16044 raft_consensus.cc:3060] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.577529 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.559001 16082 raft_consensus.cc:2804] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:40.579005 15336 leader_election.cc:343] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.584381 15336 leader_election.cc:304] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.578159 15436 raft_consensus.cc:2468] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.555526 16039 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.579124 15659 leader_election.cc:290] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.555053 16095 raft_consensus.cc:493] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.568720 16096 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.549355 16192 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.588464 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.583393 16082 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.592576 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.574498 16146 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.600059 15367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.617658 15367 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.589787 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:40.612372 15336 leader_election.cc:343] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.578169 16200 raft_consensus.cc:2749] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.622056 15659 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.621657 15484 leader_election.cc:304] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.614974 15369 raft_consensus.cc:2468] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:40.623080 15659 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.589442 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.611254 15333 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.621260 15336 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:40.624754 15370 raft_consensus.cc:2468] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:40.596036 16187 leader_election.cc:290] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.634645 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.635336 15370 raft_consensus.cc:3060] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.592757 16039 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.618577 16082 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.620071 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.638657 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.640717 16218 raft_consensus.cc:2804] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.659471 16218 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.641410 16202 raft_consensus.cc:2804] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.659924 16218 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.660211 16202 raft_consensus.cc:493] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.660687 16202 raft_consensus.cc:3060] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.660884 16039 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.642968 16180 heartbeater.cc:344] Connected to a master server at 127.5.72.190:43355
I20260709 13:21:40.680327 16180 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:40.681314 16180 heartbeater.cc:507] Master 127.5.72.190:43355 requested a full tablet report, sending...
I20260709 13:21:40.682129 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.643891 16192 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.646013 16108 raft_consensus.cc:359] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.682951 16108 raft_consensus.cc:740] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.683940 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.614737 16095 raft_consensus.cc:515] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.655766 15481 leader_election.cc:304] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.685571 15481 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.685897 16095 leader_election.cc:290] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.686851 16208 raft_consensus.cc:493] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.686919 15407 leader_election.cc:304] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
W20260709 13:21:40.687747 15407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 53003.6us
I20260709 13:21:40.687538 16208 raft_consensus.cc:515] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.687898 16152 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:40.626564 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.689735 16208 leader_election.cc:290] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.630913 16203 raft_consensus.cc:493] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.690428 16203 raft_consensus.cc:515] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.690925 16219 raft_consensus.cc:2804] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.691360 16219 raft_consensus.cc:493] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.691706 16219 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.692258 16096 raft_consensus.cc:2804] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.692664 16203 leader_election.cc:290] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:40.641831 16044 raft_consensus.cc:515] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.644930 16187 raft_consensus.cc:2749] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.694905 16044 leader_election.cc:290] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.695253 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.695708 15364 raft_consensus.cc:2393] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 2: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:40.696512 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.662451 15511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:40.697189 16151 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.697369 15511 raft_consensus.cc:3060] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.640940 16200 raft_consensus.cc:2804] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.698851 16200 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.690481 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.699800 16152 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.700145 16208 raft_consensus.cc:2804] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.667850 16218 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.671914 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba45fbc82014336a9bddd17603ed390" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.674798 16202 raft_consensus.cc:515] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.674458 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.675366 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.677238 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.677512 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.662864 16216 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.683763 16108 consensus_queue.cc:260] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.689263 15437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.699293 16200 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.665714 15513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
I20260709 13:21:40.673506 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffef2ecb155c41daba889a3314b66527" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.671795 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.702821 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
W20260709 13:21:40.703778 15484 leader_election.cc:343] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.724987 15484 leader_election.cc:304] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:40.700560 16208 raft_consensus.cc:493] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.709666 15407 leader_election.cc:304] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.717722 16108 ts_tablet_manager.cc:1434] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.167s	user 0.007s	sys 0.000s
I20260709 13:21:40.734575 15370 raft_consensus.cc:2468] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:40.692667 16096 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.646934 15659 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.725433 16205 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.751418 16204 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.752501 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:40.752916 15410 leader_election.cc:343] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.759841 15481 leader_election.cc:304] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.761500 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.761705 16146 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:40.753917 16218 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.762051 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.762225 15364 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.762174 15659 raft_consensus.cc:2804] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.765138 16205 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.765348 15659 raft_consensus.cc:697] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:40.736028 16202 leader_election.cc:290] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.754946 16208 raft_consensus.cc:3060] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.758090 16108 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:40.766215 15659 consensus_queue.cc:237] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.767373 15511 raft_consensus.cc:2468] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:40.719430 16149 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.759491 16096 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.762514 16212 raft_consensus.cc:2804] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.774845 16212 raft_consensus.cc:493] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.775360 16212 raft_consensus.cc:3060] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.762802 15441 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 1.
I20260709 13:21:40.758412 16220 raft_consensus.cc:3060] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.782044 16082 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.784441 16082 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.785168 16212 raft_consensus.cc:515] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.786175 16220 raft_consensus.cc:2749] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.726009 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.764097 15440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:40.787073 15481 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.787585 15365 raft_consensus.cc:2468] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:40.787883 16209 raft_consensus.cc:2804] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.729163 15190 catalog_manager.cc:5697] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:40.756227 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.788275 16209 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.788776 16209 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.788160 16194 raft_consensus.cc:515] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.790045 15513 raft_consensus.cc:2393] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
W20260709 13:21:40.757807 15484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 54070.4us
I20260709 13:21:40.790422 16212 leader_election.cc:290] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
W20260709 13:21:40.791168 15484 leader_election.cc:343] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.774761 16208 raft_consensus.cc:515] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.792227 15440 raft_consensus.cc:2393] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 2: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:40.792752 16194 leader_election.cc:290] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
W20260709 13:21:40.759109 15407 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.130:46479, user_credentials={real_user=slave}} blocked reactor thread for 69194.1us
I20260709 13:21:40.792135 15484 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
W20260709 13:21:40.783254 15410 leader_election.cc:343] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:40.794445 15484 leader_election.cc:343] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.722119 16095 raft_consensus.cc:493] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.798882 15410 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.801846 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.802413 15370 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.802873 15410 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.808370 15370 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:40.717410 16145 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.809655 16227 raft_consensus.cc:2804] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.811419 15516 raft_consensus.cc:2393] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
W20260709 13:21:40.812011 15410 leader_election.cc:343] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.812740 16227 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.816461 16227 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.793396 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.799741 16220 raft_consensus.cc:2804] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.826449 15481 leader_election.cc:304] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.725751 16219 raft_consensus.cc:515] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.830530 16224 raft_consensus.cc:2804] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.831043 16224 raft_consensus.cc:697] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:40.824491 15407 leader_election.cc:304] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.823344 16212 raft_consensus.cc:2804] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.825106 15365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.841763 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:40.842451 15441 raft_consensus.cc:3060] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.828568 15363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.842782 16029 raft_consensus.cc:2804] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.843340 16029 raft_consensus.cc:493] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.843717 16029 raft_consensus.cc:3060] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.841831 16219 leader_election.cc:290] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.845821 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9fb81bccaf408cba956547d37caf3f" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.837674 15362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.832535 16220 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.847426 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:40.846757 15333 leader_election.cc:304] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.823901 16095 raft_consensus.cc:515] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.838204 16212 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.866737 16187 raft_consensus.cc:2804] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.860745 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.867916 15440 raft_consensus.cc:1275] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:40.858248 15362 raft_consensus.cc:3060] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.868027 16146 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.870818 15365 raft_consensus.cc:2393] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 2: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:40.848881 16224 consensus_queue.cc:237] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.872677 16096 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.724239 16216 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.854658 15510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:40.857144 16220 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.880249 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.843351 15363 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
W20260709 13:21:40.882974 15410 leader_election.cc:343] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.888062 16192 consensus_queue.cc:1048] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:40.888514 16095 leader_election.cc:290] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.907778 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" is_pre_election: true
I20260709 13:21:40.893909 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.895390 15361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.899837 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:40.911578 15517 raft_consensus.cc:2393] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:40.911715 15370 raft_consensus.cc:2468] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:40.899996 15441 raft_consensus.cc:2468] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:40.902562 16096 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.903046 15516 raft_consensus.cc:2468] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:40.904083 16204 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.876462 16212 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.851764 16200 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.888867 15510 raft_consensus.cc:2393] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:40.901142 16227 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:40.882268 16187 raft_consensus.cc:697] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:40.915534 15366 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.928763 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.929176 16216 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.930271 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:40.919763 15407 leader_election.cc:304] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.920708 15484 leader_election.cc:304] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:40.921686 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:40.922411 15440 raft_consensus.cc:1275] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:40.915822 15437 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 1.
I20260709 13:21:40.922948 16200 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:40.923560 15369 raft_consensus.cc:2393] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:40.927142 15361 raft_consensus.cc:2393] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 2: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
W20260709 13:21:40.933252 15410 leader_election.cc:343] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.934967 16208 leader_election.cc:290] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.936661 16152 raft_consensus.cc:3060] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.936791 16146 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:40.938201 15362 raft_consensus.cc:2468] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 2.
I20260709 13:21:40.939049 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.939694 16147 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.926195 16029 raft_consensus.cc:515] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.947827 16219 raft_consensus.cc:2804] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.946564 16209 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.948372 16219 raft_consensus.cc:697] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:40.949237 16219 consensus_queue.cc:237] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
W20260709 13:21:40.947454 15410 leader_election.cc:343] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.951424 15333 leader_election.cc:304] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.952135 15190 catalog_manager.cc:5697] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:40.956900 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" is_pre_election: true
W20260709 13:21:40.959494 15410 leader_election.cc:343] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.961480 15407 leader_election.cc:304] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:40.962900 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5546361e1c64e36951aeb942521a72b" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.964512 15370 raft_consensus.cc:3060] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.965485 16212 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:40.966974 15363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:40.967612 15363 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.968411 16095 raft_consensus.cc:2804] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.968832 16095 raft_consensus.cc:493] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.969190 16095 raft_consensus.cc:3060] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.957780 16200 raft_consensus.cc:2804] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.969892 16200 raft_consensus.cc:697] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:40.967002 15442 raft_consensus.cc:2393] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 2: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:40.970899 16200 consensus_queue.cc:237] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.959538 16096 consensus_queue.cc:1048] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:21:40.954283 16209 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.978801 15366 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:40.979205 16029 leader_election.cc:290] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:40.980921 15481 leader_election.cc:304] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:40.959872 16207 raft_consensus.cc:2804] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:40.981647 16207 raft_consensus.cc:493] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:40.981739 16204 raft_consensus.cc:2804] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:40.981814 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.982317 16204 raft_consensus.cc:697] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:40.982719 16143 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:40.983408 16204 consensus_queue.cc:237] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.984797 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:40.986822 16227 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.994486 15193 ts_manager.cc:194] Re-registered known tserver with Master: 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:40.995326 16187 consensus_queue.cc:237] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:40.997568 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
W20260709 13:21:41.000262 15410 leader_election.cc:343] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.994374 16095 raft_consensus.cc:515] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.008523 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.009263 16144 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.009892 16095 leader_election.cc:290] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:41.009775 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.006134 15407 leader_election.cc:304] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.011816 15370 raft_consensus.cc:2468] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 2.
I20260709 13:21:40.982087 16207 raft_consensus.cc:3060] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:41.023590 15410 leader_election.cc:343] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:41.023635 15336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:43313, user_credentials={real_user=slave}} blocked reactor thread for 402440us
I20260709 13:21:40.964601 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
I20260709 13:21:41.031543 15407 leader_election.cc:304] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.018591 16095 raft_consensus.cc:2804] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.033635 16095 raft_consensus.cc:697] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:41.034538 16095 consensus_queue.cc:237] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.036522 16143 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 2.
I20260709 13:21:41.018582 15363 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:41.039393 16212 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:41.040329 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3" is_pre_election: true
I20260709 13:21:41.045374 15370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260309c1df6b490ea4502dd13e14f692" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:41.048183 15370 raft_consensus.cc:3060] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.049566 15336 leader_election.cc:304] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:41.046628 15369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:41.050571 15369 raft_consensus.cc:2393] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:41.046010 15517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:41.051016 16044 raft_consensus.cc:2804] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:41.051213 15517 raft_consensus.cc:2393] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:41.051301 16044 raft_consensus.cc:493] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:41.051682 16044 raft_consensus.cc:3060] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:41.052940 15336 leader_election.cc:343] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.053308 15336 leader_election.cc:304] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.047427 15366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f34db7828d184304ab99b4216ac1798e" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:41.054471 15366 raft_consensus.cc:3060] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.060495 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.061295 16149 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.071321 16044 raft_consensus.cc:515] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.077252 15370 raft_consensus.cc:2468] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 2.
I20260709 13:21:41.079035 15410 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:41.078449 15407 leader_election.cc:304] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.074747 15481 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:41.084403 16225 raft_consensus.cc:2804] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.085005 16225 raft_consensus.cc:697] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:41.085901 16225 consensus_queue.cc:237] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.080463 16218 raft_consensus.cc:2749] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.090893 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.091789 16145 raft_consensus.cc:2393] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 2: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:41.048748 16216 raft_consensus.cc:2804] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.093730 16044 leader_election.cc:290] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:41.043790 16146 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 1.
I20260709 13:21:41.096256 16108 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:41.091866 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de52504e9e244dc99cc89e67e9281f3d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50" is_pre_election: true
W20260709 13:21:41.099700 15336 leader_election.cc:343] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:41.101532 15336 leader_election.cc:343] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:41.104812 15336 leader_election.cc:343] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.097002 16223 raft_consensus.cc:2804] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:21:41.105948 15336 leader_election.cc:343] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.106463 15336 leader_election.cc:304] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:41.106909 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af51ee9eaabb4a29a989844d36c3f423" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.107416 16044 raft_consensus.cc:2749] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.108474 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.108893 15336 leader_election.cc:304] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:41.109499 16150 raft_consensus.cc:2393] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:41.109977 16044 raft_consensus.cc:2764] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:21:41.083577 16212 raft_consensus.cc:2804] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.147966 16108 tablet_bootstrap.cc:492] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:41.149072 16108 ts_tablet_manager.cc:1403] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.391s	user 0.021s	sys 0.003s
I20260709 13:21:41.145927 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.112833 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.111583 16216 raft_consensus.cc:697] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:41.114602 16223 raft_consensus.cc:697] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
W20260709 13:21:41.161602 15336 leader_election.cc:343] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.162081 15336 leader_election.cc:304] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:41.153188 16212 raft_consensus.cc:697] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:41.171379 16152 raft_consensus.cc:1275] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.172053 16147 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
W20260709 13:21:41.173707 15336 leader_election.cc:343] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.172300 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.178524 15336 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.153375 16220 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:41.184257 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
W20260709 13:21:41.187371 15484 leader_election.cc:343] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.172899 16218 raft_consensus.cc:2749] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.188060 15410 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:41.188861 15436 raft_consensus.cc:2393] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 2: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:41.191639 16216 consensus_queue.cc:237] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.201668 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:41.201902 15193 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52944
I20260709 13:21:41.189075 15484 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:41.174600 15366 raft_consensus.cc:2468] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 2.
I20260709 13:21:41.180053 16145 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.204684 15191 catalog_manager.cc:5697] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.218981 15516 raft_consensus.cc:2393] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
I20260709 13:21:41.218932 15191 catalog_manager.cc:5697] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.220585 15191 catalog_manager.cc:5697] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.183599 16200 raft_consensus.cc:2749] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.204574 16144 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:41.228225 15407 leader_election.cc:304] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.218575 16202 raft_consensus.cc:2804] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:21:41.237866 15484 leader_election.cc:343] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:41.239286 15484 consensus_peers.cc:597] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.243774 15484 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:41.246948 16192 raft_consensus.cc:2749] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.247160 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.210815 16223 consensus_queue.cc:237] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:41.249550 16108 raft_consensus.cc:359] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.241802 16208 raft_consensus.cc:2804] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.252432 15336 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.252511 15659 consensus_queue.cc:1048] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:41.260624 15336 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:41.254349 15484 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:41.256831 16108 raft_consensus.cc:740] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.263027 16044 raft_consensus.cc:2749] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.265489 15659 raft_consensus.cc:2804] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.266016 15659 raft_consensus.cc:697] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:41.267714 16108 consensus_queue.cc:260] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.268924 16108 ts_tablet_manager.cc:1434] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.119s	user 0.007s	sys 0.000s
W20260709 13:21:41.269572 16234 log.cc:927] Time spent T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: Append to log took a long time: real 0.117s	user 0.000s	sys 0.005s
I20260709 13:21:41.254119 16202 raft_consensus.cc:697] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:41.271497 16108 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:41.273087 16144 raft_consensus.cc:3060] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.272897 15659 consensus_queue.cc:237] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:41.243644 16212 consensus_queue.cc:237] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:41.263924 16199 raft_consensus.cc:2804] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:41.287221 16199 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:41.286916 16202 consensus_queue.cc:237] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:41.287694 16199 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.267055 16208 raft_consensus.cc:697] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:41.289228 16208 consensus_queue.cc:237] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.291203 16180 heartbeater.cc:499] Master 127.5.72.190:43355 was elected leader, sending a full tablet report...
I20260709 13:21:41.263741 16218 raft_consensus.cc:2749] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.295723 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.296202 15366 raft_consensus.cc:1275] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.296383 16150 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:41.276916 16220 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:41.306926 15441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:41.313462 16150 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:41.315742 15193 catalog_manager.cc:5697] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 2 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:41.333570 15484 consensus_peers.cc:597] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.267511 16149 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d646a2b597824d24b522d9ef5ff5adc3 in term 2.
I20260709 13:21:41.348858 16207 raft_consensus.cc:515] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.359002 16207 leader_election.cc:290] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:41.360948 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.365139 15363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bde75c2a4c44b0beb23e22159b9fa2" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:41.365968 15363 raft_consensus.cc:2393] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate ee5629f4db5540b9a61ea4d7a180ee6b in this term.
I20260709 13:21:41.368192 16145 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:41.372841 15410 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:41.339457 16204 consensus_queue.cc:1048] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:21:41.381090 15442 raft_consensus.cc:2393] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5629f4db5540b9a61ea4d7a180ee6b in current term 2: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:41.374563 15484 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:41.386857 15410 leader_election.cc:304] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:41.383308 16227 raft_consensus.cc:2804] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.392941 16227 raft_consensus.cc:697] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Leader. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:41.393937 16227 consensus_queue.cc:237] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.388329 16211 raft_consensus.cc:2804] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.399562 16211 raft_consensus.cc:697] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:41.400465 16211 consensus_queue.cc:237] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:41.419620 15366 raft_consensus.cc:1275] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.425464 15193 catalog_manager.cc:5697] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 2 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.386004 16144 raft_consensus.cc:2468] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:41.426468 16199 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:41.390818 16208 raft_consensus.cc:2749] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.436730 16224 consensus_queue.cc:1048] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:21:41.467377 15442 raft_consensus.cc:1275] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.474728 16150 raft_consensus.cc:1275] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.475473 15441 raft_consensus.cc:2393] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in current term 2: Already voted for candidate d646a2b597824d24b522d9ef5ff5adc3 in this term.
I20260709 13:21:41.475972 15510 raft_consensus.cc:1275] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.490325 15191 catalog_manager.cc:5697] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.492126 15191 catalog_manager.cc:5697] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.493475 15369 raft_consensus.cc:1275] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.493677 15191 catalog_manager.cc:5697] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.495561 15191 catalog_manager.cc:5697] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:41.496690 15484 consensus_peers.cc:597] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.497562 16209 consensus_queue.cc:1048] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:41.499010 16187 consensus_queue.cc:1048] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:41.502360 16217 consensus_queue.cc:1048] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:41.511731 16205 consensus_queue.cc:1048] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:21:41.518672 15193 catalog_manager.cc:5697] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.533808 15193 catalog_manager.cc:5697] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.535276 15193 catalog_manager.cc:5697] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.541810 16108 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:41.544873 15336 leader_election.cc:304] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: d646a2b597824d24b522d9ef5ff5adc3
I20260709 13:21:41.548084 16200 raft_consensus.cc:2804] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:41.548722 16200 raft_consensus.cc:697] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:41.549789 16200 consensus_queue.cc:237] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.558928 16108 tablet_bootstrap.cc:492] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:41.559725 16108 ts_tablet_manager.cc:1403] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.288s	user 0.028s	sys 0.022s
I20260709 13:21:41.559633 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:41.561666 16199 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313)
I20260709 13:21:41.574245 15516 raft_consensus.cc:1275] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.591712 16044 consensus_queue.cc:1048] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:41.599664 15336 consensus_peers.cc:597] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.630244 16108 raft_consensus.cc:359] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.637655 16108 raft_consensus.cc:740] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.640686 15369 raft_consensus.cc:1275] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.650642 16108 consensus_queue.cc:260] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:41.658258 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "d646a2b597824d24b522d9ef5ff5adc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
I20260709 13:21:41.659281 16144 raft_consensus.cc:2393] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d646a2b597824d24b522d9ef5ff5adc3 in current term 2: Already voted for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in this term.
W20260709 13:21:41.662161 15336 consensus_peers.cc:597] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:41.664214 15410 consensus_peers.cc:597] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.650878 16208 consensus_queue.cc:1048] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:21:41.671556 15361 raft_consensus.cc:1275] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.672827 15410 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3; no voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:41.673489 16209 consensus_queue.cc:1048] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:41.650444 16253 log.cc:927] Time spent T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:21:41.677083 16199 raft_consensus.cc:2749] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:21:41.677640 15484 consensus_peers.cc:597] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:41.679140 16257 log.cc:927] Time spent T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:21:41.682837 16108 ts_tablet_manager.cc:1434] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.123s	user 0.008s	sys 0.000s
I20260709 13:21:41.683770 16108 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
W20260709 13:21:41.674093 15410 consensus_peers.cc:597] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:41.685451 15410 consensus_peers.cc:597] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:41.689186 15410 consensus_peers.cc:597] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:41.689754 15484 consensus_peers.cc:597] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:41.690469 15484 consensus_peers.cc:597] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:41.674793 15193 catalog_manager.cc:5697] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.772835 16150 raft_consensus.cc:3060] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:41.779397 15410 consensus_peers.cc:597] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:41.788195 15336 consensus_peers.cc:597] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:41.792739 15368 raft_consensus.cc:1275] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.793843 15370 raft_consensus.cc:1275] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.815021 16227 consensus_queue.cc:1048] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:41.805912 16212 consensus_queue.cc:1048] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:21:41.831709 15363 raft_consensus.cc:1275] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.833381 16209 consensus_queue.cc:1048] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:41.834795 16144 raft_consensus.cc:1275] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.838438 16212 consensus_queue.cc:1048] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:41.839305 16147 raft_consensus.cc:1275] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.830442 16150 raft_consensus.cc:1275] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:41.841286 15336 consensus_peers.cc:597] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:41.844218 15190 catalog_manager.cc:5697] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.848009 15190 catalog_manager.cc:5697] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "d646a2b597824d24b522d9ef5ff5adc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.848932 16144 raft_consensus.cc:1275] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.845288 15193 catalog_manager.cc:5697] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 1 to 2, leader changed from 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129) to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 2 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.872671 16227 consensus_queue.cc:1048] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:41.872041 16220 consensus_queue.cc:1048] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:21:41.876809 15366 raft_consensus.cc:1275] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.903375 15510 raft_consensus.cc:1275] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.900883 16192 consensus_queue.cc:1048] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:21:41.906682 16199 consensus_queue.cc:1048] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:41.908053 16144 raft_consensus.cc:1275] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.911221 16219 consensus_queue.cc:1048] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:41.939695 15366 raft_consensus.cc:1275] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.941515 16039 consensus_queue.cc:1048] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:41.948467 15364 raft_consensus.cc:1275] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:41.960438 15410 consensus_peers.cc:597] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:41.982215 15410 consensus_peers.cc:597] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:41.983610 15410 consensus_peers.cc:597] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:41.962538 16220 consensus_queue.cc:1048] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 13:21:41.985775 15410 consensus_peers.cc:597] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:41.992663 16148 raft_consensus.cc:1275] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:41.993495 15484 consensus_peers.cc:597] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:41.997192 16267 consensus_queue.cc:1048] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:42.000350 15441 raft_consensus.cc:1275] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:41.983580 16204 consensus_queue.cc:1048] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:21:42.002705 16267 consensus_queue.cc:1048] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:42.033155 16108 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.039208 16151 raft_consensus.cc:1275] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer d646a2b597824d24b522d9ef5ff5adc3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:42.168570 16227 consensus_queue.cc:1048] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:42.177953 16108 tablet_bootstrap.cc:492] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.178889 16108 ts_tablet_manager.cc:1403] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.495s	user 0.032s	sys 0.034s
W20260709 13:21:42.192845 15333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.132:41633, user_credentials={real_user=slave}} blocked reactor thread for 146842us
I20260709 13:21:42.222874 16108 raft_consensus.cc:359] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.223690 16108 raft_consensus.cc:740] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.232131 16108 consensus_queue.cc:260] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
W20260709 13:21:42.234045 15410 consensus_peers.cc:597] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:42.242792 16108 ts_tablet_manager.cc:1434] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 13:21:42.243942 16108 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
W20260709 13:21:42.265847 15484 consensus_peers.cc:597] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:42.269640 16147 raft_consensus.cc:1275] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:42.275655 16187 consensus_queue.cc:1048] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:42.288851 16108 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.289813 16108 tablet_bootstrap.cc:492] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.290349 16108 ts_tablet_manager.cc:1403] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.001s
I20260709 13:21:42.298244 16108 raft_consensus.cc:359] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.299031 16108 raft_consensus.cc:740] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.300457 16108 consensus_queue.cc:260] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.303395 16108 ts_tablet_manager.cc:1434] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:21:42.304243 16108 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.324668 15442 raft_consensus.cc:1275] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:42.327795 15336 consensus_peers.cc:597] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:42.328454 15336 consensus_peers.cc:597] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:42.329001 15336 consensus_peers.cc:597] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:42.338878 16108 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.339955 16108 tablet_bootstrap.cc:492] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.340669 16108 ts_tablet_manager.cc:1403] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.000s
I20260709 13:21:42.351867 16108 raft_consensus.cc:359] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
W20260709 13:21:42.329576 15336 consensus_peers.cc:597] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:42.352679 16108 raft_consensus.cc:740] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.353559 16108 consensus_queue.cc:260] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.356340 16108 ts_tablet_manager.cc:1434] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:21:42.357169 16108 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
W20260709 13:21:42.360630 15336 consensus_peers.cc:597] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:42.392098 16218 consensus_queue.cc:1048] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:42.396672 16108 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.397758 16108 tablet_bootstrap.cc:492] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.398711 16108 ts_tablet_manager.cc:1403] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.012s
I20260709 13:21:42.401330 16108 raft_consensus.cc:359] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.402032 16108 raft_consensus.cc:740] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.402740 16108 consensus_queue.cc:260] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.405417 16108 ts_tablet_manager.cc:1434] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:21:42.406865 16108 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.428030 16151 raft_consensus.cc:3060] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:42.428956 16108 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.429904 16108 tablet_bootstrap.cc:492] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.430521 16108 ts_tablet_manager.cc:1403] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.000s
I20260709 13:21:42.434229 16108 raft_consensus.cc:359] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.434919 16108 raft_consensus.cc:740] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.437844 16108 consensus_queue.cc:260] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.444415 16108 ts_tablet_manager.cc:1434] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:21:42.445257 16108 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.477860 16151 raft_consensus.cc:3060] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:42.481159 16108 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.482180 16108 tablet_bootstrap.cc:492] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.482841 16108 ts_tablet_manager.cc:1403] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.007s
I20260709 13:21:42.485473 16108 raft_consensus.cc:359] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.486191 16108 raft_consensus.cc:740] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.487069 16108 consensus_queue.cc:260] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.493957 16108 ts_tablet_manager.cc:1434] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:42.494791 16108 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.515192 16108 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.516036 16108 tablet_bootstrap.cc:492] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.516675 16108 ts_tablet_manager.cc:1403] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.000s
I20260709 13:21:42.518808 16108 raft_consensus.cc:359] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.519292 16108 raft_consensus.cc:740] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.519857 16108 consensus_queue.cc:260] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.521997 16108 ts_tablet_manager.cc:1434] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:21:42.523888 16108 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.539624 16108 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.540526 16108 tablet_bootstrap.cc:492] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.541163 16108 ts_tablet_manager.cc:1403] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.017s	user 0.005s	sys 0.008s
I20260709 13:21:42.543488 16108 raft_consensus.cc:359] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.543996 16108 raft_consensus.cc:740] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.544577 16108 consensus_queue.cc:260] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.546008 16108 ts_tablet_manager.cc:1434] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:21:42.546890 16108 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.560926 16108 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.561865 16108 tablet_bootstrap.cc:492] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.562503 16108 ts_tablet_manager.cc:1403] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:21:42.564582 16108 raft_consensus.cc:359] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.565145 16108 raft_consensus.cc:740] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.565758 16108 consensus_queue.cc:260] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.567489 16108 ts_tablet_manager.cc:1434] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:21:42.568704 16108 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.596932 16108 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.597888 16108 tablet_bootstrap.cc:492] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.600121 16108 ts_tablet_manager.cc:1403] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.012s
I20260709 13:21:42.602360 16108 raft_consensus.cc:359] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.602921 16108 raft_consensus.cc:740] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.603549 16108 consensus_queue.cc:260] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.605674 16108 ts_tablet_manager.cc:1434] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:21:42.606968 16108 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.622653 16108 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.623628 16108 tablet_bootstrap.cc:492] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.624303 16108 ts_tablet_manager.cc:1403] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:21:42.626142 16108 raft_consensus.cc:359] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.626677 16108 raft_consensus.cc:740] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.627195 16108 consensus_queue.cc:260] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.628907 16108 ts_tablet_manager.cc:1434] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:21:42.629900 16108 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.643455 16108 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.644196 16108 tablet_bootstrap.cc:492] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.644764 16108 ts_tablet_manager.cc:1403] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.002s
I20260709 13:21:42.646507 16108 raft_consensus.cc:359] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.647043 16108 raft_consensus.cc:740] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.647749 16108 consensus_queue.cc:260] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.649507 16108 ts_tablet_manager.cc:1434] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.002s
I20260709 13:21:42.650544 16108 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.674718 16108 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.675733 16108 tablet_bootstrap.cc:492] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.676414 16108 ts_tablet_manager.cc:1403] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.003s
I20260709 13:21:42.678956 16108 raft_consensus.cc:359] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.679683 16108 raft_consensus.cc:740] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.680289 16108 consensus_queue.cc:260] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.682302 16108 ts_tablet_manager.cc:1434] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:21:42.689606 16108 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.712677 16108 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.713430 16108 tablet_bootstrap.cc:492] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.714035 16108 ts_tablet_manager.cc:1403] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.004s
I20260709 13:21:42.716337 16108 raft_consensus.cc:359] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.716895 16108 raft_consensus.cc:740] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.717504 16108 consensus_queue.cc:260] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.719494 16108 ts_tablet_manager.cc:1434] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:21:42.720590 16108 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.734148 16108 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.734923 16108 tablet_bootstrap.cc:492] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.735424 16108 ts_tablet_manager.cc:1403] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.008s
I20260709 13:21:42.737718 16108 raft_consensus.cc:359] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.738385 16108 raft_consensus.cc:740] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.739049 16108 consensus_queue.cc:260] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.740916 16108 ts_tablet_manager.cc:1434] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:21:42.742723 16108 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.762343 16108 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.763412 16108 tablet_bootstrap.cc:492] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.764096 16108 ts_tablet_manager.cc:1403] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:21:42.766525 16108 raft_consensus.cc:359] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.767208 16108 raft_consensus.cc:740] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.767865 16108 consensus_queue.cc:260] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.769814 16108 ts_tablet_manager.cc:1434] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:21:42.770635 16108 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.785894 16108 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.787093 16108 tablet_bootstrap.cc:492] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.787655 16108 ts_tablet_manager.cc:1403] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:21:42.789950 16108 raft_consensus.cc:359] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.790606 16108 raft_consensus.cc:740] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.791273 16108 consensus_queue.cc:260] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.792956 16108 ts_tablet_manager.cc:1434] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:21:42.795157 16108 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.795879 16146 raft_consensus.cc:3060] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:42.822165 16108 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.823187 16108 tablet_bootstrap.cc:492] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.823864 16108 ts_tablet_manager.cc:1403] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.008s
I20260709 13:21:42.826333 16108 raft_consensus.cc:359] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.826960 16108 raft_consensus.cc:740] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.827602 16108 consensus_queue.cc:260] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.829975 16108 ts_tablet_manager.cc:1434] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:21:42.830765 16108 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.852581 16108 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.853482 16108 tablet_bootstrap.cc:492] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.854141 16108 ts_tablet_manager.cc:1403] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:21:42.856676 16108 raft_consensus.cc:359] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.857322 16108 raft_consensus.cc:740] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.857928 16108 consensus_queue.cc:260] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.860208 16108 ts_tablet_manager.cc:1434] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:21:42.860998 16108 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.874854 16108 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.875811 16108 tablet_bootstrap.cc:492] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.876468 16108 ts_tablet_manager.cc:1403] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:21:42.878916 16108 raft_consensus.cc:359] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.879375 16108 raft_consensus.cc:740] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.879899 16108 consensus_queue.cc:260] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:42.882121 16108 ts_tablet_manager.cc:1434] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:21:42.882910 16108 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.898537 16108 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.899549 16108 tablet_bootstrap.cc:492] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.900184 16108 ts_tablet_manager.cc:1403] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:21:42.902581 16108 raft_consensus.cc:359] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.903298 16108 raft_consensus.cc:740] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.904000 16108 consensus_queue.cc:260] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.906209 16108 ts_tablet_manager.cc:1434] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:21:42.906957 16108 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.921918 16108 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.922740 16108 tablet_bootstrap.cc:492] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.923290 16108 ts_tablet_manager.cc:1403] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:21:42.925009 16108 raft_consensus.cc:359] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.925549 16108 raft_consensus.cc:740] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.926127 16108 consensus_queue.cc:260] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.927968 16108 ts_tablet_manager.cc:1434] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:21:42.929123 16108 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.940322 16108 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.941021 16108 tablet_bootstrap.cc:492] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.941553 16108 ts_tablet_manager.cc:1403] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:21:42.943590 16108 raft_consensus.cc:359] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.944072 16108 raft_consensus.cc:740] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.944641 16108 consensus_queue.cc:260] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.946004 16108 ts_tablet_manager.cc:1434] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:21:42.947023 16108 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.960680 16108 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.961596 16108 tablet_bootstrap.cc:492] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.962237 16108 ts_tablet_manager.cc:1403] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:21:42.964638 16108 raft_consensus.cc:359] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.965225 16108 raft_consensus.cc:740] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.965881 16108 consensus_queue.cc:260] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:42.967453 16108 ts_tablet_manager.cc:1434] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:21:42.968647 16108 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:42.987489 16108 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:42.988363 16108 tablet_bootstrap.cc:492] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:42.988989 16108 ts_tablet_manager.cc:1403] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 13:21:42.991405 16108 raft_consensus.cc:359] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.992163 16108 raft_consensus.cc:740] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.992815 16108 consensus_queue.cc:260] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:42.994802 16108 ts_tablet_manager.cc:1434] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:21:42.995525 16108 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:43.011430 16108 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:43.012434 16108 tablet_bootstrap.cc:492] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:43.013180 16108 ts_tablet_manager.cc:1403] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 13:21:43.015719 16108 raft_consensus.cc:359] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:43.016359 16108 raft_consensus.cc:740] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.017113 16108 consensus_queue.cc:260] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:43.019066 16108 ts_tablet_manager.cc:1434] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:21:43.019817 16108 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap starting.
I20260709 13:21:43.033318 16108 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:43.034250 16108 tablet_bootstrap.cc:492] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Bootstrap complete.
I20260709 13:21:43.034901 16108 ts_tablet_manager.cc:1403] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Time spent bootstrapping tablet: real 0.015s	user 0.015s	sys 0.000s
I20260709 13:21:43.037335 16146 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:43.038368 16108 raft_consensus.cc:359] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:43.039086 16108 raft_consensus.cc:740] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.039736 16108 consensus_queue.cc:260] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:43.041433 16108 ts_tablet_manager.cc:1434] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:21:43.057660 16146 raft_consensus.cc:3060] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:43.132959 16146 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:43.153721 16146 raft_consensus.cc:3060] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:43.216070 16146 raft_consensus.cc:3060] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:43.394866 16151 raft_consensus.cc:3060] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.280277  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:50.347007 15516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:50.347973 15516 raft_consensus.cc:606] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:50.350855 15516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:50.351423 15516 raft_consensus.cc:606] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Received request to transfer leadership to TS ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:50.353879 15516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:50.354396 15516 raft_consensus.cc:606] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:50.356801 15516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:50.357250 15516 raft_consensus.cc:606] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Received request to transfer leadership to TS ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:50.369247 15442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04"
dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:44578
I20260709 13:21:50.369833 15442 raft_consensus.cc:606] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:50.372148 15516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:50.372619 15516 raft_consensus.cc:606] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:50.373654  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:21:50.375286  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:50.377094 16346 raft_consensus.cc:993] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.377765 16353 raft_consensus.cc:1081] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.379217 16145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:50.379698 16145 raft_consensus.cc:493] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:50.380016 16145 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:50.386176 16145 raft_consensus.cc:515] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:50.388779 16145 leader_election.cc:290] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:50.401726 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:50.402665 15516 raft_consensus.cc:3055] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:50.402961 15516 raft_consensus.cc:740] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:50.403856 15516 consensus_queue.cc:260] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:50.405619 15516 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:50.406399 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "677797bb8c284c9d8a85ce81e8052fed" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:50.407040 15442 raft_consensus.cc:3060] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:50.411945 15516 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:50.413313 16102 leader_election.cc:304] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:50.413898 15442 raft_consensus.cc:2468] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:50.415401 16356 raft_consensus.cc:2804] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:50.416676 16356 raft_consensus.cc:697] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:50.417465 16356 consensus_queue.cc:237] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:50.431649 15192 catalog_manager.cc:5697] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:50.490589 16353 raft_consensus.cc:993] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.491089 16346 raft_consensus.cc:1081] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.492496 16145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:50.493016 16145 raft_consensus.cc:493] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:50.493299 16145 raft_consensus.cc:3060] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.498970 16145 raft_consensus.cc:515] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.500550 16145 leader_election.cc:290] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:50.501319 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:50.501538 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efc8987a44524529b4fcedae3a7ff32f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:50.501935 15442 raft_consensus.cc:3060] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.502094 15516 raft_consensus.cc:3055] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:50.502514 15516 raft_consensus.cc:740] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:50.503369 15516 consensus_queue.cc:260] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.504638 15516 raft_consensus.cc:3060] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.509084 15442 raft_consensus.cc:2468] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:50.510257 16105 leader_election.cc:304] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:50.511137 15516 raft_consensus.cc:2468] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:50.511144 16356 raft_consensus.cc:2804] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.511905 16356 raft_consensus.cc:697] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:50.512897 16356 consensus_queue.cc:237] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.529503 15192 catalog_manager.cc:5697] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 1 to 2, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:50.546037 16353 raft_consensus.cc:993] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower ee5629f4db5540b9a61ea4d7a180ee6b to start an election
I20260709 13:21:50.546460 16346 raft_consensus.cc:1081] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Signalling peer ee5629f4db5540b9a61ea4d7a180ee6b to start an election
I20260709 13:21:50.547816 15362 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e"
dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
 from {username='slave'} at 127.0.0.1:34528
I20260709 13:21:50.548321 15362 raft_consensus.cc:493] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:50.548641 15362 raft_consensus.cc:3060] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.553411 15362 raft_consensus.cc:515] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.555474 15362 leader_election.cc:290] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:50.556157 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:50.556444 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc5b0260a048f4a581d736b3b2191e" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:50.556803 15442 raft_consensus.cc:3060] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.557001 15516 raft_consensus.cc:3055] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:50.557400 15516 raft_consensus.cc:740] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:50.558169 15516 consensus_queue.cc:260] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.559307 15516 raft_consensus.cc:3060] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.563446 15442 raft_consensus.cc:2468] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:50.564695 15336 leader_election.cc:304] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:50.565593 16342 raft_consensus.cc:2804] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.565536 15516 raft_consensus.cc:2468] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:50.567044 16342 raft_consensus.cc:697] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:50.567909 16342 consensus_queue.cc:237] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.578341 15192 catalog_manager.cc:5697] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 1 to 2, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 2 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:50.642740 16347 raft_consensus.cc:993] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.643383 16365 raft_consensus.cc:1081] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.645378 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60726
I20260709 13:21:50.645905 16151 raft_consensus.cc:493] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:50.646241 16151 raft_consensus.cc:3060] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.653767 16151 raft_consensus.cc:515] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:50.655582 16151 leader_election.cc:290] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:50.656414 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:50.656795 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4539a0cb3c54de2862153a2f08eba04" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:50.657042 15442 raft_consensus.cc:3055] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:50.657356 15516 raft_consensus.cc:3060] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.657375 15442 raft_consensus.cc:740] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:50.658346 15442 consensus_queue.cc:260] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:50.659654 15442 raft_consensus.cc:3060] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.664737 15516 raft_consensus.cc:2468] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:50.664978 16353 raft_consensus.cc:993] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower ee5629f4db5540b9a61ea4d7a180ee6b to start an election
I20260709 13:21:50.665602 16346 raft_consensus.cc:1081] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Signalling peer ee5629f4db5540b9a61ea4d7a180ee6b to start an election
I20260709 13:21:50.666090 16102 leader_election.cc:304] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:50.666320 15442 raft_consensus.cc:2468] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:50.667070 16356 raft_consensus.cc:2804] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.667390 15363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec"
dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
 from {username='slave'} at 127.0.0.1:34528
I20260709 13:21:50.668241 16356 raft_consensus.cc:697] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:50.669173 15363 raft_consensus.cc:493] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:50.669533 15363 raft_consensus.cc:3060] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.669740 16356 consensus_queue.cc:237] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:50.679911 15363 raft_consensus.cc:515] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.680460 15192 catalog_manager.cc:5697] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 1 to 2, leader changed from d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:50.682163 15363 leader_election.cc:290] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:50.683724 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:50.684310 15442 raft_consensus.cc:3060] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.684649 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c2f416704b4f56b52001ddbc062bec" candidate_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:50.685300 15516 raft_consensus.cc:3055] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:50.685674 15516 raft_consensus.cc:740] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:50.686564 15516 consensus_queue.cc:260] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.691174 15516 raft_consensus.cc:3060] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.694743 15442 raft_consensus.cc:2468] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:50.695791 15336 leader_election.cc:304] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d646a2b597824d24b522d9ef5ff5adc3, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:50.696552 16342 raft_consensus.cc:2804] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.696986 16342 raft_consensus.cc:697] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Becoming Leader. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:50.697774 16342 consensus_queue.cc:237] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:50.698580 15516 raft_consensus.cc:2468] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5629f4db5540b9a61ea4d7a180ee6b in term 2.
I20260709 13:21:50.706884 15192 catalog_manager.cc:5697] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b reported cstate change: term changed from 1 to 2, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130). New cstate: current_term: 2 leader_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:50.826906 16346 raft_consensus.cc:993] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.827373 16353 raft_consensus.cc:1081] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:50.829068 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:50.829623 16151 raft_consensus.cc:493] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:50.829914 16151 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:50.835717 16151 raft_consensus.cc:515] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:50.837549 16151 leader_election.cc:290] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:50.840226 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:50.841125 15516 raft_consensus.cc:3055] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:50.843586 15516 raft_consensus.cc:740] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:50.845988 15516 consensus_queue.cc:260] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:50.854676 15516 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:50.863306 15362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b6805b1cee04dff9be875c8d6d74e7d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:50.864593 15362 raft_consensus.cc:3060] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:50.865255 15516 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:50.867852 16102 leader_election.cc:304] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:50.870333 16356 raft_consensus.cc:2804] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:50.871721 16356 raft_consensus.cc:697] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:50.873363 16356 consensus_queue.cc:237] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:50.880492 15436 raft_consensus.cc:1275] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:50.905987 16357 consensus_queue.cc:1048] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:50.919905 15362 raft_consensus.cc:2468] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:50.923359 15192 catalog_manager.cc:5697] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:50.932559 15516 raft_consensus.cc:1275] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:50.934361 16372 consensus_queue.cc:1048] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:51.000167 15436 raft_consensus.cc:1275] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.001343 16372 consensus_queue.cc:1048] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:51.010344 15516 raft_consensus.cc:1275] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.012328 16356 consensus_queue.cc:1048] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:51.053608 15510 raft_consensus.cc:1275] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.055107 16345 consensus_queue.cc:1048] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:51.067414 15436 raft_consensus.cc:1275] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.075974 16345 consensus_queue.cc:1048] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:51.093137 15436 raft_consensus.cc:1275] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.094419 16356 consensus_queue.cc:1048] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:51.108847 15510 raft_consensus.cc:1275] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.110831 16356 consensus_queue.cc:1048] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:51.222198 15436 raft_consensus.cc:1275] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.223950 16345 consensus_queue.cc:1048] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:51.238268 15510 raft_consensus.cc:1275] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer ee5629f4db5540b9a61ea4d7a180ee6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.240494 16345 consensus_queue.cc:1048] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:51.376371  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:51.412050 15364 raft_consensus.cc:1275] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:51.413564 16372 consensus_queue.cc:1048] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:51.433107 15510 raft_consensus.cc:1275] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:51.435257 16356 consensus_queue.cc:1048] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:51.440474  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:51.495837 15510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:51.496384 15510 raft_consensus.cc:606] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:51.498705 15510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "318aa120381647db94d137ca2de5d87b"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:51.499204 15510 raft_consensus.cc:606] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:51.501358 15510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:51.501859 15510 raft_consensus.cc:606] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:51.504076 15510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:51.504667 15510 raft_consensus.cc:606] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:51.507069 15510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bad35d090e945218853dd63c340674b"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:51.507614 15510 raft_consensus.cc:606] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:51.510814 15436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587"
dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:44578
I20260709 13:21:51.511444 15436 raft_consensus.cc:606] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:51.526242 15364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94"
dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
mode: GRACEFUL
new_leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
 from {username='slave'} at 127.0.0.1:33064
I20260709 13:21:51.526953 15364 raft_consensus.cc:606] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Received request to transfer leadership to TS 1b7f36fa6b484a7b8859acbcd86c4f0b
I20260709 13:21:51.528205  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:21:51.529501  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:51.666419 16345 raft_consensus.cc:993] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: : Instructing follower 1b7f36fa6b484a7b8859acbcd86c4f0b to start an election
I20260709 13:21:51.666939 16342 raft_consensus.cc:1081] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Signalling peer 1b7f36fa6b484a7b8859acbcd86c4f0b to start an election
I20260709 13:21:51.668500 15516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
 from {username='slave'} at 127.0.0.1:35050
I20260709 13:21:51.669032 15516 raft_consensus.cc:493] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:51.669826 15516 raft_consensus.cc:3060] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.676765 15516 raft_consensus.cc:515] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:51.678880 15516 leader_election.cc:290] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:51.679477 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:51.680145 15364 raft_consensus.cc:3055] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:51.680336 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e02c064babd84e109140e2fa50a9ad94" candidate_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:51.680397 15364 raft_consensus.cc:740] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee5629f4db5540b9a61ea4d7a180ee6b, State: Running, Role: LEADER
I20260709 13:21:51.680943 15436 raft_consensus.cc:3060] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.681377 15364 consensus_queue.cc:260] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:51.682596 15364 raft_consensus.cc:3060] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.686771 15436 raft_consensus.cc:2468] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:51.688405 15484 leader_election.cc:304] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:51.689406 16390 raft_consensus.cc:2804] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.689630 16349 raft_consensus.cc:993] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.689941 16390 raft_consensus.cc:697] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Leader. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:51.690575 16353 raft_consensus.cc:1081] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.692286 15364 raft_consensus.cc:2468] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7f36fa6b484a7b8859acbcd86c4f0b in term 2.
I20260709 13:21:51.691039 16390 consensus_queue.cc:237] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } }
I20260709 13:21:51.692680 16145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bad35d090e945218853dd63c340674b"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:51.693243 16145 raft_consensus.cc:493] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:51.693616 16145 raft_consensus.cc:3060] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.700122 16145 raft_consensus.cc:515] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.702939 15510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:51.703506 15510 raft_consensus.cc:3055] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:51.703773 15510 raft_consensus.cc:740] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:51.704061 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bad35d090e945218853dd63c340674b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:51.704645 15364 raft_consensus.cc:3060] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.704501 15510 consensus_queue.cc:260] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.704459 15190 catalog_manager.cc:5697] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b reported cstate change: term changed from 1 to 2, leader changed from ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130) to 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132). New cstate: current_term: 2 leader_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:51.705893 15510 raft_consensus.cc:3060] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.707252 16145 leader_election.cc:290] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:51.713028 15364 raft_consensus.cc:2468] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:51.713622 15510 raft_consensus.cc:2468] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:51.714311 16102 leader_election.cc:304] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:51.715343 16372 raft_consensus.cc:2804] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.715786 16372 raft_consensus.cc:697] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:51.716658 16372 consensus_queue.cc:237] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.731585 15192 catalog_manager.cc:5697] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 1 to 2, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.839664 16353 raft_consensus.cc:993] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.840196 16390 raft_consensus.cc:1081] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.841651 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:51.842217 16151 raft_consensus.cc:493] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:51.842571 16151 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.847965 16151 raft_consensus.cc:515] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.849534 16151 leader_election.cc:290] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:51.850217 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:51.850494 15510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1ddcb87ba840e68a9c62551336aa0c" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:51.850890 15436 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.851158 15510 raft_consensus.cc:3055] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:51.851529 15510 raft_consensus.cc:740] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:51.852296 15510 consensus_queue.cc:260] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.853302 15510 raft_consensus.cc:3060] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.856419 15436 raft_consensus.cc:2468] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:51.857432 16105 leader_election.cc:304] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:51.858246 16372 raft_consensus.cc:2804] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:51.858798 16372 raft_consensus.cc:697] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:51.858974 15510 raft_consensus.cc:2468] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:51.859692 16372 consensus_queue.cc:237] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.863879 16353 raft_consensus.cc:993] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.864430 16349 raft_consensus.cc:1081] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.866336 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:51.866865 16151 raft_consensus.cc:493] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:51.867283 16151 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.872660 15192 catalog_manager.cc:5697] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:51.873306 16151 raft_consensus.cc:515] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.875222 16151 leader_election.cc:290] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:51.875710 15510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:51.876192 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12527dc46ef54e4f944ce04478f71b8d" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:51.876339 15510 raft_consensus.cc:3055] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:51.876780 15510 raft_consensus.cc:740] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:51.876781 15364 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.877794 15510 consensus_queue.cc:260] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.879916 15510 raft_consensus.cc:3060] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.884718 16353 raft_consensus.cc:993] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.885242 16353 raft_consensus.cc:1081] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.886322 15364 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:51.886794 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "318aa120381647db94d137ca2de5d87b"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:51.887429 16151 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:51.887449 16102 leader_election.cc:304] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:51.887883 16151 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.888451 16372 raft_consensus.cc:2804] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:51.889120 15510 raft_consensus.cc:2468] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:51.889210 16372 raft_consensus.cc:697] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:51.891129 16372 consensus_queue.cc:237] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.894711 16151 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.896592 16151 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:51.897210 15510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:51.897629 15364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "318aa120381647db94d137ca2de5d87b" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:51.897799 15510 raft_consensus.cc:3055] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:51.898204 15364 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.898244 15510 raft_consensus.cc:740] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:51.899190 15510 consensus_queue.cc:260] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.900439 15510 raft_consensus.cc:3060] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.902379 15192 catalog_manager.cc:5697] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.904858 15364 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:51.906232 15510 raft_consensus.cc:2468] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:51.907086 16102 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:51.909185 16372 raft_consensus.cc:2804] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:51.909667 16372 raft_consensus.cc:697] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:51.910390 16372 consensus_queue.cc:237] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:51.919157 15192 catalog_manager.cc:5697] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.948884 16353 raft_consensus.cc:993] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.949393 16349 raft_consensus.cc:1081] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.950789 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:51.951241 16151 raft_consensus.cc:493] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:51.951550 16151 raft_consensus.cc:3060] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.957091 16151 raft_consensus.cc:515] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.958559 16151 leader_election.cc:290] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:51.959326 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:51.959620 15510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f14bc0f2237437888f16c60a62840de" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:51.959951 15436 raft_consensus.cc:3060] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.960151 15510 raft_consensus.cc:3055] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:51.960460 15510 raft_consensus.cc:740] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:51.961222 15510 consensus_queue.cc:260] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.962385 15510 raft_consensus.cc:3060] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.966899 15436 raft_consensus.cc:2468] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:51.968726 16105 leader_election.cc:304] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, d646a2b597824d24b522d9ef5ff5adc3; no voters: 
I20260709 13:21:51.969656 16372 raft_consensus.cc:2804] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.970516 16372 raft_consensus.cc:697] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:51.971241 16347 raft_consensus.cc:993] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.970382 15510 raft_consensus.cc:2468] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:51.971947 16347 raft_consensus.cc:1081] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:51.971753 16372 consensus_queue.cc:237] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.973749 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60726
I20260709 13:21:51.974290 16151 raft_consensus.cc:493] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:51.974690 16151 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:51.982406 16151 raft_consensus.cc:515] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.984659 16151 leader_election.cc:290] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:51.985610 15510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:51.984882 15436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a49a8fabd3bc4946993584f83f34f587" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:51.998889 15436 raft_consensus.cc:3055] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:51.999310 15436 raft_consensus.cc:740] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:52.000100 15436 consensus_queue.cc:260] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:51.989176 15192 catalog_manager.cc:5697] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 1 to 2, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:51.993399 15510 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.010952 15436 raft_consensus.cc:3060] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.013243 15510 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:52.014496 16102 leader_election.cc:304] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:52.015321 16372 raft_consensus.cc:2804] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:52.015846 16372 raft_consensus.cc:697] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:52.016741 16372 consensus_queue.cc:237] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:52.017795 15436 raft_consensus.cc:2468] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:52.027873 15192 catalog_manager.cc:5697] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:52.157864 15364 raft_consensus.cc:1275] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.159884 16372 consensus_queue.cc:1048] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:52.164062 15364 raft_consensus.cc:1275] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.165326 16353 consensus_queue.cc:1048] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:52.179268 15516 raft_consensus.cc:1275] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.181686 16372 consensus_queue.cc:1048] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:52.202554 15436 raft_consensus.cc:1275] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 1b7f36fa6b484a7b8859acbcd86c4f0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.204048 16353 consensus_queue.cc:1048] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:52.293835 15516 raft_consensus.cc:1275] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.301260 15510 raft_consensus.cc:1275] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.301362 16372 consensus_queue.cc:1048] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:52.305951 16372 consensus_queue.cc:1048] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:52.308974 15442 raft_consensus.cc:1275] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.310565 16356 consensus_queue.cc:1048] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:52.358455 15364 raft_consensus.cc:1275] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.369680 16415 consensus_queue.cc:1048] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:52.415056 15364 raft_consensus.cc:1275] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.416234 16415 consensus_queue.cc:1048] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:52.418236 15442 raft_consensus.cc:1275] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.419518 16415 consensus_queue.cc:1048] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:52.427374 15510 raft_consensus.cc:1275] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.429343 16417 consensus_queue.cc:1048] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:52.441473 15517 raft_consensus.cc:1275] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:52.442814 16356 consensus_queue.cc:1048] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:52.454663 15510 raft_consensus.cc:1275] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.457417 16357 consensus_queue.cc:1048] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:52.489141 15442 raft_consensus.cc:1275] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.490541 16415 consensus_queue.cc:1048] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:52.534175  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:52.575984  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:52.612628 15442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f"
dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:44578
I20260709 13:21:52.613102 15442 raft_consensus.cc:606] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:52.615543 15510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:52.616111 15510 raft_consensus.cc:606] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:52.618582 15510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483"
dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:59210
I20260709 13:21:52.619166 15510 raft_consensus.cc:606] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:52.622586 15442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f"
dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
mode: GRACEFUL
new_leader_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:44578
I20260709 13:21:52.623116 15442 raft_consensus.cc:606] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Received request to transfer leadership to TS 88eead6a17f54dd88c723744e0aa5b50
I20260709 13:21:52.624094  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:21:52.625371  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:52.699605 16390 raft_consensus.cc:993] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:52.700098 16353 raft_consensus.cc:1081] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:52.701323 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:52.701707 16151 raft_consensus.cc:493] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:52.701917 16151 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.707576 16151 raft_consensus.cc:515] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:52.708920 16151 leader_election.cc:290] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:52.709705 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:52.710101 15362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6e85352a3ac49939b3bdab9c612b76e" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:52.710284 15516 raft_consensus.cc:3055] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:52.710671 15362 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.710697 15516 raft_consensus.cc:740] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:52.711521 15516 consensus_queue.cc:260] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:52.712513 15516 raft_consensus.cc:3060] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.715229 15362 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:52.716168 16102 leader_election.cc:304] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:52.716866 16415 raft_consensus.cc:2804] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:52.717264 15516 raft_consensus.cc:2468] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:52.717376 16415 raft_consensus.cc:697] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:52.718525 16415 consensus_queue.cc:237] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:52.727506 15193 catalog_manager.cc:5697] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:52.733716 16330 raft_consensus.cc:993] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:52.734174 16330 raft_consensus.cc:1081] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:52.736855 16145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60726
I20260709 13:21:52.737594 16145 raft_consensus.cc:493] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:52.738102 16145 raft_consensus.cc:3060] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.744445 16145 raft_consensus.cc:515] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:52.746940 16145 leader_election.cc:290] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Requested vote from peers d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805), 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633)
I20260709 13:21:52.747143 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:52.747685 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c87965874f942deb1c496acbfed4a1f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:52.747781 15516 raft_consensus.cc:3060] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.748322 15442 raft_consensus.cc:3055] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:21:52.748680 15442 raft_consensus.cc:740] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:52.749431 15442 consensus_queue.cc:260] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:52.750725 15442 raft_consensus.cc:3060] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:52.755091 15516 raft_consensus.cc:2468] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:52.756259 16102 leader_election.cc:304] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:52.757020 16415 raft_consensus.cc:2804] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:52.757157 15442 raft_consensus.cc:2468] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 2.
I20260709 13:21:52.757481 16415 raft_consensus.cc:697] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:52.758452 16415 consensus_queue.cc:237] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } }
I20260709 13:21:52.768569 16399 raft_consensus.cc:993] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:52.769121 16399 raft_consensus.cc:1081] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:52.770527 16145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60726
I20260709 13:21:52.771055 16145 raft_consensus.cc:493] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:52.771381 16145 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.771545 15193 catalog_manager.cc:5697] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 1 to 2, leader changed from d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:52.778093 16145 raft_consensus.cc:515] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:52.781914 15362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:52.782465 15362 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.782402 15442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7b5a27333a34cfeb46c592917d72a5f" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d646a2b597824d24b522d9ef5ff5adc3"
I20260709 13:21:52.783160 15442 raft_consensus.cc:3055] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:52.783489 15442 raft_consensus.cc:740] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d646a2b597824d24b522d9ef5ff5adc3, State: Running, Role: LEADER
I20260709 13:21:52.784296 15442 consensus_queue.cc:260] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:52.785377 15442 raft_consensus.cc:3060] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:52.786211 16145 leader_election.cc:290] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479), d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805)
I20260709 13:21:52.790830 15362 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:52.792186 16102 leader_election.cc:304] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b; no voters: 
I20260709 13:21:52.792959 16415 raft_consensus.cc:2804] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:52.793094 15442 raft_consensus.cc:2468] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:52.793412 16415 raft_consensus.cc:697] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:52.794392 16415 consensus_queue.cc:237] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } }
I20260709 13:21:52.804567 15193 catalog_manager.cc:5697] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:53.036715 16390 raft_consensus.cc:993] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: : Instructing follower 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:53.037319 16353 raft_consensus.cc:1081] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Signalling peer 88eead6a17f54dd88c723744e0aa5b50 to start an election
I20260709 13:21:53.038825 16145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483"
dest_uuid: "88eead6a17f54dd88c723744e0aa5b50"
 from {username='slave'} at 127.0.0.1:60724
I20260709 13:21:53.039283 16145 raft_consensus.cc:493] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:53.039595 16145 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:53.046206 16145 raft_consensus.cc:515] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:53.048596 15516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b"
I20260709 13:21:53.049144 15516 raft_consensus.cc:3055] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:21:53.049443 15516 raft_consensus.cc:740] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b7f36fa6b484a7b8859acbcd86c4f0b, State: Running, Role: LEADER
I20260709 13:21:53.050108 15516 consensus_queue.cc:260] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:53.051151 15516 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:53.052744 16145 leader_election.cc:290] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Requested vote from peers 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132:41633), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:53.054436 15362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e04a0fb0fb0342bd935451e239964483" candidate_uuid: "88eead6a17f54dd88c723744e0aa5b50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b"
I20260709 13:21:53.054957 15362 raft_consensus.cc:3060] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:21:53.057145 15516 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:53.058286 16102 leader_election.cc:304] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b, 88eead6a17f54dd88c723744e0aa5b50; no voters: 
I20260709 13:21:53.059099 16415 raft_consensus.cc:2804] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:21:53.059554 16415 raft_consensus.cc:697] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Becoming Leader. State: Replica: 88eead6a17f54dd88c723744e0aa5b50, State: Running, Role: LEADER
I20260709 13:21:53.060459 15362 raft_consensus.cc:2468] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88eead6a17f54dd88c723744e0aa5b50 in term 3.
I20260709 13:21:53.060456 16415 consensus_queue.cc:237] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:53.070900 15193 catalog_manager.cc:5697] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 reported cstate change: term changed from 2 to 3, leader changed from 1b7f36fa6b484a7b8859acbcd86c4f0b (127.5.72.132) to 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129). New cstate: current_term: 3 leader_uuid: "88eead6a17f54dd88c723744e0aa5b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:53.131639 15516 raft_consensus.cc:1275] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:53.132848 16415 consensus_queue.cc:1048] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:53.143746 15364 raft_consensus.cc:1275] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:53.145210 16356 consensus_queue.cc:1048] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:53.331485 15442 raft_consensus.cc:1275] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:53.333038 16356 consensus_queue.cc:1048] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:53.344084 15516 raft_consensus.cc:1275] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:53.345659 16357 consensus_queue.cc:1048] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:21:53.358340 15362 raft_consensus.cc:1275] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:53.360739 16357 consensus_queue.cc:1048] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:53.383356 15442 raft_consensus.cc:1275] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:53.389276 16356 consensus_queue.cc:1048] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646a2b597824d24b522d9ef5ff5adc3" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:21:53.619601 15364 raft_consensus.cc:1275] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:53.620812 16357 consensus_queue.cc:1048] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:53.630677  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:53.657156 15510 raft_consensus.cc:1275] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Refusing update from remote peer 88eead6a17f54dd88c723744e0aa5b50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:21:53.658794 16415 consensus_queue.cc:1048] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:21:53.688158  5410 tablet_server.cc:179] TabletServer@127.5.72.129:43313 shutting down...
I20260709 13:21:53.729355  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:53.732262  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.733171  5410 raft_consensus.cc:2243] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.733641  5410 raft_consensus.cc:2272] T de52504e9e244dc99cc89e67e9281f3d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.736743  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.737632  5410 raft_consensus.cc:2243] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.738363  5410 raft_consensus.cc:2272] T 7bad35d090e945218853dd63c340674b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.740867  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.741834  5410 raft_consensus.cc:2243] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.742398  5410 raft_consensus.cc:2272] T 260309c1df6b490ea4502dd13e14f692 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:53.742440 15336 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 68 similar messages]
W20260709 13:21:53.745527 15336 consensus_peers.cc:597] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.745700  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.746500  5410 raft_consensus.cc:2243] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.747413  5410 raft_consensus.cc:2272] T 5e1ddcb87ba840e68a9c62551336aa0c P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.749717  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.750303  5410 raft_consensus.cc:2243] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.750741  5410 raft_consensus.cc:2272] T 67f48e3fa27c4c6487669b0d45130af7 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.752775  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.753393  5410 raft_consensus.cc:2243] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.753798  5410 raft_consensus.cc:2272] T f542e50c9876452b88a3f17ea981a62c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.756532  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.757269  5410 raft_consensus.cc:2243] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.757748  5410 raft_consensus.cc:2272] T 3593559594274cc7aef7c38772401110 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.760270  5410 tablet_replica.cc:333] stopping tablet replica
W20260709 13:21:53.760349 15410 consensus_peers.cc:597] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.761071  5410 raft_consensus.cc:2243] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.761467  5410 raft_consensus.cc:2272] T 4d4413c641a248a9a0cf781768951edc P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.763545  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.764377  5410 raft_consensus.cc:2243] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.764839  5410 raft_consensus.cc:2272] T 20d52192ec7445f4a801452a645af6ba P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.767565  5410 tablet_replica.cc:333] stopping tablet replica
W20260709 13:21:53.767946 15410 consensus_peers.cc:597] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.768347  5410 raft_consensus.cc:2243] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.768782  5410 raft_consensus.cc:2272] T 69858931805f420db0a026c73d7b0f06 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.771025  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.771559  5410 raft_consensus.cc:2243] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.771901  5410 raft_consensus.cc:2272] T 979c935f63114ce6b2c31919b88a250f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.773927  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.774744  5410 raft_consensus.cc:2243] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.775563  5410 raft_consensus.cc:2272] T f133a6db45994178b473a5b1809d367f P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.778555  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.779450  5410 raft_consensus.cc:2243] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.779938  5410 raft_consensus.cc:2272] T 4e39f7656f2748f994ca243f51380c1c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:53.780099 15410 consensus_peers.cc:597] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.783027  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.783794  5410 raft_consensus.cc:2243] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.784286  5410 raft_consensus.cc:2272] T 3288c26b19cb47dfb6035bddf30566df P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.787134  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.787770  5410 raft_consensus.cc:2243] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.788272  5410 raft_consensus.cc:2272] T 8d531517bd614ae3af03be27016eb596 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:53.788589 15484 consensus_peers.cc:597] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.790568  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.791390  5410 raft_consensus.cc:2243] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.791848  5410 raft_consensus.cc:2272] T 55c5fe7b4ba54f94a897b776dcc58565 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.794607  5410 tablet_replica.cc:333] stopping tablet replica
W20260709 13:21:53.794808 15484 consensus_peers.cc:597] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.795310  5410 raft_consensus.cc:2243] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.795814  5410 raft_consensus.cc:2272] T 427b294c2f944e7ca234578e92b6126a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.798799  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.799626  5410 raft_consensus.cc:2243] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:53.799857 15484 consensus_peers.cc:597] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.800137  5410 raft_consensus.cc:2272] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.802953  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.803577  5410 raft_consensus.cc:2243] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.804381  5410 raft_consensus.cc:2272] T e04a0fb0fb0342bd935451e239964483 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.807183  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.807979  5410 raft_consensus.cc:2243] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.808456  5410 raft_consensus.cc:2272] T ffef2ecb155c41daba889a3314b66527 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.810559  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.811187  5410 raft_consensus.cc:2243] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.811625  5410 raft_consensus.cc:2272] T af51ee9eaabb4a29a989844d36c3f423 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.814075  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.814796  5410 raft_consensus.cc:2243] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.815783  5410 raft_consensus.cc:2272] T 1c87965874f942deb1c496acbfed4a1f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.818989  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.819710  5410 raft_consensus.cc:2243] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:21:53.819870 15410 consensus_peers.cc:597] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.820612  5410 raft_consensus.cc:2272] T 12527dc46ef54e4f944ce04478f71b8d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.822839  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.823453  5410 raft_consensus.cc:2243] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.823926  5410 raft_consensus.cc:2272] T 3ba45fbc82014336a9bddd17603ed390 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.826388  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.827004  5410 raft_consensus.cc:2243] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.827430  5410 raft_consensus.cc:2272] T e5546361e1c64e36951aeb942521a72b P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.830056  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.830783  5410 raft_consensus.cc:2243] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.831281  5410 raft_consensus.cc:2272] T e8d49dec28d44105b80839cb58705961 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.833398  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.834045  5410 raft_consensus.cc:2243] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.834501  5410 raft_consensus.cc:2272] T c124e9e01448409d9a64892dddf7c610 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:53.836184 15410 consensus_peers.cc:597] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.837066  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.837677  5410 raft_consensus.cc:2243] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.838114  5410 raft_consensus.cc:2272] T 65aff9e32079465b85d18fff1c73b28a P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.840806  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.841522  5410 raft_consensus.cc:2243] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.842458  5410 raft_consensus.cc:2272] T a6e85352a3ac49939b3bdab9c612b76e P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.845362  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.847329  5410 raft_consensus.cc:2243] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.848021  5410 raft_consensus.cc:2272] T b99ac0daff22426fb7dfa484f28c548d P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.851337  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.852010  5410 raft_consensus.cc:2243] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.852598  5410 raft_consensus.cc:2272] T c6bde75c2a4c44b0beb23e22159b9fa2 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.855861  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.856638  5410 raft_consensus.cc:2243] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.857798  5410 raft_consensus.cc:2272] T efc8987a44524529b4fcedae3a7ff32f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.860350  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.861159  5410 raft_consensus.cc:2243] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.861753  5410 raft_consensus.cc:2272] T 2c9fb81bccaf408cba956547d37caf3f P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.864634  5410 tablet_replica.cc:333] stopping tablet replica
W20260709 13:21:53.864774 15336 consensus_peers.cc:597] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.865293  5410 raft_consensus.cc:2243] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.866113  5410 raft_consensus.cc:2272] T 4f14bc0f2237437888f16c60a62840de P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.868229  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.868831  5410 raft_consensus.cc:2243] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.869243  5410 raft_consensus.cc:2272] T 54afc04cc1b742d1b309764bb898bc6b P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.871543  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.872203  5410 raft_consensus.cc:2243] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.872682  5410 raft_consensus.cc:2272] T d95073a3b15444a4880921aaad3f82a1 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.876565  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.878113  5410 raft_consensus.cc:2243] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.880268  5410 raft_consensus.cc:2272] T a49a8fabd3bc4946993584f83f34f587 P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:53.884106 15336 consensus_peers.cc:597] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.886286  5410 tablet_replica.cc:333] stopping tablet replica
W20260709 13:21:53.886745 15484 consensus_peers.cc:597] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.888648  5410 raft_consensus.cc:2243] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.890041  5410 raft_consensus.cc:2272] T 2588887022d8411bba3692826b2fe934 P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.896605  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.898267  5410 raft_consensus.cc:2243] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.899214  5410 raft_consensus.cc:2272] T f34db7828d184304ab99b4216ac1798e P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.902072  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.902774  5410 raft_consensus.cc:2243] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:53.903177  5410 raft_consensus.cc:2272] T 3ed4922079864d94a4a771fb9466d63c P 88eead6a17f54dd88c723744e0aa5b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.905311  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.905834  5410 raft_consensus.cc:2243] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.906603  5410 raft_consensus.cc:2272] T d4539a0cb3c54de2862153a2f08eba04 P 88eead6a17f54dd88c723744e0aa5b50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.908802  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.909454  5410 raft_consensus.cc:2243] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.910446  5410 raft_consensus.cc:2272] T f7b5a27333a34cfeb46c592917d72a5f P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.913533  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.914268  5410 raft_consensus.cc:2243] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.915258  5410 raft_consensus.cc:2272] T 0b6805b1cee04dff9be875c8d6d74e7d P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.917448  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.918103  5410 raft_consensus.cc:2243] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.918999  5410 raft_consensus.cc:2272] T 677797bb8c284c9d8a85ce81e8052fed P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:53.921898  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:21:53.922448  5410 raft_consensus.cc:2243] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:21:53.923437  5410 raft_consensus.cc:2272] T 318aa120381647db94d137ca2de5d87b P 88eead6a17f54dd88c723744e0aa5b50 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:53.961066 15336 consensus_peers.cc:597] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:53.990907 15336 consensus_peers.cc:597] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:53.992555 15410 consensus_peers.cc:597] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:53.994824  5410 tablet_server.cc:196] TabletServer@127.5.72.129:43313 shutdown complete.
W20260709 13:21:53.998553 15484 consensus_peers.cc:597] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.020610 15410 consensus_peers.cc:597] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.023001 15336 consensus_peers.cc:597] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.030367 15410 consensus_peers.cc:597] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.040639 15336 consensus_peers.cc:597] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.047021 15336 consensus_peers.cc:597] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.048920 15484 consensus_peers.cc:597] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.086723  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
W20260709 13:21:54.094789 15336 consensus_peers.cc:597] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.109496 15336 consensus_peers.cc:597] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.118671 15481 consensus_peers.cc:597] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.121822 15407 consensus_peers.cc:597] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.131054 15407 consensus_peers.cc:597] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.132591  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:54.133354  5410 tablet_replica.cc:333] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.134238  5410 raft_consensus.cc:2243] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.135285  5410 raft_consensus.cc:2272] T 20d52192ec7445f4a801452a645af6ba P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.137746  5410 tablet_replica.cc:333] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.138314  5410 raft_consensus.cc:2243] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.139087  5410 raft_consensus.cc:2272] T 979c935f63114ce6b2c31919b88a250f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.140967  5410 tablet_replica.cc:333] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.141778  5410 raft_consensus.cc:2243] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.142266  5410 raft_consensus.cc:2272] T 4d4413c641a248a9a0cf781768951edc P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.144861  5410 tablet_replica.cc:333] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
W20260709 13:21:54.145242 15410 consensus_peers.cc:597] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.145681  5410 raft_consensus.cc:2243] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.146811  5410 raft_consensus.cc:2272] T d95073a3b15444a4880921aaad3f82a1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.150415  5410 tablet_replica.cc:333] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.151283  5410 raft_consensus.cc:2243] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:54.151212 15336 consensus_peers.cc:597] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.152053  5410 raft_consensus.cc:2272] T 7bad35d090e945218853dd63c340674b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.154078  5410 tablet_replica.cc:333] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.155154  5410 raft_consensus.cc:2243] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.155761  5410 raft_consensus.cc:2272] T 260309c1df6b490ea4502dd13e14f692 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.158581  5410 tablet_replica.cc:333] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.159412  5410 raft_consensus.cc:2243] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.159935  5410 raft_consensus.cc:2272] T 500dbd579fd848328ce9984276eb0e04 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.162281  5410 tablet_replica.cc:333] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.163084  5410 raft_consensus.cc:2243] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.163998  5410 raft_consensus.cc:2272] T e5546361e1c64e36951aeb942521a72b P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.166929  5410 tablet_replica.cc:333] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
W20260709 13:21:54.167552 15481 consensus_peers.cc:597] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.167747  5410 raft_consensus.cc:2243] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.168690  5410 raft_consensus.cc:2272] T 3593559594274cc7aef7c38772401110 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.169553 15484 consensus_peers.cc:597] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.169836 15407 consensus_peers.cc:597] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.171182  5410 tablet_replica.cc:333] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.172011  5410 raft_consensus.cc:2243] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.172461  5410 raft_consensus.cc:2272] T f542e50c9876452b88a3f17ea981a62c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.172410 15410 consensus_peers.cc:597] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.175269  5410 tablet_replica.cc:333] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.176007  5410 raft_consensus.cc:2243] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.176512  5410 raft_consensus.cc:2272] T 2588887022d8411bba3692826b2fe934 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.179291  5410 tablet_replica.cc:333] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.180022  5410 raft_consensus.cc:2243] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.180591  5410 raft_consensus.cc:2272] T d81a8d43d2a34dc38f37e6c6c03c29c3 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.183557  5410 tablet_replica.cc:333] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.184336  5410 raft_consensus.cc:2243] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.184860  5410 raft_consensus.cc:2272] T f9cf658f5d1340238f97e4d01fa8c07e P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.187841  5410 tablet_replica.cc:333] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.188647  5410 raft_consensus.cc:2243] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.189599  5410 raft_consensus.cc:2272] T 43fc5b0260a048f4a581d736b3b2191e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.192260  5410 tablet_replica.cc:333] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.193084  5410 raft_consensus.cc:2243] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.193637  5410 raft_consensus.cc:2272] T 0b6805b1cee04dff9be875c8d6d74e7d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.196179  5410 tablet_replica.cc:333] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.196936  5410 raft_consensus.cc:2243] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.197423  5410 raft_consensus.cc:2272] T f7b5a27333a34cfeb46c592917d72a5f P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.197553 15481 consensus_peers.cc:597] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.200189  5410 tablet_replica.cc:333] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
W20260709 13:21:54.200666 15484 consensus_peers.cc:597] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.201027  5410 raft_consensus.cc:2243] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.201917  5410 raft_consensus.cc:2272] T f133a6db45994178b473a5b1809d367f P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.204021  5410 tablet_replica.cc:333] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.204627  5410 raft_consensus.cc:2243] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.205034  5410 raft_consensus.cc:2272] T 73732c746e334164b459508a1eeda5ec P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.207384  5410 tablet_replica.cc:333] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.208142  5410 raft_consensus.cc:2243] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.208644  5410 raft_consensus.cc:2272] T 100da53293964979b47f6fe528856727 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.211040 15407 consensus_peers.cc:597] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.211328  5410 tablet_replica.cc:333] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.212220  5410 raft_consensus.cc:2243] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.213207  5410 raft_consensus.cc:2272] T 128c6f471a6b4c53bdd2c868f551b542 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.216081  5410 tablet_replica.cc:333] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.216678  5410 raft_consensus.cc:2243] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.217541  5410 raft_consensus.cc:2272] T 8d531517bd614ae3af03be27016eb596 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.219727  5410 tablet_replica.cc:333] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.220302  5410 raft_consensus.cc:2243] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.220731  5410 raft_consensus.cc:2272] T 3288c26b19cb47dfb6035bddf30566df P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.222702  5410 tablet_replica.cc:333] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.223286  5410 raft_consensus.cc:2243] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.224205  5410 raft_consensus.cc:2272] T 55c5fe7b4ba54f94a897b776dcc58565 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.226604  5410 tablet_replica.cc:333] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.227360  5410 raft_consensus.cc:2243] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.227815  5410 raft_consensus.cc:2272] T ffef2ecb155c41daba889a3314b66527 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.229808 15410 consensus_peers.cc:597] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 -> Peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Couldn't send request to peer 88eead6a17f54dd88c723744e0aa5b50. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.230399  5410 tablet_replica.cc:333] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.231089  5410 raft_consensus.cc:2243] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.231789  5410 raft_consensus.cc:2272] T e04a0fb0fb0342bd935451e239964483 P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.234189  5410 tablet_replica.cc:333] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.234853  5410 raft_consensus.cc:2243] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:54.235018 15481 consensus_peers.cc:597] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.235275  5410 raft_consensus.cc:2272] T 1fc719509cbf4d3aad5be6412ec80022 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.237617  5410 tablet_replica.cc:333] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.238188  5410 raft_consensus.cc:2243] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.238672  5410 raft_consensus.cc:2272] T e02c064babd84e109140e2fa50a9ad94 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.240490  5410 tablet_replica.cc:333] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.241037  5410 raft_consensus.cc:2243] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.241887  5410 raft_consensus.cc:2272] T c8c2f416704b4f56b52001ddbc062bec P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.243949  5410 tablet_replica.cc:333] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.244481  5410 raft_consensus.cc:2243] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.245236  5410 raft_consensus.cc:2272] T 3ba45fbc82014336a9bddd17603ed390 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.247212  5410 tablet_replica.cc:333] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.247735  5410 raft_consensus.cc:2243] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.248235  5410 raft_consensus.cc:2272] T 12527dc46ef54e4f944ce04478f71b8d P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.250319  5410 tablet_replica.cc:333] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.250957  5410 raft_consensus.cc:2243] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.251449  5410 raft_consensus.cc:2272] T 318aa120381647db94d137ca2de5d87b P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.251631 15407 consensus_peers.cc:597] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.253742  5410 tablet_replica.cc:333] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.254566  5410 raft_consensus.cc:2243] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.255153  5410 raft_consensus.cc:2272] T f585eec076224b80b0b6d6447d292eeb P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.257184  5410 tablet_replica.cc:333] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.257728  5410 raft_consensus.cc:2243] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.258136  5410 raft_consensus.cc:2272] T 3942225e71d7482d869c941ce55f4971 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.260149  5410 tablet_replica.cc:333] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.260726  5410 raft_consensus.cc:2243] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.261567  5410 raft_consensus.cc:2272] T af51ee9eaabb4a29a989844d36c3f423 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.263598  5410 tablet_replica.cc:333] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.264130  5410 raft_consensus.cc:2243] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.264549  5410 raft_consensus.cc:2272] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.266460  5410 tablet_replica.cc:333] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.267009  5410 raft_consensus.cc:2243] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.267414  5410 raft_consensus.cc:2272] T 427b294c2f944e7ca234578e92b6126a P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.269136  5410 tablet_replica.cc:333] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.269598  5410 raft_consensus.cc:2243] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.270323  5410 raft_consensus.cc:2272] T c124e9e01448409d9a64892dddf7c610 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.272585  5410 tablet_replica.cc:333] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.273216  5410 raft_consensus.cc:2243] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.273659  5410 raft_consensus.cc:2272] T cd5f83d74ae74d00b588a9fd88c8b0b0 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.276185  5410 tablet_replica.cc:333] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.276810  5410 raft_consensus.cc:2243] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.277315  5410 raft_consensus.cc:2272] T a6e85352a3ac49939b3bdab9c612b76e P ee5629f4db5540b9a61ea4d7a180ee6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.279763  5410 tablet_replica.cc:333] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.280397  5410 raft_consensus.cc:2243] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.281440  5410 raft_consensus.cc:2272] T 3ed4922079864d94a4a771fb9466d63c P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.283435  5410 tablet_replica.cc:333] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.283938  5410 raft_consensus.cc:2243] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.284386  5410 raft_consensus.cc:2272] T f34db7828d184304ab99b4216ac1798e P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.286748  5410 tablet_replica.cc:333] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.287395  5410 raft_consensus.cc:2243] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.287855  5410 raft_consensus.cc:2272] T 9cc957b5c5c04495a4afb4b265db88b1 P ee5629f4db5540b9a61ea4d7a180ee6b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.288669 15407 consensus_peers.cc:597] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.290099  5410 tablet_replica.cc:333] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.290720  5410 raft_consensus.cc:2243] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.291138  5410 raft_consensus.cc:2272] T de52504e9e244dc99cc89e67e9281f3d P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.292950  5410 tablet_replica.cc:333] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b: stopping tablet replica
I20260709 13:21:54.293416  5410 raft_consensus.cc:2243] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.294253  5410 raft_consensus.cc:2272] T c6bde75c2a4c44b0beb23e22159b9fa2 P ee5629f4db5540b9a61ea4d7a180ee6b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.297238 15407 consensus_peers.cc:597] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.323669 15407 consensus_peers.cc:597] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.365435 15407 consensus_peers.cc:597] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.373189  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
W20260709 13:21:54.403707 15481 consensus_peers.cc:597] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.411309 15481 consensus_peers.cc:597] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.428076 15407 consensus_peers.cc:597] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.475256  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
W20260709 13:21:54.498799 15484 consensus_peers.cc:597] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.515355 15481 consensus_peers.cc:597] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.530027 15407 consensus_peers.cc:597] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.545876 15481 consensus_peers.cc:597] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.547097 15484 consensus_peers.cc:597] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.548729  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:21:54.548800 15484 consensus_peers.cc:597] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.549779  5410 tablet_replica.cc:333] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.550791  5410 raft_consensus.cc:2243] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.551378  5410 raft_consensus.cc:2272] T 69858931805f420db0a026c73d7b0f06 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.554318  5410 tablet_replica.cc:333] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.555155  5410 raft_consensus.cc:2243] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.555702  5410 raft_consensus.cc:2272] T 979c935f63114ce6b2c31919b88a250f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.558647  5410 tablet_replica.cc:333] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.559434  5410 raft_consensus.cc:2243] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.560446  5410 raft_consensus.cc:2272] T 260309c1df6b490ea4502dd13e14f692 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.563153  5410 tablet_replica.cc:333] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.563917  5410 raft_consensus.cc:2243] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.564451  5410 raft_consensus.cc:2272] T 677797bb8c284c9d8a85ce81e8052fed P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.566315  5410 tablet_replica.cc:333] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.566975  5410 raft_consensus.cc:2243] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.567497  5410 raft_consensus.cc:2272] T a49a8fabd3bc4946993584f83f34f587 P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.569967  5410 tablet_replica.cc:333] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.570879  5410 raft_consensus.cc:2243] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.571895  5410 raft_consensus.cc:2272] T d81a8d43d2a34dc38f37e6c6c03c29c3 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.574914  5410 tablet_replica.cc:333] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
W20260709 13:21:54.575440 15407 consensus_peers.cc:597] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.575647  5410 raft_consensus.cc:2243] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.576717  5410 raft_consensus.cc:2272] T 2588887022d8411bba3692826b2fe934 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.579303  5410 tablet_replica.cc:333] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
W20260709 13:21:54.579638 15407 consensus_peers.cc:597] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.580021  5410 raft_consensus.cc:2243] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.580513  5410 raft_consensus.cc:2272] T 500dbd579fd848328ce9984276eb0e04 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.582737  5410 tablet_replica.cc:333] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.583420  5410 raft_consensus.cc:2243] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.584138  5410 raft_consensus.cc:2272] T e5546361e1c64e36951aeb942521a72b P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.586222  5410 tablet_replica.cc:333] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.586933  5410 raft_consensus.cc:2243] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.587329  5410 raft_consensus.cc:2272] T f585eec076224b80b0b6d6447d292eeb P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.589093  5410 tablet_replica.cc:333] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.589752  5410 raft_consensus.cc:2243] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.590189  5410 raft_consensus.cc:2272] T 1c87965874f942deb1c496acbfed4a1f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.592566  5410 tablet_replica.cc:333] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.593268  5410 raft_consensus.cc:2243] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.593778  5410 raft_consensus.cc:2272] T c124e9e01448409d9a64892dddf7c610 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.596087  5410 tablet_replica.cc:333] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.596586  5410 raft_consensus.cc:2243] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.596966  5410 raft_consensus.cc:2272] T 43fc5b0260a048f4a581d736b3b2191e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.598778  5410 tablet_replica.cc:333] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.599254  5410 raft_consensus.cc:2243] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.599608  5410 raft_consensus.cc:2272] T efc8987a44524529b4fcedae3a7ff32f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.601536  5410 tablet_replica.cc:333] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.602041  5410 raft_consensus.cc:2243] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.602442  5410 raft_consensus.cc:2272] T f9cf658f5d1340238f97e4d01fa8c07e P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.604801  5410 tablet_replica.cc:333] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.605502  5410 raft_consensus.cc:2243] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.605929  5410 raft_consensus.cc:2272] T d4539a0cb3c54de2862153a2f08eba04 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.608657  5410 tablet_replica.cc:333] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.609401  5410 raft_consensus.cc:2243] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.609833  5410 raft_consensus.cc:2272] T f7b5a27333a34cfeb46c592917d72a5f P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.610158 15484 consensus_peers.cc:597] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.612247  5410 tablet_replica.cc:333] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.613131  5410 raft_consensus.cc:2243] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.614147  5410 raft_consensus.cc:2272] T 4e39f7656f2748f994ca243f51380c1c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.614565 15484 consensus_peers.cc:597] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.615422 15481 consensus_peers.cc:597] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Couldn't send request to peer ee5629f4db5540b9a61ea4d7a180ee6b. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.616559  5410 tablet_replica.cc:333] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.617192  5410 raft_consensus.cc:2243] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.617671  5410 raft_consensus.cc:2272] T f133a6db45994178b473a5b1809d367f P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.619850  5410 tablet_replica.cc:333] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.620409  5410 raft_consensus.cc:2243] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.620815  5410 raft_consensus.cc:2272] T 73732c746e334164b459508a1eeda5ec P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.623325  5410 tablet_replica.cc:333] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.623991  5410 raft_consensus.cc:2243] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.624575  5410 raft_consensus.cc:2272] T b99ac0daff22426fb7dfa484f28c548d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.627143  5410 tablet_replica.cc:333] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.627678  5410 raft_consensus.cc:2243] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.628099  5410 raft_consensus.cc:2272] T 100da53293964979b47f6fe528856727 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.630295  5410 tablet_replica.cc:333] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.630932  5410 raft_consensus.cc:2243] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.631474  5410 raft_consensus.cc:2272] T 128c6f471a6b4c53bdd2c868f551b542 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.634155  5410 tablet_replica.cc:333] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.634991  5410 raft_consensus.cc:2243] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.635931  5410 raft_consensus.cc:2272] T 1fc719509cbf4d3aad5be6412ec80022 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.638027  5410 tablet_replica.cc:333] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.638513  5410 raft_consensus.cc:2243] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.638924  5410 raft_consensus.cc:2272] T cd5f83d74ae74d00b588a9fd88c8b0b0 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.640674  5410 tablet_replica.cc:333] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.641144  5410 raft_consensus.cc:2243] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.641871  5410 raft_consensus.cc:2272] T 65aff9e32079465b85d18fff1c73b28a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.643833  5410 tablet_replica.cc:333] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.644282  5410 raft_consensus.cc:2243] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.644632  5410 raft_consensus.cc:2272] T e02c064babd84e109140e2fa50a9ad94 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.646242  5410 tablet_replica.cc:333] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.646817  5410 raft_consensus.cc:2243] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.647238  5410 raft_consensus.cc:2272] T c8c2f416704b4f56b52001ddbc062bec P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.649135  5410 tablet_replica.cc:333] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.649592  5410 raft_consensus.cc:2243] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.650338  5410 raft_consensus.cc:2272] T f34db7828d184304ab99b4216ac1798e P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.652307  5410 tablet_replica.cc:333] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.652809  5410 raft_consensus.cc:2243] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.653183  5410 raft_consensus.cc:2272] T 5e1ddcb87ba840e68a9c62551336aa0c P d646a2b597824d24b522d9ef5ff5adc3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.654997  5410 tablet_replica.cc:333] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.655561  5410 raft_consensus.cc:2243] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.656293  5410 raft_consensus.cc:2272] T 9cc957b5c5c04495a4afb4b265db88b1 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.658318  5410 tablet_replica.cc:333] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.658916  5410 raft_consensus.cc:2243] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.659669  5410 raft_consensus.cc:2272] T 06ec5b047f9d4549ac7e3bf2d5f71bb2 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.661413  5410 tablet_replica.cc:333] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.661922  5410 raft_consensus.cc:2243] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.662731  5410 raft_consensus.cc:2272] T 427b294c2f944e7ca234578e92b6126a P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.665113  5410 tablet_replica.cc:333] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.665881  5410 raft_consensus.cc:2243] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.666436  5410 raft_consensus.cc:2272] T c6bde75c2a4c44b0beb23e22159b9fa2 P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.668974  5410 tablet_replica.cc:333] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.669410  5410 raft_consensus.cc:2243] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.669901  5410 raft_consensus.cc:2272] T 2c9fb81bccaf408cba956547d37caf3f P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.672313  5410 tablet_replica.cc:333] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.673007  5410 raft_consensus.cc:2243] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.673491  5410 raft_consensus.cc:2272] T 4f14bc0f2237437888f16c60a62840de P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.675601  5410 tablet_replica.cc:333] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.676112  5410 raft_consensus.cc:2243] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.676898  5410 raft_consensus.cc:2272] T 3288c26b19cb47dfb6035bddf30566df P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.678937  5410 tablet_replica.cc:333] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.679430  5410 raft_consensus.cc:2243] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.679857  5410 raft_consensus.cc:2272] T 54afc04cc1b742d1b309764bb898bc6b P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.681705  5410 tablet_replica.cc:333] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.682190  5410 raft_consensus.cc:2243] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.683020  5410 raft_consensus.cc:2272] T de52504e9e244dc99cc89e67e9281f3d P d646a2b597824d24b522d9ef5ff5adc3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.685608  5410 tablet_replica.cc:333] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.686205  5410 raft_consensus.cc:2243] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.686738  5410 raft_consensus.cc:2272] T 3593559594274cc7aef7c38772401110 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.688110 15484 consensus_peers.cc:597] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:54.688752 15484 consensus_peers.cc:597] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.689145  5410 tablet_replica.cc:333] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.689766  5410 raft_consensus.cc:2243] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.690559  5410 raft_consensus.cc:2272] T f542e50c9876452b88a3f17ea981a62c P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.692883  5410 tablet_replica.cc:333] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.693432  5410 raft_consensus.cc:2243] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.693917  5410 raft_consensus.cc:2272] T 67f48e3fa27c4c6487669b0d45130af7 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.696406  5410 tablet_replica.cc:333] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.697001  5410 raft_consensus.cc:2243] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.697450  5410 raft_consensus.cc:2272] T e8d49dec28d44105b80839cb58705961 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.699805  5410 tablet_replica.cc:333] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3: stopping tablet replica
I20260709 13:21:54.700269  5410 raft_consensus.cc:2243] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:21:54.700421 15484 consensus_peers.cc:597] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.701078  5410 raft_consensus.cc:2272] T 3942225e71d7482d869c941ce55f4971 P d646a2b597824d24b522d9ef5ff5adc3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.748881 15484 consensus_peers.cc:597] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.775436  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
I20260709 13:21:54.837085  5410 tablet_server.cc:179] TabletServer@127.5.72.132:0 shutting down...
W20260709 13:21:54.862771 15484 consensus_peers.cc:597] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.131:35805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.878265  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:54.879043  5410 tablet_replica.cc:333] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.879701  5410 raft_consensus.cc:2243] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.880191  5410 raft_consensus.cc:2272] T 20d52192ec7445f4a801452a645af6ba P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.882742  5410 tablet_replica.cc:333] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.883364  5410 raft_consensus.cc:2243] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.884328  5410 raft_consensus.cc:2272] T 4d4413c641a248a9a0cf781768951edc P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.886305  5410 tablet_replica.cc:333] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.886927  5410 raft_consensus.cc:2243] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.887676  5410 raft_consensus.cc:2272] T 69858931805f420db0a026c73d7b0f06 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.889539  5410 tablet_replica.cc:333] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.890234  5410 raft_consensus.cc:2243] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.891253  5410 raft_consensus.cc:2272] T 500dbd579fd848328ce9984276eb0e04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.894024  5410 tablet_replica.cc:333] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
W20260709 13:21:54.894496 15484 consensus_peers.cc:597] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.131:35805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.894768  5410 raft_consensus.cc:2243] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.895303  5410 raft_consensus.cc:2272] T 7bad35d090e945218853dd63c340674b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.897256  5410 tablet_replica.cc:333] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.897837  5410 raft_consensus.cc:2243] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.898224  5410 raft_consensus.cc:2272] T 3ed4922079864d94a4a771fb9466d63c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.900422  5410 tablet_replica.cc:333] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.901340  5410 raft_consensus.cc:2243] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.901921  5410 raft_consensus.cc:2272] T 677797bb8c284c9d8a85ce81e8052fed P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.904388  5410 tablet_replica.cc:333] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.905257  5410 raft_consensus.cc:2243] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.905756  5410 raft_consensus.cc:2272] T a49a8fabd3bc4946993584f83f34f587 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.907968  5410 tablet_replica.cc:333] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.908620  5410 raft_consensus.cc:2243] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.909103  5410 raft_consensus.cc:2272] T d81a8d43d2a34dc38f37e6c6c03c29c3 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:54.909119 15484 consensus_peers.cc:597] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b -> Peer d646a2b597824d24b522d9ef5ff5adc3 (127.5.72.131:35805): Couldn't send request to peer d646a2b597824d24b522d9ef5ff5adc3. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.131:35805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:54.911303  5410 tablet_replica.cc:333] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.911832  5410 raft_consensus.cc:2243] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.912190  5410 raft_consensus.cc:2272] T 43fc5b0260a048f4a581d736b3b2191e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.914103  5410 tablet_replica.cc:333] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.914588  5410 raft_consensus.cc:2243] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.914988  5410 raft_consensus.cc:2272] T efc8987a44524529b4fcedae3a7ff32f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.917119  5410 tablet_replica.cc:333] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.917609  5410 raft_consensus.cc:2243] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.918457  5410 raft_consensus.cc:2272] T f9cf658f5d1340238f97e4d01fa8c07e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.918521 16453 raft_consensus.cc:493] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 88eead6a17f54dd88c723744e0aa5b50)
I20260709 13:21:54.919088 16453 raft_consensus.cc:515] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7f36fa6b484a7b8859acbcd86c4f0b" member_type: VOTER last_known_addr { host: "127.5.72.132" port: 41633 } } peers { permanent_uuid: "88eead6a17f54dd88c723744e0aa5b50" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43313 } } peers { permanent_uuid: "ee5629f4db5540b9a61ea4d7a180ee6b" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 46479 } }
I20260709 13:21:54.920753  5410 tablet_replica.cc:333] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.920780 16453 leader_election.cc:290] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313), ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479)
I20260709 13:21:54.921744  5410 raft_consensus.cc:2243] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.922345  5410 raft_consensus.cc:2272] T d4539a0cb3c54de2862153a2f08eba04 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.924666  5410 tablet_replica.cc:333] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
W20260709 13:21:54.925101 15484 leader_election.cc:336] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 88eead6a17f54dd88c723744e0aa5b50 (127.5.72.129:43313): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43313: connect: Connection refused (error 111)
I20260709 13:21:54.925472  5410 raft_consensus.cc:2243] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:54.925837 15481 leader_election.cc:336] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ee5629f4db5540b9a61ea4d7a180ee6b (127.5.72.130:46479): Network error: Client connection negotiation failed: client connection to 127.5.72.130:46479: connect: Connection refused (error 111)
I20260709 13:21:54.926028  5410 raft_consensus.cc:2272] T 0b6805b1cee04dff9be875c8d6d74e7d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.926342 15481 leader_election.cc:304] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b7f36fa6b484a7b8859acbcd86c4f0b; no voters: 88eead6a17f54dd88c723744e0aa5b50, ee5629f4db5540b9a61ea4d7a180ee6b
I20260709 13:21:54.927083 16453 raft_consensus.cc:2749] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:21:54.928738  5410 tablet_replica.cc:333] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.929453  5410 raft_consensus.cc:2243] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.930212  5410 raft_consensus.cc:2272] T 73732c746e334164b459508a1eeda5ec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.932276  5410 tablet_replica.cc:333] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.932834  5410 raft_consensus.cc:2243] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.933578  5410 raft_consensus.cc:2272] T b99ac0daff22426fb7dfa484f28c548d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.935516  5410 tablet_replica.cc:333] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.936041  5410 raft_consensus.cc:2243] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.936755  5410 raft_consensus.cc:2272] T 100da53293964979b47f6fe528856727 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.938480  5410 tablet_replica.cc:333] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.939056  5410 raft_consensus.cc:2243] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.939457  5410 raft_consensus.cc:2272] T 128c6f471a6b4c53bdd2c868f551b542 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.941203  5410 tablet_replica.cc:333] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.941687  5410 raft_consensus.cc:2243] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.942142  5410 raft_consensus.cc:2272] T d95073a3b15444a4880921aaad3f82a1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.944149  5410 tablet_replica.cc:333] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.944615  5410 raft_consensus.cc:2243] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.945297  5410 raft_consensus.cc:2272] T 54afc04cc1b742d1b309764bb898bc6b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.947170  5410 tablet_replica.cc:333] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.947659  5410 raft_consensus.cc:2243] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.948031  5410 raft_consensus.cc:2272] T 8d531517bd614ae3af03be27016eb596 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.949944  5410 tablet_replica.cc:333] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.950419  5410 raft_consensus.cc:2243] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.950825  5410 raft_consensus.cc:2272] T 55c5fe7b4ba54f94a897b776dcc58565 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.952627  5410 tablet_replica.cc:333] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.953092  5410 raft_consensus.cc:2243] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.953446  5410 raft_consensus.cc:2272] T 1fc719509cbf4d3aad5be6412ec80022 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.955413  5410 tablet_replica.cc:333] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.955931  5410 raft_consensus.cc:2243] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.956645  5410 raft_consensus.cc:2272] T ffef2ecb155c41daba889a3314b66527 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.958669  5410 tablet_replica.cc:333] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.959178  5410 raft_consensus.cc:2243] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.959517  5410 raft_consensus.cc:2272] T e04a0fb0fb0342bd935451e239964483 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.961274  5410 tablet_replica.cc:333] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.961722  5410 raft_consensus.cc:2243] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.962427  5410 raft_consensus.cc:2272] T 2c9fb81bccaf408cba956547d37caf3f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.964404  5410 tablet_replica.cc:333] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.964882  5410 raft_consensus.cc:2243] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.965222  5410 raft_consensus.cc:2272] T 4f14bc0f2237437888f16c60a62840de P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.967005  5410 tablet_replica.cc:333] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.967458  5410 raft_consensus.cc:2243] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.967792  5410 raft_consensus.cc:2272] T 5e1ddcb87ba840e68a9c62551336aa0c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.969483  5410 tablet_replica.cc:333] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.970085  5410 raft_consensus.cc:2243] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.970435  5410 raft_consensus.cc:2272] T 9cc957b5c5c04495a4afb4b265db88b1 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.972309  5410 tablet_replica.cc:333] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.972790  5410 raft_consensus.cc:2243] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.973593  5410 raft_consensus.cc:2272] T 67f48e3fa27c4c6487669b0d45130af7 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.975303  5410 tablet_replica.cc:333] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.975751  5410 raft_consensus.cc:2243] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.976461  5410 raft_consensus.cc:2272] T e8d49dec28d44105b80839cb58705961 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.978354  5410 tablet_replica.cc:333] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.978844  5410 raft_consensus.cc:2243] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.979209  5410 raft_consensus.cc:2272] T 4e39f7656f2748f994ca243f51380c1c P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.981758  5410 tablet_replica.cc:333] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.982950  5410 raft_consensus.cc:2243] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:54.984206  5410 raft_consensus.cc:2272] T cd5f83d74ae74d00b588a9fd88c8b0b0 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.988364  5410 tablet_replica.cc:333] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.989358  5410 raft_consensus.cc:2243] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.990111  5410 raft_consensus.cc:2272] T a6e85352a3ac49939b3bdab9c612b76e P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:54.993779  5410 tablet_replica.cc:333] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:54.995153  5410 raft_consensus.cc:2243] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:54.996236  5410 raft_consensus.cc:2272] T 65aff9e32079465b85d18fff1c73b28a P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.000510  5410 tablet_replica.cc:333] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.001819  5410 raft_consensus.cc:2243] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:21:55.003798  5410 raft_consensus.cc:2272] T e02c064babd84e109140e2fa50a9ad94 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.008307  5410 tablet_replica.cc:333] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.009653  5410 raft_consensus.cc:2243] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:55.010586  5410 raft_consensus.cc:2272] T c8c2f416704b4f56b52001ddbc062bec P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.014096  5410 tablet_replica.cc:333] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.014797  5410 raft_consensus.cc:2243] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:55.015269  5410 raft_consensus.cc:2272] T 1c87965874f942deb1c496acbfed4a1f P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.017179  5410 tablet_replica.cc:333] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.017709  5410 raft_consensus.cc:2243] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:55.018141  5410 raft_consensus.cc:2272] T 3ba45fbc82014336a9bddd17603ed390 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.019958  5410 tablet_replica.cc:333] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.020475  5410 raft_consensus.cc:2243] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:55.020893  5410 raft_consensus.cc:2272] T 12527dc46ef54e4f944ce04478f71b8d P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.022750  5410 tablet_replica.cc:333] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.023283  5410 raft_consensus.cc:2243] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:55.023695  5410 raft_consensus.cc:2272] T af51ee9eaabb4a29a989844d36c3f423 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.025514  5410 tablet_replica.cc:333] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.026000  5410 raft_consensus.cc:2243] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:55.026391  5410 raft_consensus.cc:2272] T 318aa120381647db94d137ca2de5d87b P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.028230  5410 tablet_replica.cc:333] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.028707  5410 raft_consensus.cc:2243] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:55.029500  5410 raft_consensus.cc:2272] T f585eec076224b80b0b6d6447d292eeb P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.031278  5410 tablet_replica.cc:333] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b: stopping tablet replica
I20260709 13:21:55.031721  5410 raft_consensus.cc:2243] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:55.032128  5410 raft_consensus.cc:2272] T 3942225e71d7482d869c941ce55f4971 P 1b7f36fa6b484a7b8859acbcd86c4f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.102005  5410 tablet_server.cc:196] TabletServer@127.5.72.132:0 shutdown complete.
I20260709 13:21:55.164867  5410 master.cc:562] Master@127.5.72.190:43355 shutting down...
I20260709 13:21:55.189005  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:55.189601  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:55.189924  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 63d400ef7f0049b8a99c2b99b44294f8: stopping tablet replica
I20260709 13:21:55.210793  5410 master.cc:584] Master@127.5.72.190:43355 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (29783 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:21:55.303683  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:45721
I20260709 13:21:55.304756  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:55.310467 16456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:55.310599 16457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:55.311926 16459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:55.312994  5410 server_base.cc:1061] running on GCE node
I20260709 13:21:55.313968  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:55.314145  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:55.314263  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603315314252 us; error 0 us; skew 500 ppm
I20260709 13:21:55.314797  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:55.317085  5410 webserver.cc:533] Webserver started at http://127.5.72.190:45223/ using document root <none> and password file <none>
I20260709 13:21:55.317654  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:55.317927  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:55.318223  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:55.319355  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "6a0671b84f9c4e3191c4b6e430df9a1f"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.323945  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 13:21:55.327275 16464 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.328127  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:21:55.328418  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "6a0671b84f9c4e3191c4b6e430df9a1f"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.328660  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:55.345636  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:55.346949  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:55.384883  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:45721
I20260709 13:21:55.384997 16515 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:45721 every 8 connection(s)
I20260709 13:21:55.388728 16516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:55.399132 16516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f: Bootstrap starting.
I20260709 13:21:55.404027 16516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:55.408139 16516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f: No bootstrap required, opened a new log
I20260709 13:21:55.410244 16516 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" member_type: VOTER }
I20260709 13:21:55.410692 16516 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:55.410897 16516 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0671b84f9c4e3191c4b6e430df9a1f, State: Initialized, Role: FOLLOWER
I20260709 13:21:55.411474 16516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" member_type: VOTER }
I20260709 13:21:55.411880 16516 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:21:55.412063 16516 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:21:55.412271 16516 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:55.416738 16516 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" member_type: VOTER }
I20260709 13:21:55.417197 16516 leader_election.cc:304] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0671b84f9c4e3191c4b6e430df9a1f; no voters: 
I20260709 13:21:55.418224 16516 leader_election.cc:290] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:21:55.418583 16519 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:55.419956 16519 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 1 LEADER]: Becoming Leader. State: Replica: 6a0671b84f9c4e3191c4b6e430df9a1f, State: Running, Role: LEADER
I20260709 13:21:55.420635 16519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" member_type: VOTER }
I20260709 13:21:55.421258 16516 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:21:55.425488 16521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0671b84f9c4e3191c4b6e430df9a1f. Latest consensus state: current_term: 1 leader_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" member_type: VOTER } }
I20260709 13:21:55.426299 16521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:55.427193 16520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0671b84f9c4e3191c4b6e430df9a1f" member_type: VOTER } }
I20260709 13:21:55.427829 16520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:55.428346 16526 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:21:55.433630 16526 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:21:55.438153  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:21:55.442548 16526 catalog_manager.cc:1383] Generated new cluster ID: c34a703a36af4761a7334802e27f560e
I20260709 13:21:55.442880 16526 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:21:55.469245 16526 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:21:55.470530 16526 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:21:55.484869 16526 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f: Generated new TSK 0
I20260709 13:21:55.485538 16526 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:21:55.505894  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:55.511688 16537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:55.512880 16538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:55.514518  5410 server_base.cc:1061] running on GCE node
W20260709 13:21:55.514830 16540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:55.515826  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:55.516007  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:55.516131  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603315516121 us; error 0 us; skew 500 ppm
I20260709 13:21:55.516598  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:55.518870  5410 webserver.cc:533] Webserver started at http://127.5.72.129:43673/ using document root <none> and password file <none>
I20260709 13:21:55.519317  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:55.519485  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:55.519707  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:55.520717  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "1837d31dc08e4130a103d7e1566c32f2"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.524937  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:55.528218 16545 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.528927  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:21:55.529224  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "1837d31dc08e4130a103d7e1566c32f2"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.529520  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:55.557998  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:55.559274  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:55.560798  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:55.563110  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:55.563340  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.563598  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:55.563786  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.604894  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:43739
I20260709 13:21:55.605005 16607 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:43739 every 8 connection(s)
I20260709 13:21:55.609938  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:55.617033 16612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:55.618130 16613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:55.620056  5410 server_base.cc:1061] running on GCE node
W20260709 13:21:55.621492 16615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:55.622452  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:55.622702  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:55.622867  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603315622855 us; error 0 us; skew 500 ppm
I20260709 13:21:55.623484  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:55.626319  5410 webserver.cc:533] Webserver started at http://127.5.72.130:42415/ using document root <none> and password file <none>
I20260709 13:21:55.626847  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:55.627054  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:55.627374  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:55.628072 16608 heartbeater.cc:344] Connected to a master server at 127.5.72.190:45721
I20260709 13:21:55.628501 16608 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:55.628793  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.629273 16608 heartbeater.cc:507] Master 127.5.72.190:45721 requested a full tablet report, sending...
I20260709 13:21:55.631891 16481 ts_manager.cc:194] Registered new tserver with Master: 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739)
I20260709 13:21:55.634255 16481 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45314
I20260709 13:21:55.634769  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:21:55.638808 16620 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.639719  5410 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:21:55.640061  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.640362  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:55.656317  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:55.657600  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:55.659544  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:55.664147  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:55.664618  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:21:55.665043  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:55.665282  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.714397  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:42469
I20260709 13:21:55.714551 16682 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:42469 every 8 connection(s)
I20260709 13:21:55.719858  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:55.727564 16686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:55.728575 16687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:55.729943 16689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:55.730479 16683 heartbeater.cc:344] Connected to a master server at 127.5.72.190:45721
I20260709 13:21:55.731015 16683 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:55.731556  5410 server_base.cc:1061] running on GCE node
I20260709 13:21:55.732194 16683 heartbeater.cc:507] Master 127.5.72.190:45721 requested a full tablet report, sending...
I20260709 13:21:55.732805  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:55.733101  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:55.733299  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603315733256 us; error 0 us; skew 500 ppm
I20260709 13:21:55.733896  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:55.734428 16481 ts_manager.cc:194] Registered new tserver with Master: bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:55.736053 16481 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45328
I20260709 13:21:55.736477  5410 webserver.cc:533] Webserver started at http://127.5.72.131:46343/ using document root <none> and password file <none>
I20260709 13:21:55.736996  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:55.737213  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:55.737491  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:55.738675  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "470119605b5d4d6e9fe5cd713df24b96"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.779270  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.040s	user 0.008s	sys 0.000s
I20260709 13:21:55.783946 16695 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.784969  5410 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:21:55.785439  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "470119605b5d4d6e9fe5cd713df24b96"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:21:55.785913  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:55.799407  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:55.800729  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:55.802259  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:55.804862  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:55.805105  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.805364  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:55.805559  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:55.849308  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:32875
I20260709 13:21:55.849411 16757 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:32875 every 8 connection(s)
I20260709 13:21:55.867709 16758 heartbeater.cc:344] Connected to a master server at 127.5.72.190:45721
I20260709 13:21:55.868251 16758 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:55.869127 16758 heartbeater.cc:507] Master 127.5.72.190:45721 requested a full tablet report, sending...
I20260709 13:21:55.871711 16481 ts_manager.cc:194] Registered new tserver with Master: 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:55.871819  5410 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0178462s
I20260709 13:21:55.874094 16481 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45342
I20260709 13:21:55.901021 16481 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45344:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:55.904141 16481 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:21:55.992604 16648 tablet_service.cc:1511] Processing CreateTablet for tablet 2010eb67c6434e8b86e7cda6ce403680 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:21:55.993768 16647 tablet_service.cc:1511] Processing CreateTablet for tablet 28db6bee0b60430fafa2173b7e33d009 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:21:55.994463 16648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2010eb67c6434e8b86e7cda6ce403680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:55.995647 16647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28db6bee0b60430fafa2173b7e33d009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.000497 16573 tablet_service.cc:1511] Processing CreateTablet for tablet 2010eb67c6434e8b86e7cda6ce403680 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:21:56.000705 16572 tablet_service.cc:1511] Processing CreateTablet for tablet 28db6bee0b60430fafa2173b7e33d009 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:21:56.002446 16573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2010eb67c6434e8b86e7cda6ce403680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.005415 16571 tablet_service.cc:1511] Processing CreateTablet for tablet 47e30a67505e4ab8b4897b4dcd758b5f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:21:56.006506 16572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28db6bee0b60430fafa2173b7e33d009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.009182 16571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e30a67505e4ab8b4897b4dcd758b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.019189 16646 tablet_service.cc:1511] Processing CreateTablet for tablet 47e30a67505e4ab8b4897b4dcd758b5f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:21:56.021219 16646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e30a67505e4ab8b4897b4dcd758b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.029706 16570 tablet_service.cc:1511] Processing CreateTablet for tablet a83226d19f4f410dbc6b18522f88650f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:21:56.031286 16570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a83226d19f4f410dbc6b18522f88650f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.040366 16723 tablet_service.cc:1511] Processing CreateTablet for tablet 2010eb67c6434e8b86e7cda6ce403680 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:21:56.041294 16774 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.042408 16723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2010eb67c6434e8b86e7cda6ce403680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.047298 16645 tablet_service.cc:1511] Processing CreateTablet for tablet a83226d19f4f410dbc6b18522f88650f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:21:56.048882 16645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a83226d19f4f410dbc6b18522f88650f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.050279 16774 tablet_bootstrap.cc:654] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.058058 16722 tablet_service.cc:1511] Processing CreateTablet for tablet 28db6bee0b60430fafa2173b7e33d009 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:21:56.060256 16722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28db6bee0b60430fafa2173b7e33d009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.068434 16721 tablet_service.cc:1511] Processing CreateTablet for tablet 47e30a67505e4ab8b4897b4dcd758b5f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:21:56.070592 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e30a67505e4ab8b4897b4dcd758b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.083071 16571 tablet_service.cc:1511] Processing CreateTablet for tablet 2b9c76f486114f7c9bcb6d9c127049aa (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:21:56.084573 16571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b9c76f486114f7c9bcb6d9c127049aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.086796 16644 tablet_service.cc:1511] Processing CreateTablet for tablet 2b9c76f486114f7c9bcb6d9c127049aa (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:21:56.088768 16644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b9c76f486114f7c9bcb6d9c127049aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.101018 16720 tablet_service.cc:1511] Processing CreateTablet for tablet a83226d19f4f410dbc6b18522f88650f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:21:56.101606 16569 tablet_service.cc:1511] Processing CreateTablet for tablet 4a5ed3262ca74c7db89c4ce746224d55 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:21:56.103222 16720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a83226d19f4f410dbc6b18522f88650f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.103493 16569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a5ed3262ca74c7db89c4ce746224d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.121129 16719 tablet_service.cc:1511] Processing CreateTablet for tablet 2b9c76f486114f7c9bcb6d9c127049aa (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:21:56.124367 16643 tablet_service.cc:1511] Processing CreateTablet for tablet 4a5ed3262ca74c7db89c4ce746224d55 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:21:56.127404 16719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b9c76f486114f7c9bcb6d9c127049aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.126403 16643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a5ed3262ca74c7db89c4ce746224d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.151374 16570 tablet_service.cc:1511] Processing CreateTablet for tablet 73ebda95e06549f78a6f2fa1ae77b78d (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:21:56.155789 16718 tablet_service.cc:1511] Processing CreateTablet for tablet 4a5ed3262ca74c7db89c4ce746224d55 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:21:56.159747 16718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a5ed3262ca74c7db89c4ce746224d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.176239 16642 tablet_service.cc:1511] Processing CreateTablet for tablet 73ebda95e06549f78a6f2fa1ae77b78d (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:21:56.183382 16642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ebda95e06549f78a6f2fa1ae77b78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.191982 16570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ebda95e06549f78a6f2fa1ae77b78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.208082 16773 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.212481 16717 tablet_service.cc:1511] Processing CreateTablet for tablet 73ebda95e06549f78a6f2fa1ae77b78d (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:21:56.214095 16717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ebda95e06549f78a6f2fa1ae77b78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.225315 16774 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.227458 16774 ts_tablet_manager.cc:1403] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.187s	user 0.062s	sys 0.088s
I20260709 13:21:56.238134 16645 tablet_service.cc:1511] Processing CreateTablet for tablet 864d99ab49f4491480a400ecaf86679d (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:21:56.239671 16716 tablet_service.cc:1511] Processing CreateTablet for tablet 864d99ab49f4491480a400ecaf86679d (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:21:56.240381 16645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864d99ab49f4491480a400ecaf86679d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.241065 16716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864d99ab49f4491480a400ecaf86679d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.244256 16774 raft_consensus.cc:359] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.246658 16774 raft_consensus.cc:385] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.248442 16774 raft_consensus.cc:740] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.249720 16571 tablet_service.cc:1511] Processing CreateTablet for tablet a4760b87abc447518a87c5bc3085d3a3 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:21:56.263640 16571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4760b87abc447518a87c5bc3085d3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.252871 16644 tablet_service.cc:1511] Processing CreateTablet for tablet a4760b87abc447518a87c5bc3085d3a3 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:21:56.253630 16721 tablet_service.cc:1511] Processing CreateTablet for tablet a4760b87abc447518a87c5bc3085d3a3 (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:21:56.246465 16569 tablet_service.cc:1511] Processing CreateTablet for tablet 864d99ab49f4491480a400ecaf86679d (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:21:56.267598 16644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4760b87abc447518a87c5bc3085d3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.267650 16721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4760b87abc447518a87c5bc3085d3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.267664 16569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864d99ab49f4491480a400ecaf86679d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.225884 16773 tablet_bootstrap.cc:654] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.295634 16570 tablet_service.cc:1511] Processing CreateTablet for tablet 534a772500454949950c26737dd55d1f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:21:56.297418 16570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 534a772500454949950c26737dd55d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.299539 16643 tablet_service.cc:1511] Processing CreateTablet for tablet 534a772500454949950c26737dd55d1f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:21:56.300868 16643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 534a772500454949950c26737dd55d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.302141 16723 tablet_service.cc:1511] Processing CreateTablet for tablet 534a772500454949950c26737dd55d1f (DEFAULT_TABLE table=test-workload [id=4eef0eea3e3d4ab2ac95e0aadfe98eb7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:21:56.302886 16774 consensus_queue.cc:260] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.303891 16723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 534a772500454949950c26737dd55d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:56.317106 16776 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.320160 16608 heartbeater.cc:499] Master 127.5.72.190:45721 was elected leader, sending a full tablet report...
I20260709 13:21:56.321610 16773 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.322157 16773 ts_tablet_manager.cc:1403] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.006s
I20260709 13:21:56.324306 16774 ts_tablet_manager.cc:1434] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.090s	user 0.004s	sys 0.004s
I20260709 13:21:56.325369 16774 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.327745 16776 tablet_bootstrap.cc:654] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.329785 16773 raft_consensus.cc:359] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.331121 16773 raft_consensus.cc:385] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.331458 16773 raft_consensus.cc:740] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.332940 16774 tablet_bootstrap.cc:654] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.332834 16773 consensus_queue.cc:260] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.350147 16773 ts_tablet_manager.cc:1434] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 13:21:56.352276 16683 heartbeater.cc:499] Master 127.5.72.190:45721 was elected leader, sending a full tablet report...
I20260709 13:21:56.353133 16774 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.353896 16776 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.354277 16774 ts_tablet_manager.cc:1403] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.007s
I20260709 13:21:56.354928 16776 ts_tablet_manager.cc:1403] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.014s
I20260709 13:21:56.359951 16773 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.360560 16776 raft_consensus.cc:359] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.361932 16776 raft_consensus.cc:385] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.362432 16776 raft_consensus.cc:740] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.363754 16776 consensus_queue.cc:260] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.370003 16774 raft_consensus.cc:359] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.371160 16778 raft_consensus.cc:493] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.372028 16778 raft_consensus.cc:515] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.376570 16773 tablet_bootstrap.cc:654] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.377004 16758 heartbeater.cc:499] Master 127.5.72.190:45721 was elected leader, sending a full tablet report...
I20260709 13:21:56.377396 16776 ts_tablet_manager.cc:1434] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.022s	user 0.014s	sys 0.005s
I20260709 13:21:56.378473 16776 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.384626 16773 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.384922 16776 tablet_bootstrap.cc:654] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.385200 16773 ts_tablet_manager.cc:1403] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.000s
I20260709 13:21:56.389365 16778 leader_election.cc:290] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.371460 16774 raft_consensus.cc:385] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.390201 16774 raft_consensus.cc:740] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.388638 16773 raft_consensus.cc:359] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.391044 16773 raft_consensus.cc:385] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.391492 16773 raft_consensus.cc:740] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.391266 16774 consensus_queue.cc:260] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.393493 16773 consensus_queue.cc:260] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.395345 16774 ts_tablet_manager.cc:1434] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.002s
I20260709 13:21:56.400133 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:56.401580 16773 ts_tablet_manager.cc:1434] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:21:56.402118 16774 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
W20260709 13:21:56.402513 16547 leader_election.cc:343] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.403044 16773 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.403347 16782 raft_consensus.cc:493] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.403895 16782 raft_consensus.cc:515] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.412819 16773 tablet_bootstrap.cc:654] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.417182 16774 tablet_bootstrap.cc:654] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.428505 16776 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.429051 16776 ts_tablet_manager.cc:1403] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.035s
I20260709 13:21:56.432085 16776 raft_consensus.cc:359] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.432811 16776 raft_consensus.cc:385] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.433120 16776 raft_consensus.cc:740] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.433820 16776 consensus_queue.cc:260] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.437460 16782 leader_election.cc:290] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739)
W20260709 13:21:56.437922 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:56.444784 16776 ts_tablet_manager.cc:1434] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.004s
I20260709 13:21:56.446214 16776 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.456125 16776 tablet_bootstrap.cc:654] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.485880 16776 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.486441 16776 ts_tablet_manager.cc:1403] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.004s
I20260709 13:21:56.488981 16733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:56.489907 16583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:56.489781 16776 raft_consensus.cc:359] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.490387 16774 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.490705 16776 raft_consensus.cc:385] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.491101 16776 raft_consensus.cc:740] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.491251 16774 ts_tablet_manager.cc:1403] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.089s	user 0.026s	sys 0.031s
I20260709 13:21:56.491263 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
W20260709 13:21:56.492203 16696 leader_election.cc:343] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.492035 16776 consensus_queue.cc:260] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
W20260709 13:21:56.493279 16697 leader_election.cc:343] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.493726 16697 leader_election.cc:304] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96; no voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:21:56.494400 16776 ts_tablet_manager.cc:1434] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:21:56.495131 16790 raft_consensus.cc:2749] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:56.495522 16776 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.495970 16733 raft_consensus.cc:2468] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 0.
I20260709 13:21:56.495919 16774 raft_consensus.cc:359] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.496986 16774 raft_consensus.cc:385] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.497376 16774 raft_consensus.cc:740] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.498338 16774 consensus_queue.cc:260] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.499558 16773 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.500097 16773 ts_tablet_manager.cc:1403] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.097s	user 0.023s	sys 0.029s
I20260709 13:21:56.501271 16774 ts_tablet_manager.cc:1434] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:21:56.502449 16774 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.503230 16773 raft_consensus.cc:359] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.503985 16773 raft_consensus.cc:385] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.504300 16773 raft_consensus.cc:740] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.505081 16773 consensus_queue.cc:260] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.506582 16548 leader_election.cc:304] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:21:56.507675 16773 ts_tablet_manager.cc:1434] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:21:56.508503 16778 raft_consensus.cc:2804] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.508735 16773 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.509084 16778 raft_consensus.cc:493] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.509590 16778 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.539512 16776 tablet_bootstrap.cc:654] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.542795 16774 tablet_bootstrap.cc:654] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.545526 16773 tablet_bootstrap.cc:654] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.548527 16774 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.549172 16774 ts_tablet_manager.cc:1403] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.016s
I20260709 13:21:56.552706 16774 raft_consensus.cc:359] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.553370 16774 raft_consensus.cc:385] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.553843 16790 raft_consensus.cc:493] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.553895 16774 raft_consensus.cc:740] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.554140 16778 raft_consensus.cc:515] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.554482 16790 raft_consensus.cc:515] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.555065 16774 consensus_queue.cc:260] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.557142 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:56.558744 16583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:56.559322 16733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:56.559424 16583 raft_consensus.cc:2468] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 0.
I20260709 13:21:56.559978 16774 ts_tablet_manager.cc:1434] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.002s
I20260709 13:21:56.560020 16733 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.560837 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:56.561239 16774 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
W20260709 13:21:56.562296 16547 leader_election.cc:343] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.563129 16696 leader_election.cc:304] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:21:56.563970 16782 raft_consensus.cc:2804] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.564330 16782 raft_consensus.cc:493] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:56.564460 16697 leader_election.cc:343] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.564692 16782 raft_consensus.cc:3060] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.565138 16790 leader_election.cc:290] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:56.554440 16773 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.566350 16773 ts_tablet_manager.cc:1403] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.058s	user 0.025s	sys 0.020s
I20260709 13:21:56.568521 16774 tablet_bootstrap.cc:654] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.569482 16773 raft_consensus.cc:359] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.570276 16773 raft_consensus.cc:385] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.570588 16773 raft_consensus.cc:740] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.571523 16773 consensus_queue.cc:260] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.577327 16773 ts_tablet_manager.cc:1434] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:21:56.578729 16773 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.580775 16733 raft_consensus.cc:2468] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 1.
I20260709 13:21:56.582078 16548 leader_election.cc:304] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:21:56.583736 16782 raft_consensus.cc:515] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.586740 16583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:21:56.587335 16583 raft_consensus.cc:3060] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.588198 16796 raft_consensus.cc:2804] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:56.587843 16773 tablet_bootstrap.cc:654] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.590983 16774 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.591562 16774 ts_tablet_manager.cc:1403] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.021s
I20260709 13:21:56.595547 16774 raft_consensus.cc:359] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.596499 16774 raft_consensus.cc:385] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.596881 16774 raft_consensus.cc:740] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.597833 16774 consensus_queue.cc:260] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.599910 16782 leader_election.cc:290] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:56.600265 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:56.601191 16778 leader_election.cc:290] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Requested vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
W20260709 13:21:56.602253 16697 leader_election.cc:343] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.605577 16583 raft_consensus.cc:2468] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 1.
I20260709 13:21:56.607112 16780 raft_consensus.cc:493] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.607381 16696 leader_election.cc:304] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:21:56.607643 16780 raft_consensus.cc:515] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.609095 16782 raft_consensus.cc:2804] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:56.609730 16782 raft_consensus.cc:697] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:21:56.610880 16782 consensus_queue.cc:237] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.615002 16774 ts_tablet_manager.cc:1434] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.010s
I20260709 13:21:56.615968 16776 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.616163 16774 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.616706 16776 ts_tablet_manager.cc:1403] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.121s	user 0.014s	sys 0.001s
I20260709 13:21:56.621090 16776 raft_consensus.cc:359] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.622045 16776 raft_consensus.cc:385] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.622414 16776 raft_consensus.cc:740] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.622695 16780 leader_election.cc:290] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.624981 16796 raft_consensus.cc:697] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Becoming Leader. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Running, Role: LEADER
I20260709 13:21:56.625152 16774 tablet_bootstrap.cc:654] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.626092 16796 consensus_queue.cc:237] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.626314 16776 consensus_queue.cc:260] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.628896 16776 ts_tablet_manager.cc:1434] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:21:56.629942 16776 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.635974 16773 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.636556 16773 ts_tablet_manager.cc:1403] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.011s
I20260709 13:21:56.637671 16774 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.639176 16479 catalog_manager.cc:5697] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:56.641191 16774 ts_tablet_manager.cc:1403] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.001s
I20260709 13:21:56.644470 16774 raft_consensus.cc:359] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.645146 16774 raft_consensus.cc:385] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.645463 16774 raft_consensus.cc:740] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.646266 16774 consensus_queue.cc:260] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.645987 16481 catalog_manager.cc:5697] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "1837d31dc08e4130a103d7e1566c32f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:56.648840 16774 ts_tablet_manager.cc:1434] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:56.650071 16774 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.653738 16773 raft_consensus.cc:359] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.654464 16773 raft_consensus.cc:385] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.654786 16773 raft_consensus.cc:740] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.655540 16773 consensus_queue.cc:260] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.656278 16776 tablet_bootstrap.cc:654] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.658145 16773 ts_tablet_manager.cc:1434] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:21:56.661046 16583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:56.661734 16583 raft_consensus.cc:2468] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 0.
I20260709 13:21:56.663275 16621 leader_election.cc:304] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:56.664098 16780 raft_consensus.cc:2804] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.664440 16780 raft_consensus.cc:493] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.664753 16780 raft_consensus.cc:3060] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.666143 16774 tablet_bootstrap.cc:654] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.669821 16773 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.672724 16780 raft_consensus.cc:515] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.675889 16583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:21:56.676468 16583 raft_consensus.cc:3060] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.677227 16773 tablet_bootstrap.cc:654] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.683964 16780 leader_election.cc:290] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.686671 16774 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.687282 16774 ts_tablet_manager.cc:1403] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.000s
I20260709 13:21:56.690531 16774 raft_consensus.cc:359] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.691373 16774 raft_consensus.cc:385] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.691740 16774 raft_consensus.cc:740] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.692503 16774 consensus_queue.cc:260] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.694108 16776 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.694679 16776 ts_tablet_manager.cc:1403] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.019s
I20260709 13:21:56.695394 16774 ts_tablet_manager.cc:1434] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:21:56.696547 16774 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.697690 16776 raft_consensus.cc:359] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.687314 16583 raft_consensus.cc:2468] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 1.
I20260709 13:21:56.704147 16774 tablet_bootstrap.cc:654] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.704131 16621 leader_election.cc:304] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:56.705637 16780 raft_consensus.cc:2804] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:56.714052 16782 raft_consensus.cc:493] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.714848 16782 raft_consensus.cc:515] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.718436 16782 leader_election.cc:290] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:56.719940 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:56.720908 16658 raft_consensus.cc:2468] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 0.
I20260709 13:21:56.725673 16697 leader_election.cc:304] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:56.726979 16782 raft_consensus.cc:2804] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.727494 16782 raft_consensus.cc:493] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.727908 16782 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.698479 16776 raft_consensus.cc:385] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.730020 16776 raft_consensus.cc:740] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.731348 16773 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.731913 16773 ts_tablet_manager.cc:1403] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.062s	user 0.029s	sys 0.032s
I20260709 13:21:56.731612 16776 consensus_queue.cc:260] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.735137 16773 raft_consensus.cc:359] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.735874 16773 raft_consensus.cc:385] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.735707 16782 raft_consensus.cc:515] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.736227 16773 raft_consensus.cc:740] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.737031 16773 consensus_queue.cc:260] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.739145 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:21:56.740058 16582 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.743738 16776 ts_tablet_manager.cc:1434] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.000s
I20260709 13:21:56.745273 16776 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.746379 16773 ts_tablet_manager.cc:1434] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 13:21:56.748169 16773 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.753414 16774 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.754040 16774 ts_tablet_manager.cc:1403] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.058s	user 0.035s	sys 0.018s
I20260709 13:21:56.754884 16776 tablet_bootstrap.cc:654] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.755277 16773 tablet_bootstrap.cc:654] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.757210 16774 raft_consensus.cc:359] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.757855 16583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:56.758332 16774 raft_consensus.cc:385] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.758925 16774 raft_consensus.cc:740] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.760073 16774 consensus_queue.cc:260] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.757269 16582 raft_consensus.cc:2468] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 1.
I20260709 13:21:56.763178 16696 leader_election.cc:304] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:21:56.764230 16790 raft_consensus.cc:2804] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:56.767607 16773 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.768352 16773 ts_tablet_manager.cc:1403] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.005s
I20260709 13:21:56.769796 16774 ts_tablet_manager.cc:1434] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:21:56.773766 16790 raft_consensus.cc:697] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:21:56.772379 16773 raft_consensus.cc:359] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.778410 16774 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.774863 16773 raft_consensus.cc:385] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.781986 16814 raft_consensus.cc:493] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.775230 16796 raft_consensus.cc:493] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.785126 16776 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.785099 16814 raft_consensus.cc:515] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.774773 16790 consensus_queue.cc:237] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.785213 16796 raft_consensus.cc:515] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.787451 16776 ts_tablet_manager.cc:1403] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.042s	user 0.023s	sys 0.011s
I20260709 13:21:56.787935 16814 leader_election.cc:290] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.788808 16773 raft_consensus.cc:740] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.790078 16773 consensus_queue.cc:260] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.791142 16776 raft_consensus.cc:359] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.791718 16583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:56.792424 16774 tablet_bootstrap.cc:654] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.793134 16773 ts_tablet_manager.cc:1434] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.024s	user 0.015s	sys 0.004s
W20260709 13:21:56.793313 16621 leader_election.cc:343] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.794157 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:56.794245 16773 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.794896 16658 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 0.
I20260709 13:21:56.796432 16547 leader_election.cc:304] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:56.797127 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:56.797516 16733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:56.798296 16782 leader_election.cc:290] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:56.798386 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:56.799237 16731 raft_consensus.cc:2468] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 0.
I20260709 13:21:56.801005 16623 leader_election.cc:304] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2
I20260709 13:21:56.791936 16776 raft_consensus.cc:385] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.802472 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:56.804813 16773 tablet_bootstrap.cc:654] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.802757 16814 raft_consensus.cc:2804] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.807499 16814 raft_consensus.cc:493] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.807914 16814 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.806823 16658 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.811124 16796 leader_election.cc:290] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.812569 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:56.813298 16731 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 0.
I20260709 13:21:56.815790 16479 catalog_manager.cc:5697] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:56.817083 16814 raft_consensus.cc:515] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.820184 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
W20260709 13:21:56.821713 16621 leader_election.cc:343] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.803675 16798 raft_consensus.cc:2804] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.822662 16798 raft_consensus.cc:493] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.823143 16798 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.802631 16780 raft_consensus.cc:697] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:21:56.802767 16776 raft_consensus.cc:740] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.828135 16773 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.827817 16780 consensus_queue.cc:237] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.828116 16776 consensus_queue.cc:260] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.829398 16732 raft_consensus.cc:3060] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.832216 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:56.832930 16731 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.833087 16773 ts_tablet_manager.cc:1403] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.010s
I20260709 13:21:56.834913 16798 raft_consensus.cc:515] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.838737 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:56.839504 16657 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.839350 16773 raft_consensus.cc:359] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.839627 16732 raft_consensus.cc:2468] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 1.
I20260709 13:21:56.840214 16773 raft_consensus.cc:385] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.840593 16773 raft_consensus.cc:740] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.841243 16731 raft_consensus.cc:2468] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 1.
I20260709 13:21:56.842784 16623 leader_election.cc:304] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2
I20260709 13:21:56.843698 16814 leader_election.cc:290] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.843811 16773 consensus_queue.cc:260] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.845719 16814 raft_consensus.cc:2804] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:56.845916 16773 ts_tablet_manager.cc:1434] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:21:56.846397 16776 ts_tablet_manager.cc:1434] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.058s	user 0.010s	sys 0.001s
I20260709 13:21:56.848730 16776 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.854660 16773 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.856474 16814 raft_consensus.cc:697] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
W20260709 13:21:56.858472 16759 tablet.cc:2406] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:21:56.857860 16814 consensus_queue.cc:237] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.862262 16822 raft_consensus.cc:493] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.862851 16822 raft_consensus.cc:515] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.864399 16481 catalog_manager.cc:5697] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:56.865607 16773 tablet_bootstrap.cc:654] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.868624 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:56.869318 16582 raft_consensus.cc:2468] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 0.
I20260709 13:21:56.870113 16798 leader_election.cc:290] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Requested vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.870695 16621 leader_election.cc:304] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:56.821946 16658 raft_consensus.cc:2468] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 1.
I20260709 13:21:56.871641 16780 raft_consensus.cc:2804] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.872128 16780 raft_consensus.cc:493] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.869341 16657 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 1.
I20260709 13:21:56.872481 16780 raft_consensus.cc:3060] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.873621 16547 leader_election.cc:304] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:56.874465 16798 raft_consensus.cc:2804] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:56.874737 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:56.875682 16798 raft_consensus.cc:697] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Becoming Leader. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Running, Role: LEADER
W20260709 13:21:56.876209 16623 leader_election.cc:343] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.876830 16822 leader_election.cc:290] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.878328 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:56.876576 16798 consensus_queue.cc:237] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.879211 16732 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.880168 16780 raft_consensus.cc:515] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.882308 16780 leader_election.cc:290] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.884446 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:56.884522 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:21:56.885586 16582 raft_consensus.cc:3060] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:56.886458 16623 leader_election.cc:343] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.889025 16774 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.889604 16774 ts_tablet_manager.cc:1403] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.111s	user 0.019s	sys 0.021s
I20260709 13:21:56.892621 16774 raft_consensus.cc:359] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.893306 16774 raft_consensus.cc:385] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.893580 16774 raft_consensus.cc:740] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.894287 16774 consensus_queue.cc:260] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.895433 16773 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.896025 16773 ts_tablet_manager.cc:1403] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.018s
I20260709 13:21:56.896458 16774 ts_tablet_manager.cc:1434] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:21:56.895967 16732 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 1.
I20260709 13:21:56.897228 16776 tablet_bootstrap.cc:654] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.898542 16774 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:21:56.900318 16773 raft_consensus.cc:359] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.901721 16773 raft_consensus.cc:385] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.902225 16773 raft_consensus.cc:740] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.903705 16773 consensus_queue.cc:260] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.897243 16481 catalog_manager.cc:5697] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:56.908453 16773 ts_tablet_manager.cc:1434] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:21:56.910800 16773 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1: Bootstrap starting.
I20260709 13:21:56.915794 16776 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.917136 16776 ts_tablet_manager.cc:1403] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.069s	user 0.019s	sys 0.006s
I20260709 13:21:56.923255 16773 tablet_bootstrap.cc:654] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.916465 16582 raft_consensus.cc:2468] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 1.
I20260709 13:21:56.923204 16776 raft_consensus.cc:359] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.924499 16776 raft_consensus.cc:385] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.925017 16776 raft_consensus.cc:740] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.926203 16621 leader_election.cc:304] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:21:56.926438 16776 consensus_queue.cc:260] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.929539 16814 raft_consensus.cc:2804] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:56.930794 16814 raft_consensus.cc:697] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:21:56.931669 16774 tablet_bootstrap.cc:654] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.932126 16814 consensus_queue.cc:237] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.933266 16479 catalog_manager.cc:5697] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "1837d31dc08e4130a103d7e1566c32f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:56.946776 16774 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: No bootstrap required, opened a new log
I20260709 13:21:56.947294 16774 ts_tablet_manager.cc:1403] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.049s	user 0.023s	sys 0.002s
I20260709 13:21:56.948344 16776 ts_tablet_manager.cc:1434] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.030s	user 0.014s	sys 0.000s
I20260709 13:21:56.949880 16776 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.950304 16774 raft_consensus.cc:359] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.951210 16774 raft_consensus.cc:385] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.951489 16774 raft_consensus.cc:740] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.952282 16774 consensus_queue.cc:260] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.954638 16774 ts_tablet_manager.cc:1434] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:56.958457 16481 catalog_manager.cc:5697] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:56.959836 16776 tablet_bootstrap.cc:654] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.961553 16773 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1: No bootstrap required, opened a new log
I20260709 13:21:56.962123 16773 ts_tablet_manager.cc:1403] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1: Time spent bootstrapping tablet: real 0.052s	user 0.026s	sys 0.022s
I20260709 13:21:56.966149 16773 raft_consensus.cc:359] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.967151 16773 raft_consensus.cc:385] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.967554 16773 raft_consensus.cc:740] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.968634 16773 consensus_queue.cc:260] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.971784 16773 ts_tablet_manager.cc:1434] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:21:56.972693 16814 raft_consensus.cc:493] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.973233 16814 raft_consensus.cc:515] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.976724 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
W20260709 13:21:56.978196 16623 leader_election.cc:343] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:56.975322 16814 leader_election.cc:290] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.978997 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:56.979570 16582 raft_consensus.cc:2468] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 0.
I20260709 13:21:56.975126 16776 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.981178 16776 ts_tablet_manager.cc:1403] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.007s
I20260709 13:21:56.980930 16621 leader_election.cc:304] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:21:56.981865 16814 raft_consensus.cc:2804] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:56.982285 16814 raft_consensus.cc:493] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.982672 16814 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.983986 16776 raft_consensus.cc:359] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.984913 16776 raft_consensus.cc:385] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:56.985380 16776 raft_consensus.cc:740] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:56.986146 16776 consensus_queue.cc:260] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.988425 16776 ts_tablet_manager.cc:1434] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:21:56.989382 16776 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:56.993659 16776 tablet_bootstrap.cc:654] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:56.994665 16814 raft_consensus.cc:515] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:56.997473 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:21:56.997627 16814 leader_election.cc:290] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:56.997830 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:56.998081 16582 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:56.998214 16776 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:56.998785 16776 ts_tablet_manager.cc:1403] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.010s	user 0.009s	sys 0.000s
W20260709 13:21:56.999049 16623 leader_election.cc:343] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:57.001555 16776 raft_consensus.cc:359] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.002246 16776 raft_consensus.cc:385] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:57.002487 16776 raft_consensus.cc:740] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:57.003353 16776 consensus_queue.cc:260] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.005718 16776 ts_tablet_manager.cc:1434] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:21:57.005679 16582 raft_consensus.cc:2468] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 1.
I20260709 13:21:57.006909 16776 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96: Bootstrap starting.
I20260709 13:21:57.006983 16621 leader_election.cc:304] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:21:57.007896 16814 raft_consensus.cc:2804] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:57.008399 16814 raft_consensus.cc:697] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:21:57.009120 16814 consensus_queue.cc:237] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.013262 16776 tablet_bootstrap.cc:654] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:57.018237 16776 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96: No bootstrap required, opened a new log
I20260709 13:21:57.018744 16776 ts_tablet_manager.cc:1403] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96: Time spent bootstrapping tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:21:57.018245 16479 catalog_manager.cc:5697] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:57.020802 16776 raft_consensus.cc:359] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.021283 16776 raft_consensus.cc:385] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:57.021503 16776 raft_consensus.cc:740] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Initialized, Role: FOLLOWER
I20260709 13:21:57.021998 16776 consensus_queue.cc:260] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.023591 16776 ts_tablet_manager.cc:1434] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:21:57.030128 16790 consensus_queue.cc:1048] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.034403 16798 raft_consensus.cc:493] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.034915 16798 raft_consensus.cc:515] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.037271 16798 leader_election.cc:290] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:57.038417 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:57.038647 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:57.039194 16732 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 0.
I20260709 13:21:57.039330 16657 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 0.
I20260709 13:21:57.040196 16548 leader_election.cc:304] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:21:57.040915 16798 raft_consensus.cc:2804] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:57.041275 16798 raft_consensus.cc:493] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:57.041607 16798 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.045462 16657 raft_consensus.cc:3060] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.050086 16798 raft_consensus.cc:515] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.053123 16798 leader_election.cc:290] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Requested vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:57.053470 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:57.053735 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:57.054126 16658 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.054404 16732 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.056810 16790 consensus_queue.cc:1048] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.062914 16814 raft_consensus.cc:493] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.063723 16658 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 1.
I20260709 13:21:57.063572 16814 raft_consensus.cc:515] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.065958 16814 leader_election.cc:290] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:57.066381 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:57.066592 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:57.067222 16547 leader_election.cc:304] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:57.067478 16731 raft_consensus.cc:2393] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bab91c680a2d4eadb1f863c31be8f9c1 in current term 1: Already voted for candidate 1837d31dc08e4130a103d7e1566c32f2 in this term.
I20260709 13:21:57.068420 16798 raft_consensus.cc:2804] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:57.068758 16732 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 1.
I20260709 13:21:57.068939 16798 raft_consensus.cc:697] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Becoming Leader. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Running, Role: LEADER
I20260709 13:21:57.069850 16798 consensus_queue.cc:237] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.073022 16621 leader_election.cc:304] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:21:57.073837 16814 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.079622 16479 catalog_manager.cc:5697] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "1837d31dc08e4130a103d7e1566c32f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:57.081087 16798 raft_consensus.cc:493] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.081245 16814 raft_consensus.cc:2749] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:57.081624 16798 raft_consensus.cc:515] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.084376 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:57.084780 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:57.085556 16732 raft_consensus.cc:2393] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1837d31dc08e4130a103d7e1566c32f2 in current term 1: Already voted for candidate bab91c680a2d4eadb1f863c31be8f9c1 in this term.
I20260709 13:21:57.085601 16796 consensus_queue.cc:1048] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.086156 16798 leader_election.cc:290] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:57.087605 16547 leader_election.cc:304] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2; no voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:21:57.088719 16796 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.096329 16796 raft_consensus.cc:2749] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:57.100718 16796 consensus_queue.cc:1048] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:21:57.170780 16778 raft_consensus.cc:493] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.171271 16778 raft_consensus.cc:515] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.173038 16778 leader_election.cc:290] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:57.173864 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:57.174106 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
I20260709 13:21:57.174494 16657 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 0.
I20260709 13:21:57.174826 16732 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 0.
I20260709 13:21:57.175629 16547 leader_election.cc:304] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:57.176426 16778 raft_consensus.cc:2804] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:57.176765 16778 raft_consensus.cc:493] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:57.177042 16778 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.178093 16790 consensus_queue.cc:1048] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.184119 16778 raft_consensus.cc:515] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.187374 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:57.188091 16778 leader_election.cc:290] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Requested vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:57.188234 16657 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.189196 16782 consensus_queue.cc:1048] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.189857 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "1837d31dc08e4130a103d7e1566c32f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:21:57.190443 16732 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.202975 16732 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 1.
I20260709 13:21:57.204071 16548 leader_election.cc:304] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:21:57.204953 16778 raft_consensus.cc:2804] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:57.205435 16778 raft_consensus.cc:697] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Becoming Leader. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Running, Role: LEADER
I20260709 13:21:57.206249 16778 consensus_queue.cc:237] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.207491 16657 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837d31dc08e4130a103d7e1566c32f2 in term 1.
I20260709 13:21:57.213747 16479 catalog_manager.cc:5697] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "1837d31dc08e4130a103d7e1566c32f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:57.240679 16814 consensus_queue.cc:1048] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.252571 16780 consensus_queue.cc:1048] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.333449 16780 consensus_queue.cc:1048] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.344198 16822 consensus_queue.cc:1048] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.355238 16782 raft_consensus.cc:493] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.355713 16782 raft_consensus.cc:515] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.357533 16782 leader_election.cc:290] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:57.358140 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:57.358539 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:57.359018 16582 raft_consensus.cc:2393] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 470119605b5d4d6e9fe5cd713df24b96 in current term 1: Already voted for candidate bab91c680a2d4eadb1f863c31be8f9c1 in this term.
I20260709 13:21:57.360093 16696 leader_election.cc:304] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96; no voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:21:57.360785 16782 raft_consensus.cc:3060] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.365944 16782 raft_consensus.cc:2749] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:57.366542 16782 raft_consensus.cc:493] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.366961 16782 raft_consensus.cc:515] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:57.368623 16782 leader_election.cc:290] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:57.369366 16582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837d31dc08e4130a103d7e1566c32f2" is_pre_election: true
I20260709 13:21:57.369697 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:57.370216 16582 raft_consensus.cc:2393] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 470119605b5d4d6e9fe5cd713df24b96 in current term 1: Already voted for candidate bab91c680a2d4eadb1f863c31be8f9c1 in this term.
I20260709 13:21:57.371392 16696 leader_election.cc:304] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96; no voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:21:57.372180 16782 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:57.378690 16782 raft_consensus.cc:2749] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:57.378918  5410 ts_manager.cc:295] Set tserver state for 1837d31dc08e4130a103d7e1566c32f2 to MAINTENANCE_MODE
I20260709 13:21:57.379694  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
I20260709 13:21:57.398480 16814 consensus_queue.cc:1048] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:57.410077 16621 consensus_peers.cc:597] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:57.410274  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:57.411891  5410 tablet_replica.cc:333] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.413242  5410 raft_consensus.cc:2243] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.413937  5410 raft_consensus.cc:2272] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.417198  5410 tablet_replica.cc:333] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.418025  5410 raft_consensus.cc:2243] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.418599  5410 raft_consensus.cc:2272] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.421476  5410 tablet_replica.cc:333] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.422325  5410 raft_consensus.cc:2243] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.422896  5410 raft_consensus.cc:2272] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.425304  5410 tablet_replica.cc:333] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.426100  5410 raft_consensus.cc:2243] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.426872  5410 pending_rounds.cc:70] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Trying to abort 1 pending ops.
I20260709 13:21:57.427084  5410 pending_rounds.cc:77] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639186726096896 op_type: NO_OP noop_request { }
I20260709 13:21:57.427423  5410 raft_consensus.cc:2889] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:57.427708  5410 raft_consensus.cc:2272] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:57.430078 16621 consensus_peers.cc:597] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:57.430670  5410 tablet_replica.cc:333] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.431021 16778 consensus_queue.cc:1048] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:57.431540  5410 raft_consensus.cc:2243] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.432127  5410 raft_consensus.cc:2272] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.435045  5410 tablet_replica.cc:333] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.435907  5410 raft_consensus.cc:2243] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.436681  5410 pending_rounds.cc:70] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Trying to abort 1 pending ops.
I20260709 13:21:57.436904  5410 pending_rounds.cc:77] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639187285991424 op_type: NO_OP noop_request { }
I20260709 13:21:57.437237  5410 raft_consensus.cc:2889] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:57.437487  5410 raft_consensus.cc:2272] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:57.439332 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:57.441689  5410 tablet_replica.cc:333] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.442931  5410 raft_consensus.cc:2243] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.442984 16778 consensus_queue.cc:1048] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:57.443464  5410 raft_consensus.cc:2272] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.445981  5410 tablet_replica.cc:333] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.446729  5410 raft_consensus.cc:2243] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.447640  5410 raft_consensus.cc:2272] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.450574  5410 tablet_replica.cc:333] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.451234  5410 raft_consensus.cc:2243] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:57.451715  5410 raft_consensus.cc:2272] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.453953  5410 tablet_replica.cc:333] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: stopping tablet replica
I20260709 13:21:57.454609  5410 raft_consensus.cc:2243] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:57.455646  5410 raft_consensus.cc:2272] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:57.476981  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
I20260709 13:21:57.533350 16780 consensus_queue.cc:1048] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:57.596750 16696 consensus_peers.cc:597] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:57.634264 16696 consensus_peers.cc:597] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:57.721933 16621 consensus_peers.cc:597] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:57.732112 16621 consensus_peers.cc:597] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:58.440428 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:58.599752 16812 raft_consensus.cc:493] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1837d31dc08e4130a103d7e1566c32f2)
I20260709 13:21:58.600307 16812 raft_consensus.cc:515] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:58.601980 16812 leader_election.cc:290] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:21:58.603091 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "470119605b5d4d6e9fe5cd713df24b96" is_pre_election: true
W20260709 13:21:58.605427 16621 leader_election.cc:336] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:58.605749 16621 leader_election.cc:304] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:21:58.606293 16812 raft_consensus.cc:2749] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:58.607461 16817 raft_consensus.cc:493] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1837d31dc08e4130a103d7e1566c32f2)
I20260709 13:21:58.607971 16817 raft_consensus.cc:515] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:58.609638 16817 leader_election.cc:290] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:58.610738 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:58.611406 16658 raft_consensus.cc:2468] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 1.
I20260709 13:21:58.612596 16697 leader_election.cc:304] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:58.613308 16817 raft_consensus.cc:2804] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:58.613649 16696 leader_election.cc:336] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:58.613660 16817 raft_consensus.cc:493] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1837d31dc08e4130a103d7e1566c32f2)
I20260709 13:21:58.614332 16817 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:58.619848 16817 raft_consensus.cc:515] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:58.621651 16817 leader_election.cc:290] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:58.622897 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:58.623586 16658 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:58.625836 16696 leader_election.cc:336] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:58.629426 16658 raft_consensus.cc:2468] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:21:58.630589 16697 leader_election.cc:304] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2
I20260709 13:21:58.631373 16817 raft_consensus.cc:2804] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:58.631910 16817 raft_consensus.cc:697] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:21:58.632721 16817 consensus_queue.cc:237] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:58.640237 16479 catalog_manager.cc:5697] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:58.796614 16621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111) [suppressed 116 similar messages]
I20260709 13:21:58.941560 16817 raft_consensus.cc:493] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1837d31dc08e4130a103d7e1566c32f2)
I20260709 13:21:58.942152 16817 raft_consensus.cc:515] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:58.943992 16817 leader_election.cc:290] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739)
I20260709 13:21:58.944809 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:58.945690 16658 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 1.
I20260709 13:21:58.946782 16697 leader_election.cc:304] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:58.947471 16817 raft_consensus.cc:2804] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:58.947781 16817 raft_consensus.cc:493] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1837d31dc08e4130a103d7e1566c32f2)
I20260709 13:21:58.948077 16817 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:58.948136 16696 leader_election.cc:336] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:58.954077 16817 raft_consensus.cc:515] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:58.956084 16817 leader_election.cc:290] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469), 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739)
I20260709 13:21:58.956851 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:58.957464 16658 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:58.959980 16696 leader_election.cc:336] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:58.963335 16658 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:21:58.964627 16697 leader_election.cc:304] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2
I20260709 13:21:58.965457 16817 raft_consensus.cc:2804] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:58.965988 16817 raft_consensus.cc:697] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:21:58.966882 16817 consensus_queue.cc:237] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:58.974453 16479 catalog_manager.cc:5697] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:59.228924 16817 raft_consensus.cc:493] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.229444 16817 raft_consensus.cc:515] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:59.230262 16658 raft_consensus.cc:1275] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:59.231626 16817 leader_election.cc:290] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:59.231860 16782 consensus_queue.cc:1048] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:59.232805 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:59.233530 16658 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 1.
W20260709 13:21:59.234452 16696 leader_election.cc:336] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:59.235080 16697 leader_election.cc:304] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2
I20260709 13:21:59.235870 16782 raft_consensus.cc:2804] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:59.236191 16782 raft_consensus.cc:493] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:59.236500 16782 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.246584 16782 raft_consensus.cc:515] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:59.249579 16782 leader_election.cc:290] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:59.250228 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:59.251350 16657 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.266119 16657 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:21:59.267517 16697 leader_election.cc:304] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:59.268875 16864 raft_consensus.cc:2804] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:59.269348 16864 raft_consensus.cc:697] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
W20260709 13:21:59.270084 16696 leader_election.cc:336] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:59.270205 16864 consensus_queue.cc:237] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
W20260709 13:21:59.271061 16696 consensus_peers.cc:597] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.278803 16481 catalog_manager.cc:5697] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:59.284242 16864 raft_consensus.cc:493] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.284754 16864 raft_consensus.cc:515] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:59.286718 16864 leader_election.cc:290] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:59.287806 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" is_pre_election: true
I20260709 13:21:59.288525 16658 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 1.
I20260709 13:21:59.289453 16697 leader_election.cc:304] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:21:59.290141 16864 raft_consensus.cc:2804] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:59.290159 16696 leader_election.cc:336] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:59.290467 16864 raft_consensus.cc:493] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:59.290853 16864 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.296617 16864 raft_consensus.cc:515] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:59.298460 16864 leader_election.cc:290] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:21:59.299677 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:21:59.300380 16658 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:59.302038 16696 leader_election.cc:336] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111)
I20260709 13:21:59.306118 16658 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:21:59.307266 16697 leader_election.cc:304] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 470119605b5d4d6e9fe5cd713df24b96, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 1837d31dc08e4130a103d7e1566c32f2
I20260709 13:21:59.308013 16864 raft_consensus.cc:2804] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:59.308467 16864 raft_consensus.cc:697] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:21:59.309187 16864 consensus_queue.cc:237] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:21:59.316684 16481 catalog_manager.cc:5697] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:59.441363 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:21:59.513214 16658 raft_consensus.cc:1275] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:59.514653 16864 consensus_queue.cc:1048] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:59.527107 16696 consensus_peers.cc:597] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:59.718761 16696 consensus_peers.cc:597] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.756095 16658 raft_consensus.cc:1275] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:59.757491 16782 consensus_queue.cc:1048] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:59.771876 16696 consensus_peers.cc:597] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:59.789335 16658 raft_consensus.cc:1275] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:21:59.790704 16782 consensus_queue.cc:1048] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:21:59.873246 16621 consensus_peers.cc:597] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:59.965152 16696 consensus_peers.cc:597] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:00.039330 16621 consensus_peers.cc:597] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:00.208673 16696 consensus_peers.cc:597] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:00.383484 16621 consensus_peers.cc:597] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:00.428957 16621 consensus_peers.cc:597] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:00.442312 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:01.443370 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:01.807410 16696 consensus_peers.cc:597] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:02.196688 16696 consensus_peers.cc:597] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:02.236076 16696 consensus_peers.cc:597] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:02.281862 16621 consensus_peers.cc:597] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:02.352957 16621 consensus_peers.cc:597] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:02.421957 16696 consensus_peers.cc:597] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:02.444308 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:02.497148  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:02.504473 16875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:02.505555 16874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:02.506748 16877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:02.508149  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:02.509946  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:02.510151  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:02.510296  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603322510285 us; error 0 us; skew 500 ppm
I20260709 13:22:02.510867  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:02.513244  5410 webserver.cc:533] Webserver started at http://127.5.72.129:43673/ using document root <none> and password file <none>
I20260709 13:22:02.513669  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:02.513821  5410 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:22:02.517410  5410 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:22:02.521258 16882 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:02.522051  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:22:02.522439  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "1837d31dc08e4130a103d7e1566c32f2"
format_stamp: "Formatted at 2026-07-09 13:21:55 on dist-test-slave-n5kz"
I20260709 13:22:02.522779  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:22:02.539786 16621 consensus_peers.cc:597] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:02.540275 16696 consensus_peers.cc:597] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:02.541632  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:02.542809  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:02.544196  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:02.547540 16889 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:22:02.601183  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:22:02.601511  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.055s	user 0.002s	sys 0.000s
I20260709 13:22:02.601867  5410 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:22:02.606575 16889 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.620772 16889 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.621651 16889 tablet_bootstrap.cc:492] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.622359 16889 ts_tablet_manager.cc:1403] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:22:02.625146 16889 raft_consensus.cc:359] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.625892 16889 raft_consensus.cc:740] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.626721 16889 consensus_queue.cc:260] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.629179 16889 ts_tablet_manager.cc:1434] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:22:02.629926 16889 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.643312 16889 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.644109 16889 tablet_bootstrap.cc:492] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.644675  5410 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:22:02.644670 16889 ts_tablet_manager.cc:1403] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 13:22:02.645046  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.043s	user 0.038s	sys 0.003s
I20260709 13:22:02.647332 16889 raft_consensus.cc:359] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.647888 16889 raft_consensus.cc:740] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.648518 16889 consensus_queue.cc:260] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.650115 16889 ts_tablet_manager.cc:1434] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 13:22:02.651160 16889 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.667805 16889 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.668632 16889 tablet_bootstrap.cc:492] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.669257 16889 ts_tablet_manager.cc:1403] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:22:02.671371 16889 raft_consensus.cc:359] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.671941 16889 raft_consensus.cc:740] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.672564 16889 consensus_queue.cc:260] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.674170 16889 ts_tablet_manager.cc:1434] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:22:02.674870 16889 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.688874 16889 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.689831 16889 tablet_bootstrap.cc:492] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.690378 16889 ts_tablet_manager.cc:1403] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:22:02.692358 16889 raft_consensus.cc:359] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.692884 16889 raft_consensus.cc:740] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.693511 16889 consensus_queue.cc:260] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.694921 16889 ts_tablet_manager.cc:1434] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:02.695547 16889 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.705932 16889 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:22:02.706835 16889 tablet_bootstrap.cc:492] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.707403 16889 ts_tablet_manager.cc:1403] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 13:22:02.708786  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:43739
I20260709 13:22:02.708902 16951 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:43739 every 8 connection(s)
I20260709 13:22:02.709353 16889 raft_consensus.cc:359] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.710045 16889 raft_consensus.cc:740] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.710737 16889 consensus_queue.cc:260] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.712599 16889 ts_tablet_manager.cc:1434] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:22:02.713493 16889 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.713590  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:02.752348 16889 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.754067 16889 tablet_bootstrap.cc:492] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.755538 16889 ts_tablet_manager.cc:1403] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.042s	user 0.025s	sys 0.008s
I20260709 13:22:02.763123 16952 heartbeater.cc:344] Connected to a master server at 127.5.72.190:45721
I20260709 13:22:02.764256 16952 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:02.765188 16889 raft_consensus.cc:359] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.766036 16952 heartbeater.cc:507] Master 127.5.72.190:45721 requested a full tablet report, sending...
I20260709 13:22:02.766515 16889 raft_consensus.cc:740] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.767937 16889 consensus_queue.cc:260] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.770993 16889 ts_tablet_manager.cc:1434] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:22:02.771927 16889 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.772673 16658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
mode: GRACEFUL
new_leader_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:52844
I20260709 13:22:02.773386 16658 raft_consensus.cc:606] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Received request to transfer leadership to TS 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:22:02.776610 16479 ts_manager.cc:194] Re-registered known tserver with Master: 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739)
I20260709 13:22:02.784005 16889 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.784986 16889 tablet_bootstrap.cc:492] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.785712 16889 ts_tablet_manager.cc:1403] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.007s
I20260709 13:22:02.788513 16732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:02.788636 16889 raft_consensus.cc:359] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.789350 16732 raft_consensus.cc:606] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:02.789495 16889 raft_consensus.cc:740] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.790269 16889 consensus_queue.cc:260] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.791538 16479 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37332
I20260709 13:22:02.792972 16889 ts_tablet_manager.cc:1434] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:22:02.793721 16889 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.794740 16952 heartbeater.cc:499] Master 127.5.72.190:45721 was elected leader, sending a full tablet report...
I20260709 13:22:02.795274 16732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:02.795868 16732 raft_consensus.cc:606] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:02.798018  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:02.798715  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:02.799459  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:02.831529 16889 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.832587 16889 tablet_bootstrap.cc:492] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.833288 16889 ts_tablet_manager.cc:1403] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.008s
I20260709 13:22:02.835417 16889 raft_consensus.cc:359] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.835927 16889 raft_consensus.cc:740] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.836571 16889 consensus_queue.cc:260] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.838532 16889 ts_tablet_manager.cc:1434] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.000s
I20260709 13:22:02.839341 16889 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.851945 16889 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:02.852717 16889 tablet_bootstrap.cc:492] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.853289 16889 ts_tablet_manager.cc:1403] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:22:02.855234 16889 raft_consensus.cc:359] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.855733 16889 raft_consensus.cc:740] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.856396 16889 consensus_queue.cc:260] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.857864 16889 ts_tablet_manager.cc:1434] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:02.858795 16889 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap starting.
I20260709 13:22:02.872243 16889 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:22:02.873093 16889 tablet_bootstrap.cc:492] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Bootstrap complete.
I20260709 13:22:02.873737 16889 ts_tablet_manager.cc:1403] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:22:02.875806 16889 raft_consensus.cc:359] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.876572 16889 raft_consensus.cc:740] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837d31dc08e4130a103d7e1566c32f2, State: Initialized, Role: FOLLOWER
I20260709 13:22:02.877241 16889 consensus_queue.cc:260] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.878755 16889 ts_tablet_manager.cc:1434] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:22:02.891520 16969 raft_consensus.cc:993] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1: : Instructing follower 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:02.891984 16873 raft_consensus.cc:1081] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Signalling peer 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:02.893476 16732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:34520
I20260709 13:22:02.893985 16732 raft_consensus.cc:493] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:02.894286 16732 raft_consensus.cc:3060] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:02.899351 16732 raft_consensus.cc:515] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.901294 16732 leader_election.cc:290] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:22:02.902076 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:02.902760 16924 raft_consensus.cc:3060] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:02.907888 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b9c76f486114f7c9bcb6d9c127049aa" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:22:02.908574 16658 raft_consensus.cc:3055] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:02.908908 16658 raft_consensus.cc:740] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:02.909708 16658 consensus_queue.cc:260] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.910821 16924 raft_consensus.cc:2468] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:02.911525 16658 raft_consensus.cc:3060] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:02.912302 16696 leader_election.cc:304] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:22:02.913112 16782 raft_consensus.cc:2804] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:02.914382 16782 raft_consensus.cc:697] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:02.915143 16782 consensus_queue.cc:237] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.919281 16658 raft_consensus.cc:2468] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:02.922802 16481 catalog_manager.cc:5697] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.940133 16924 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:02.945451 16924 pending_rounds.cc:85] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Aborting all ops after (but not including) 0
I20260709 13:22:02.945686 16924 pending_rounds.cc:107] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:22:02.945880 16924 raft_consensus.cc:2889] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:02.951447 16971 raft_consensus.cc:993] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:02.951925 16782 raft_consensus.cc:1081] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:02.953390 16658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:02.953974 16658 raft_consensus.cc:493] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:02.954384 16658 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:02.961083 16658 raft_consensus.cc:515] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.962821 16658 leader_election.cc:290] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:02.963632 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:02.963856 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:02.964387 16924 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:02.964558 16732 raft_consensus.cc:3055] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:02.965039 16732 raft_consensus.cc:740] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:02.965723 16732 consensus_queue.cc:260] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.966706 16732 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:02.970909 16924 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:02.971638 16732 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:02.972232 16621 leader_election.cc:304] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:02.973067 16873 raft_consensus.cc:2804] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:02.973474 16873 raft_consensus.cc:697] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:02.974293 16873 consensus_queue.cc:237] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:02.981032 16481 catalog_manager.cc:5697] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 2 to 3, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:03.038415 16924 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:03.045959 16926 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:03.052389 16926 pending_rounds.cc:85] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Aborting all ops after (but not including) 0
I20260709 13:22:03.052690 16926 pending_rounds.cc:107] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:22:03.052947 16926 raft_consensus.cc:2889] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:03.212579 16926 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:03.286176 16971 raft_consensus.cc:993] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:03.286643 16782 raft_consensus.cc:1081] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:03.288281 16658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:03.288892 16658 raft_consensus.cc:493] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:03.289199 16658 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:03.294356 16658 raft_consensus.cc:515] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:03.295817 16658 leader_election.cc:290] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:03.296723 16926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:03.296782 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:03.297623 16926 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:03.297660 16732 raft_consensus.cc:3055] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:03.298144 16732 raft_consensus.cc:740] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:03.299024 16732 consensus_queue.cc:260] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:03.300297 16732 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:03.305141 16926 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:03.306421 16621 leader_election.cc:304] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:03.307126 16732 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:03.307189 16873 raft_consensus.cc:2804] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:03.307863 16873 raft_consensus.cc:697] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:03.308710 16873 consensus_queue.cc:237] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:03.317301 16481 catalog_manager.cc:5697] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 2 to 3, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:03.376060 16658 raft_consensus.cc:1275] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:03.377261 16971 consensus_queue.cc:1048] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:03.383033 16926 raft_consensus.cc:1275] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:03.384424 16983 consensus_queue.cc:1048] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:03.392288 16924 raft_consensus.cc:1275] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:03.394769 16971 consensus_queue.cc:1048] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:03.400072 16732 raft_consensus.cc:1275] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:03.401827 16969 consensus_queue.cc:1048] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:03.445276 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:03.769650 16926 raft_consensus.cc:1275] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:03.771107 16969 consensus_queue.cc:1048] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:03.779357 16732 raft_consensus.cc:1275] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:03.780889 16873 consensus_queue.cc:1048] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:03.810499  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:03.817891 16658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "534a772500454949950c26737dd55d1f"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
mode: GRACEFUL
new_leader_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:52844
I20260709 13:22:03.818559 16658 raft_consensus.cc:606] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Received request to transfer leadership to TS 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:22:03.821569 16732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:03.822079 16732 raft_consensus.cc:606] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:03.824795 16658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
mode: GRACEFUL
new_leader_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:52844
I20260709 13:22:03.825249 16658 raft_consensus.cc:606] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Received request to transfer leadership to TS 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:22:03.826197  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:03.826802  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:03.827410  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:03.937203 16983 raft_consensus.cc:993] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1: : Instructing follower 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:03.937656 16873 raft_consensus.cc:1081] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Signalling peer 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:03.939046 16732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "534a772500454949950c26737dd55d1f"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:34520
I20260709 13:22:03.939440 16732 raft_consensus.cc:493] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:03.939667 16732 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:03.943997 16732 raft_consensus.cc:515] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:03.945326 16732 leader_election.cc:290] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:22:03.946216 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:22:03.946216 16926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:03.947067 16658 raft_consensus.cc:3055] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:03.947129 16926 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:03.947502 16658 raft_consensus.cc:740] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:03.948261 16658 consensus_queue.cc:260] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:03.949450 16658 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:03.954324 16926 raft_consensus.cc:2468] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:03.955586 16696 leader_election.cc:304] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:22:03.956076 16658 raft_consensus.cc:2468] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:03.956404 16988 raft_consensus.cc:2804] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:03.956965 16988 raft_consensus.cc:697] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:03.957706 16988 consensus_queue.cc:237] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:03.964335 16479 catalog_manager.cc:5697] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:04.289887 16782 raft_consensus.cc:993] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:04.290293 16988 raft_consensus.cc:1081] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:04.291822 16658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:04.292238 16658 raft_consensus.cc:493] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:04.292470 16658 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:04.296988 16658 raft_consensus.cc:515] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:04.298350 16658 leader_election.cc:290] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:04.299264 16926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:04.299384 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:04.299981 16926 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:04.300071 16732 raft_consensus.cc:3055] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:04.300469 16732 raft_consensus.cc:740] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:04.301059 16732 consensus_queue.cc:260] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:04.301913 16732 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:04.305153 16926 raft_consensus.cc:2468] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 2.
I20260709 13:22:04.306170 16732 raft_consensus.cc:2468] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 2.
I20260709 13:22:04.306329 16621 leader_election.cc:304] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:04.307125 16873 raft_consensus.cc:2804] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:04.307607 16873 raft_consensus.cc:697] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:04.308341 16873 consensus_queue.cc:237] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:04.316032 16983 raft_consensus.cc:993] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1: : Instructing follower 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:04.316452 16873 raft_consensus.cc:1081] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Signalling peer 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:04.316166 16479 catalog_manager.cc:5697] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 1 to 2, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:04.318044 16732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:34520
I20260709 13:22:04.318459 16732 raft_consensus.cc:493] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:04.318769 16732 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:04.325033 16732 raft_consensus.cc:515] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:04.327028 16732 leader_election.cc:290] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 4 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:22:04.327720 16926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:04.328480 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864d99ab49f4491480a400ecaf86679d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:22:04.329095 16926 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:04.329198 16658 raft_consensus.cc:3055] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:04.329655 16658 raft_consensus.cc:740] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:04.330397 16658 consensus_queue.cc:260] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:04.332123 16658 raft_consensus.cc:3060] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:04.336083 16926 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 4.
I20260709 13:22:04.337193 16696 leader_election.cc:304] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:22:04.337416 16658 raft_consensus.cc:2468] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 4.
I20260709 13:22:04.337841 16988 raft_consensus.cc:2804] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:04.338411 16988 raft_consensus.cc:697] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 4 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:04.339195 16988 consensus_queue.cc:237] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:04.345170 16479 catalog_manager.cc:5697] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 3 to 4, leader changed from bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 4 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:04.392565 16657 raft_consensus.cc:1275] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:04.393785 16782 consensus_queue.cc:1048] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:04.404682 16924 raft_consensus.cc:1275] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:04.406692 16988 consensus_queue.cc:1048] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:04.446170 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:04.780416 16732 raft_consensus.cc:1275] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:04.781805 16873 consensus_queue.cc:1048] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:04.790845 16924 raft_consensus.cc:1275] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:04.793157 17003 consensus_queue.cc:1048] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:04.837042  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:04.837379 16924 raft_consensus.cc:1275] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:04.838884 16988 consensus_queue.cc:1048] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:04.846308 16732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:04.846864 16732 raft_consensus.cc:606] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:04.850183 16732 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "534a772500454949950c26737dd55d1f"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:04.850932 16732 raft_consensus.cc:606] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:04.853343 16657 raft_consensus.cc:1275] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:04.854480 16658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
mode: GRACEFUL
new_leader_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:52844
I20260709 13:22:04.855147 16658 raft_consensus.cc:606] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Received request to transfer leadership to TS 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:22:04.855095 16988 consensus_queue.cc:1048] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:04.856966  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:04.859793  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:04.860603  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:05.171945 16998 raft_consensus.cc:993] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:05.172344 16988 raft_consensus.cc:1081] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:05.173782 16658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:05.174203 16658 raft_consensus.cc:493] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:05.174425 16658 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:05.179267 16658 raft_consensus.cc:515] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.180696 16658 leader_election.cc:290] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:05.181741 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:05.181743 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e30a67505e4ab8b4897b4dcd758b5f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:05.182327 16732 raft_consensus.cc:3055] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:05.182544 16924 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:05.182718 16732 raft_consensus.cc:740] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:05.183569 16732 consensus_queue.cc:260] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.184441 16732 raft_consensus.cc:3060] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:05.188695 16924 raft_consensus.cc:2468] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:05.189035 16732 raft_consensus.cc:2468] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:05.190223 16621 leader_election.cc:304] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:05.190932 16983 raft_consensus.cc:2804] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:05.191337 16983 raft_consensus.cc:697] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:05.192080 16983 consensus_queue.cc:237] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.198817 16479 catalog_manager.cc:5697] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 2 to 3, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:05.242379 17003 raft_consensus.cc:993] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1: : Instructing follower 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:05.242858 16983 raft_consensus.cc:1081] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Signalling peer 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:05.244350 16732 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:34520
I20260709 13:22:05.244830 16732 raft_consensus.cc:493] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:05.245113 16732 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:05.249626 16732 raft_consensus.cc:515] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.251124 16732 leader_election.cc:290] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 4 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:22:05.252110 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:05.252383 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4760b87abc447518a87c5bc3085d3a3" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:22:05.252782 16924 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:05.252910 16658 raft_consensus.cc:3055] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:05.253279 16658 raft_consensus.cc:740] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:05.254032 16658 consensus_queue.cc:260] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.255265 16658 raft_consensus.cc:3060] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:05.260066 16924 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 4.
I20260709 13:22:05.261351 16696 leader_election.cc:304] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:22:05.261727 16658 raft_consensus.cc:2468] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 4.
I20260709 13:22:05.262032 16988 raft_consensus.cc:2804] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:05.262549 16988 raft_consensus.cc:697] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 4 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:05.263360 16988 consensus_queue.cc:237] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.269613 16479 catalog_manager.cc:5697] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 3 to 4, leader changed from bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 4 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:05.280243 16998 raft_consensus.cc:993] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:05.280745 16988 raft_consensus.cc:1081] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:05.282162 16658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "534a772500454949950c26737dd55d1f"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:05.282765 16658 raft_consensus.cc:493] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:05.283159 16658 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:05.289371 16658 raft_consensus.cc:515] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.291220 16658 leader_election.cc:290] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:05.292052 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:05.292260 16732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "534a772500454949950c26737dd55d1f" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:05.292752 16924 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:05.292846 16732 raft_consensus.cc:3055] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:05.293259 16732 raft_consensus.cc:740] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:05.293880 16732 consensus_queue.cc:260] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.294857 16732 raft_consensus.cc:3060] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:05.298293 16924 raft_consensus.cc:2468] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:05.299377 16621 leader_election.cc:304] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:05.299793 16732 raft_consensus.cc:2468] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:05.300065 16983 raft_consensus.cc:2804] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:05.301354 16983 raft_consensus.cc:697] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:05.302320 16983 consensus_queue.cc:237] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:05.309505 16479 catalog_manager.cc:5697] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 2 to 3, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:05.447104 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:05.636528 16924 raft_consensus.cc:1275] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:05.637755 17012 consensus_queue.cc:1048] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:05.646986 16732 raft_consensus.cc:1275] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:05.648562 16983 consensus_queue.cc:1048] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:05.747787 16924 raft_consensus.cc:1275] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:05.748968 17012 consensus_queue.cc:1048] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:05.757256 16731 raft_consensus.cc:1275] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:05.759833 17013 consensus_queue.cc:1048] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:05.767232 16658 raft_consensus.cc:1275] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:05.768829 16988 consensus_queue.cc:1048] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:05.778400 16926 raft_consensus.cc:1275] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:05.779826 16988 consensus_queue.cc:1048] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:05.869869  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:05.879649 16658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
mode: GRACEFUL
new_leader_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:52844
I20260709 13:22:05.880281 16658 raft_consensus.cc:606] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Received request to transfer leadership to TS 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:22:05.883373 16731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:05.883903 16731 raft_consensus.cc:606] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:05.886855 16658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
mode: GRACEFUL
new_leader_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:52844
I20260709 13:22:05.887530 16658 raft_consensus.cc:606] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 LEADER]: Received request to transfer leadership to TS 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:22:05.888698  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:05.889178  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:05.889856  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:06.194317 16988 raft_consensus.cc:993] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:06.194922 16998 raft_consensus.cc:1081] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:06.196559 16658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:06.197072 16658 raft_consensus.cc:493] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:06.197412 16658 raft_consensus.cc:3060] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:06.205178 16658 raft_consensus.cc:515] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.208148 16658 leader_election.cc:290] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:06.208295 16926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:06.209731 16926 raft_consensus.cc:3060] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:06.211457 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5ed3262ca74c7db89c4ce746224d55" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:06.212001 16731 raft_consensus.cc:3055] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:06.212275 16731 raft_consensus.cc:740] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:06.213239 16731 consensus_queue.cc:260] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.214931 16731 raft_consensus.cc:3060] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:06.217828 16926 raft_consensus.cc:2468] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 2.
I20260709 13:22:06.219369 16621 leader_election.cc:304] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:06.220551 17013 raft_consensus.cc:2804] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:06.220995 17013 raft_consensus.cc:697] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:06.222091 17013 consensus_queue.cc:237] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.223392 16731 raft_consensus.cc:2468] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 2.
I20260709 13:22:06.231823 17003 raft_consensus.cc:993] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1: : Instructing follower 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:06.231487 16480 catalog_manager.cc:5697] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 1 to 2, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:06.232520 17003 raft_consensus.cc:1081] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 LEADER]: Signalling peer 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:06.234515 16731 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:34520
I20260709 13:22:06.235229 16731 raft_consensus.cc:493] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:06.235639 16731 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.241519 16731 raft_consensus.cc:515] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.244642 16731 leader_election.cc:290] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 3 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:22:06.245330 16926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:06.245693 16658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ebda95e06549f78a6f2fa1ae77b78d" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:22:06.246114 16926 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.246240 16658 raft_consensus.cc:3055] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:06.246661 16658 raft_consensus.cc:740] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:06.247639 16658 consensus_queue.cc:260] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.248777 16658 raft_consensus.cc:3060] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.254516 16926 raft_consensus.cc:2468] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 3.
I20260709 13:22:06.256209 16696 leader_election.cc:304] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:22:06.256807 16658 raft_consensus.cc:2468] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 3.
I20260709 13:22:06.257269 16998 raft_consensus.cc:2804] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:06.257965 16998 raft_consensus.cc:697] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:06.259528 16998 consensus_queue.cc:237] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.268636 16480 catalog_manager.cc:5697] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 2 to 3, leader changed from bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 3 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:06.278992 17003 raft_consensus.cc:993] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1: : Instructing follower 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:06.279474 17013 raft_consensus.cc:1081] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Signalling peer 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:06.280905 16731 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:34520
I20260709 13:22:06.281397 16731 raft_consensus.cc:493] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:06.281718 16731 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:06.290472 16731 raft_consensus.cc:515] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.292161 16731 leader_election.cc:290] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:22:06.292850 16926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:06.293200 16657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:22:06.293555 16926 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:06.293828 16657 raft_consensus.cc:3055] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:06.294167 16657 raft_consensus.cc:740] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:06.294968 16657 consensus_queue.cc:260] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.296056 16657 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:06.299032 16926 raft_consensus.cc:2468] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:06.300274 16696 leader_election.cc:304] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:22:06.301088 16998 raft_consensus.cc:2804] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:06.301517 16998 raft_consensus.cc:697] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:06.302546 16998 consensus_queue.cc:237] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.303479 16657 raft_consensus.cc:2468] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:06.310041 16480 catalog_manager.cc:5697] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:06.448300 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:06.656750 16924 raft_consensus.cc:1275] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:06.658257 17013 consensus_queue.cc:1048] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:06.670899 16731 raft_consensus.cc:1275] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:06.672380 17013 consensus_queue.cc:1048] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:06.830530 16924 raft_consensus.cc:1275] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:06.832047 16998 consensus_queue.cc:1048] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:06.844494 16657 raft_consensus.cc:1275] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:06.846442 17039 consensus_queue.cc:1048] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:06.897332  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:06.904647 16731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:06.905129 16731 raft_consensus.cc:606] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:06.907333 16731 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
mode: GRACEFUL
new_leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:39866
I20260709 13:22:06.907856 16731 raft_consensus.cc:606] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Received request to transfer leadership to TS bab91c680a2d4eadb1f863c31be8f9c1
I20260709 13:22:06.910497 16656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
mode: GRACEFUL
new_leader_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:52844
I20260709 13:22:06.911161 16656 raft_consensus.cc:606] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Received request to transfer leadership to TS 470119605b5d4d6e9fe5cd713df24b96
I20260709 13:22:06.912467  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:06.912956  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:06.913542  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:06.916082 16656 raft_consensus.cc:1275] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:06.917294 17039 consensus_queue.cc:1048] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:06.923884 16998 raft_consensus.cc:993] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:06.924630 17039 raft_consensus.cc:1081] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:06.928133 16656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:06.928926 16656 raft_consensus.cc:493] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:06.929373 16656 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.931190 16924 raft_consensus.cc:1275] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:06.933053 17039 consensus_queue.cc:1048] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:06.940512 16656 raft_consensus.cc:515] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.943539 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:06.943890 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28db6bee0b60430fafa2173b7e33d009" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:06.944131 16656 leader_election.cc:290] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:06.944173 16924 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.944538 16731 raft_consensus.cc:3055] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:06.944998 16731 raft_consensus.cc:740] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:06.945848 16731 consensus_queue.cc:260] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.947098 16731 raft_consensus.cc:3060] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.953742 16924 raft_consensus.cc:2468] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:06.954523 16731 raft_consensus.cc:2468] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:06.955117 16621 leader_election.cc:304] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:06.956008 17013 raft_consensus.cc:2804] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:06.956521 17013 raft_consensus.cc:697] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:06.957345 17013 consensus_queue.cc:237] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.963405 16480 catalog_manager.cc:5697] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 2 to 3, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:06.978238 16998 raft_consensus.cc:993] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96: : Instructing follower bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:06.978662 17039 raft_consensus.cc:1081] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Signalling peer bab91c680a2d4eadb1f863c31be8f9c1 to start an election
I20260709 13:22:06.980002 16656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680"
dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
 from {username='slave'} at 127.0.0.1:54922
I20260709 13:22:06.980475 16656 raft_consensus.cc:493] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:06.980782 16656 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.985884 16656 raft_consensus.cc:515] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.987571 16656 leader_election.cc:290] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131:32875)
I20260709 13:22:06.988251 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:06.988561 16731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2010eb67c6434e8b86e7cda6ce403680" candidate_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
I20260709 13:22:06.988915 16924 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.989158 16731 raft_consensus.cc:3055] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:06.989552 16731 raft_consensus.cc:740] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:06.990542 16731 consensus_queue.cc:260] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:06.991901 16731 raft_consensus.cc:3060] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:06.996201 16924 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:06.997388 16621 leader_election.cc:304] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, bab91c680a2d4eadb1f863c31be8f9c1; no voters: 
I20260709 13:22:06.998211 17013 raft_consensus.cc:2804] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:06.998703 17013 raft_consensus.cc:697] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Becoming Leader. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:06.999085 16731 raft_consensus.cc:2468] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bab91c680a2d4eadb1f863c31be8f9c1 in term 3.
I20260709 13:22:06.999608 17013 consensus_queue.cc:237] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:07.007115 16983 raft_consensus.cc:993] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1: : Instructing follower 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:07.007664 17013 raft_consensus.cc:1081] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Signalling peer 470119605b5d4d6e9fe5cd713df24b96 to start an election
I20260709 13:22:07.007102 16480 catalog_manager.cc:5697] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 reported cstate change: term changed from 2 to 3, leader changed from 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131) to bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130). New cstate: current_term: 3 leader_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:07.009241 16731 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f"
dest_uuid: "470119605b5d4d6e9fe5cd713df24b96"
 from {username='slave'} at 127.0.0.1:34520
I20260709 13:22:07.009780 16731 raft_consensus.cc:493] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:07.010103 16731 raft_consensus.cc:3060] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:07.016506 16731 raft_consensus.cc:515] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:07.018122 16731 leader_election.cc:290] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Requested vote from peers 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739), bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469)
I20260709 13:22:07.018999 16924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1837d31dc08e4130a103d7e1566c32f2"
I20260709 13:22:07.019125 16656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a83226d19f4f410dbc6b18522f88650f" candidate_uuid: "470119605b5d4d6e9fe5cd713df24b96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bab91c680a2d4eadb1f863c31be8f9c1"
I20260709 13:22:07.019635 16924 raft_consensus.cc:3060] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:07.019786 16656 raft_consensus.cc:3055] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:07.020205 16656 raft_consensus.cc:740] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bab91c680a2d4eadb1f863c31be8f9c1, State: Running, Role: LEADER
I20260709 13:22:07.021003 16656 consensus_queue.cc:260] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:07.022006 16656 raft_consensus.cc:3060] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:07.024729 16924 raft_consensus.cc:2468] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:07.025763 16696 leader_election.cc:304] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837d31dc08e4130a103d7e1566c32f2, 470119605b5d4d6e9fe5cd713df24b96; no voters: 
I20260709 13:22:07.026597 17039 raft_consensus.cc:2804] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:07.027083 17039 raft_consensus.cc:697] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Becoming Leader. State: Replica: 470119605b5d4d6e9fe5cd713df24b96, State: Running, Role: LEADER
I20260709 13:22:07.028061 16656 raft_consensus.cc:2468] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 470119605b5d4d6e9fe5cd713df24b96 in term 2.
I20260709 13:22:07.028007 17039 consensus_queue.cc:237] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } }
I20260709 13:22:07.035548 16480 catalog_manager.cc:5697] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 reported cstate change: term changed from 1 to 2, leader changed from bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130) to 470119605b5d4d6e9fe5cd713df24b96 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "470119605b5d4d6e9fe5cd713df24b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:07.398552 16924 raft_consensus.cc:1275] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:07.399713 17049 consensus_queue.cc:1048] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:07.406973 16731 raft_consensus.cc:1275] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:07.408315 16983 consensus_queue.cc:1048] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:22:07.449415 16529 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:07.450299 16656 raft_consensus.cc:1275] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:07.451813 17050 consensus_queue.cc:1048] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab91c680a2d4eadb1f863c31be8f9c1" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:07.452447 16924 raft_consensus.cc:1275] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Refusing update from remote peer 470119605b5d4d6e9fe5cd713df24b96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:07.453711 17050 consensus_queue.cc:1048] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:07.491816 16926 raft_consensus.cc:1275] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:07.493388 17013 consensus_queue.cc:1048] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1837d31dc08e4130a103d7e1566c32f2" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:07.494995 16731 raft_consensus.cc:1275] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Refusing update from remote peer bab91c680a2d4eadb1f863c31be8f9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:07.496516 17013 consensus_queue.cc:1048] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470119605b5d4d6e9fe5cd713df24b96" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:07.920812  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:07.950183  5410 tablet_server.cc:179] TabletServer@127.5.72.129:43739 shutting down...
W20260709 13:22:07.980871 16621 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 80 similar messages]
I20260709 13:22:07.983078  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:07.984102  5410 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:07.984182 16621 consensus_peers.cc:597] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:07.984985  5410 raft_consensus.cc:2243] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:07.985558  5410 raft_consensus.cc:2272] T 864d99ab49f4491480a400ecaf86679d P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:07.988651  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:07.989346  5410 raft_consensus.cc:2243] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:07.989796  5410 raft_consensus.cc:2272] T 4a5ed3262ca74c7db89c4ce746224d55 P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:07.992133  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:07.992756  5410 raft_consensus.cc:2243] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:07.993227  5410 raft_consensus.cc:2272] T a83226d19f4f410dbc6b18522f88650f P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:07.996146  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:07.996767  5410 raft_consensus.cc:2243] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:07.997330  5410 raft_consensus.cc:2272] T 534a772500454949950c26737dd55d1f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:07.999662  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:08.000208  5410 raft_consensus.cc:2243] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.000656  5410 raft_consensus.cc:2272] T 73ebda95e06549f78a6f2fa1ae77b78d P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.002916  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:08.003711  5410 raft_consensus.cc:2243] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.004269  5410 raft_consensus.cc:2272] T 2b9c76f486114f7c9bcb6d9c127049aa P 1837d31dc08e4130a103d7e1566c32f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.006917  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:08.007684  5410 raft_consensus.cc:2243] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.008205  5410 raft_consensus.cc:2272] T a4760b87abc447518a87c5bc3085d3a3 P 1837d31dc08e4130a103d7e1566c32f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.010510  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:08.011137  5410 raft_consensus.cc:2243] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.011535  5410 raft_consensus.cc:2272] T 28db6bee0b60430fafa2173b7e33d009 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.014550  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:08.015172  5410 raft_consensus.cc:2243] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.015707  5410 raft_consensus.cc:2272] T 47e30a67505e4ab8b4897b4dcd758b5f P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.017824  5410 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:08.018309  5410 raft_consensus.cc:2243] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.018788  5410 raft_consensus.cc:2272] T 2010eb67c6434e8b86e7cda6ce403680 P 1837d31dc08e4130a103d7e1566c32f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.052608  5410 tablet_server.cc:196] TabletServer@127.5.72.129:43739 shutdown complete.
I20260709 13:22:08.078786  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
I20260709 13:22:08.107756  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:08.108668  5410 tablet_replica.cc:333] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
W20260709 13:22:08.109709 16697 consensus_peers.cc:597] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 -> Peer bab91c680a2d4eadb1f863c31be8f9c1 (127.5.72.130:42469): Couldn't send request to peer bab91c680a2d4eadb1f863c31be8f9c1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:08.109872  5410 raft_consensus.cc:2243] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.110945  5410 raft_consensus.cc:2272] T 534a772500454949950c26737dd55d1f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.113336  5410 tablet_replica.cc:333] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.113907  5410 raft_consensus.cc:2243] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.114378  5410 raft_consensus.cc:2272] T 864d99ab49f4491480a400ecaf86679d P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.116591  5410 tablet_replica.cc:333] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.117151  5410 raft_consensus.cc:2243] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.117870  5410 raft_consensus.cc:2272] T 4a5ed3262ca74c7db89c4ce746224d55 P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.119877  5410 tablet_replica.cc:333] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.120433  5410 raft_consensus.cc:2243] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.121569  5410 raft_consensus.cc:2272] T 47e30a67505e4ab8b4897b4dcd758b5f P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.123764  5410 tablet_replica.cc:333] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.124297  5410 raft_consensus.cc:2243] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.124712  5410 raft_consensus.cc:2272] T 73ebda95e06549f78a6f2fa1ae77b78d P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.126842  5410 tablet_replica.cc:333] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.127427  5410 raft_consensus.cc:2243] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.127895  5410 raft_consensus.cc:2272] T a4760b87abc447518a87c5bc3085d3a3 P bab91c680a2d4eadb1f863c31be8f9c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.129930  5410 tablet_replica.cc:333] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.130535  5410 raft_consensus.cc:2243] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.131088  5410 raft_consensus.cc:2272] T 2b9c76f486114f7c9bcb6d9c127049aa P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.133488  5410 tablet_replica.cc:333] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.133986  5410 raft_consensus.cc:2243] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.134793  5410 raft_consensus.cc:2272] T 2010eb67c6434e8b86e7cda6ce403680 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.138729  5410 tablet_replica.cc:333] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
I20260709 13:22:08.139284  5410 raft_consensus.cc:2243] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.139744  5410 raft_consensus.cc:2272] T a83226d19f4f410dbc6b18522f88650f P bab91c680a2d4eadb1f863c31be8f9c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.143435  5410 tablet_replica.cc:333] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1: stopping tablet replica
W20260709 13:22:08.143720 16696 consensus_peers.cc:597] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:08.144169  5410 raft_consensus.cc:2243] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.145097  5410 raft_consensus.cc:2272] T 28db6bee0b60430fafa2173b7e33d009 P bab91c680a2d4eadb1f863c31be8f9c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.183609  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
W20260709 13:22:08.210356 16696 consensus_peers.cc:597] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 -> Peer 1837d31dc08e4130a103d7e1566c32f2 (127.5.72.129:43739): Couldn't send request to peer 1837d31dc08e4130a103d7e1566c32f2. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:43739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:08.211200  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
I20260709 13:22:08.238021  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:08.238771  5410 tablet_replica.cc:333] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.239586  5410 raft_consensus.cc:2243] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.240139  5410 raft_consensus.cc:2272] T 534a772500454949950c26737dd55d1f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.242440  5410 tablet_replica.cc:333] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.243121  5410 raft_consensus.cc:2243] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.243902  5410 raft_consensus.cc:2272] T a4760b87abc447518a87c5bc3085d3a3 P 470119605b5d4d6e9fe5cd713df24b96 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.246017  5410 tablet_replica.cc:333] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.246544  5410 raft_consensus.cc:2243] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.247372  5410 raft_consensus.cc:2272] T 2b9c76f486114f7c9bcb6d9c127049aa P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.249507  5410 tablet_replica.cc:333] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.250033  5410 raft_consensus.cc:2243] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.250797  5410 raft_consensus.cc:2272] T a83226d19f4f410dbc6b18522f88650f P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.254086  5410 tablet_replica.cc:333] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.254793  5410 raft_consensus.cc:2243] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.255875  5410 raft_consensus.cc:2272] T 73ebda95e06549f78a6f2fa1ae77b78d P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.258878  5410 tablet_replica.cc:333] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.259661  5410 raft_consensus.cc:2243] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.260658  5410 raft_consensus.cc:2272] T 864d99ab49f4491480a400ecaf86679d P 470119605b5d4d6e9fe5cd713df24b96 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.262821  5410 tablet_replica.cc:333] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.263743  5410 raft_consensus.cc:2243] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.264120  5410 raft_consensus.cc:2272] T 4a5ed3262ca74c7db89c4ce746224d55 P 470119605b5d4d6e9fe5cd713df24b96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.266180  5410 tablet_replica.cc:333] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.266705  5410 raft_consensus.cc:2243] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.267112  5410 raft_consensus.cc:2272] T 28db6bee0b60430fafa2173b7e33d009 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.270020  5410 tablet_replica.cc:333] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.270519  5410 raft_consensus.cc:2243] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.271054  5410 raft_consensus.cc:2272] T 47e30a67505e4ab8b4897b4dcd758b5f P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.273025  5410 tablet_replica.cc:333] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96: stopping tablet replica
I20260709 13:22:08.273434  5410 raft_consensus.cc:2243] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:08.273782  5410 raft_consensus.cc:2272] T 2010eb67c6434e8b86e7cda6ce403680 P 470119605b5d4d6e9fe5cd713df24b96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.310760  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
I20260709 13:22:08.336544  5410 master.cc:562] Master@127.5.72.190:45721 shutting down...
I20260709 13:22:08.363274  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:08.363901  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:08.364269  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6a0671b84f9c4e3191c4b6e430df9a1f: stopping tablet replica
I20260709 13:22:08.384889  5410 master.cc:584] Master@127.5.72.190:45721 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13127 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:22:08.430953  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:41863
I20260709 13:22:08.432322  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:08.438524 17061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:08.439081 17063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:08.439169 17060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:08.440124  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:08.441062  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:08.441239  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:08.441354  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603328441342 us; error 0 us; skew 500 ppm
I20260709 13:22:08.441795  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:08.444352  5410 webserver.cc:533] Webserver started at http://127.5.72.190:41365/ using document root <none> and password file <none>
I20260709 13:22:08.444814  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:08.444968  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:08.445176  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:08.446224  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.450536  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:22:08.453991 17068 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.454795  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:08.455091  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.455413  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:08.489266  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:08.491861  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:08.531754  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:41863
I20260709 13:22:08.531845 17119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:41863 every 8 connection(s)
I20260709 13:22:08.535741 17120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:08.547200 17120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f: Bootstrap starting.
I20260709 13:22:08.552115 17120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:08.556561 17120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f: No bootstrap required, opened a new log
I20260709 13:22:08.559032 17120 raft_consensus.cc:359] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" member_type: VOTER }
I20260709 13:22:08.559437 17120 raft_consensus.cc:385] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:08.559674 17120 raft_consensus.cc:740] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61fe1f87e5da4bfc978ea55c1e2ba67f, State: Initialized, Role: FOLLOWER
I20260709 13:22:08.560256 17120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" member_type: VOTER }
I20260709 13:22:08.560708 17120 raft_consensus.cc:399] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:08.560930 17120 raft_consensus.cc:493] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:08.561221 17120 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:08.566169 17120 raft_consensus.cc:515] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" member_type: VOTER }
I20260709 13:22:08.566764 17120 leader_election.cc:304] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61fe1f87e5da4bfc978ea55c1e2ba67f; no voters: 
I20260709 13:22:08.567852 17120 leader_election.cc:290] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:08.568295 17123 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:08.569681 17123 raft_consensus.cc:697] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 1 LEADER]: Becoming Leader. State: Replica: 61fe1f87e5da4bfc978ea55c1e2ba67f, State: Running, Role: LEADER
I20260709 13:22:08.570389 17123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" member_type: VOTER }
I20260709 13:22:08.570907 17120 sys_catalog.cc:565] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:08.575242 17124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" member_type: VOTER } }
I20260709 13:22:08.575894 17124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:08.576947 17125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61fe1f87e5da4bfc978ea55c1e2ba67f. Latest consensus state: current_term: 1 leader_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe1f87e5da4bfc978ea55c1e2ba67f" member_type: VOTER } }
I20260709 13:22:08.577435 17125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:08.583997 17131 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:08.590826 17131 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:08.591939  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:08.600198 17131 catalog_manager.cc:1383] Generated new cluster ID: 7ae94e2cd3524b018055fac325d84d2d
I20260709 13:22:08.600521 17131 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:08.620724 17131 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:08.622040 17131 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:08.635841 17131 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f: Generated new TSK 0
I20260709 13:22:08.636507 17131 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:08.658586  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:08.665130 17141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:08.666373 17142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:08.667851 17144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:08.668166  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:08.669241  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:08.669459  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:08.669593  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603328669581 us; error 0 us; skew 500 ppm
I20260709 13:22:08.670114  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:08.672710  5410 webserver.cc:533] Webserver started at http://127.5.72.129:40281/ using document root <none> and password file <none>
I20260709 13:22:08.673158  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:08.673327  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:08.673540  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:08.674738  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.680243  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:22:08.683943 17149 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.684722  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:08.685043  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.685346  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:08.698123  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:08.699486  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:08.700968  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:08.703374  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:08.703596  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.703857  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:08.704067  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.746084  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:35091
I20260709 13:22:08.746156 17211 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:35091 every 8 connection(s)
I20260709 13:22:08.751607  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:08.758668 17216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:08.760260 17217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:08.763237  5410 server_base.cc:1061] running on GCE node
W20260709 13:22:08.763263 17219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:08.764377 17212 heartbeater.cc:344] Connected to a master server at 127.5.72.190:41863
I20260709 13:22:08.764828 17212 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:08.765185  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:08.765443  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:08.765632  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603328765595 us; error 0 us; skew 500 ppm
I20260709 13:22:08.765715 17212 heartbeater.cc:507] Master 127.5.72.190:41863 requested a full tablet report, sending...
I20260709 13:22:08.766289  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:08.768211 17085 ts_manager.cc:194] Registered new tserver with Master: 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129:35091)
I20260709 13:22:08.769258  5410 webserver.cc:533] Webserver started at http://127.5.72.130:35977/ using document root <none> and password file <none>
I20260709 13:22:08.769949  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:08.770195  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:08.770491  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:08.770916 17085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42716
I20260709 13:22:08.772156  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "f47ea03cdec9482db614e98dc1930930"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.776980  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:08.780987 17224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.781912  5410 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:22:08.782332  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "f47ea03cdec9482db614e98dc1930930"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.782691  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:08.808506  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:08.809931  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:08.811668  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:08.814388  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:08.814597  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.814958  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:08.815168  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.860361  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:41021
I20260709 13:22:08.860457 17286 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:41021 every 8 connection(s)
I20260709 13:22:08.865694  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:08.873447 17290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:08.873701 17291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:08.878521 17293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:08.879612 17287 heartbeater.cc:344] Connected to a master server at 127.5.72.190:41863
I20260709 13:22:08.880468 17287 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:08.878644  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:08.882234 17287 heartbeater.cc:507] Master 127.5.72.190:41863 requested a full tablet report, sending...
I20260709 13:22:08.884167  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:08.884495  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:08.884738  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603328884718 us; error 0 us; skew 500 ppm
I20260709 13:22:08.885026 17085 ts_manager.cc:194] Registered new tserver with Master: f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021)
I20260709 13:22:08.885653  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:08.887259 17085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42724
I20260709 13:22:08.889170  5410 webserver.cc:533] Webserver started at http://127.5.72.131:34571/ using document root <none> and password file <none>
I20260709 13:22:08.889798  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:08.890053  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:08.890355  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:08.891765  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.896746  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:08.900585 17298 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.901460  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:22:08.901801  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
format_stamp: "Formatted at 2026-07-09 13:22:08 on dist-test-slave-n5kz"
I20260709 13:22:08.902122  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:08.918550  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:08.919811  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:08.921290  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:08.923520  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:08.923748  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.923993  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:08.924163  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:08.965847  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:40237
I20260709 13:22:08.965946 17360 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:40237 every 8 connection(s)
I20260709 13:22:08.979779 17361 heartbeater.cc:344] Connected to a master server at 127.5.72.190:41863
I20260709 13:22:08.980350 17361 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:08.981128 17361 heartbeater.cc:507] Master 127.5.72.190:41863 requested a full tablet report, sending...
I20260709 13:22:08.983073 17085 ts_manager.cc:194] Registered new tserver with Master: a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:08.983866  5410 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014760668s
I20260709 13:22:08.984798 17085 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42730
I20260709 13:22:09.010171 17085 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42744:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:09.012331 17085 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:09.059811 17252 tablet_service.cc:1511] Processing CreateTablet for tablet a05e50b2200d49cf872447367c25c31a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=7b30a0064e66449ba9d33983ff4bd21e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:09.061290 17251 tablet_service.cc:1511] Processing CreateTablet for tablet ed1a6f3249494de388db6ec618c972d6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=7b30a0064e66449ba9d33983ff4bd21e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:09.061354 17252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a05e50b2200d49cf872447367c25c31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.063098 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1a6f3249494de388db6ec618c972d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.078410 17326 tablet_service.cc:1511] Processing CreateTablet for tablet a05e50b2200d49cf872447367c25c31a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=7b30a0064e66449ba9d33983ff4bd21e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:09.079942 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a05e50b2200d49cf872447367c25c31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.078529 17325 tablet_service.cc:1511] Processing CreateTablet for tablet ed1a6f3249494de388db6ec618c972d6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=7b30a0064e66449ba9d33983ff4bd21e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:09.081908 17325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1a6f3249494de388db6ec618c972d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.089094 17177 tablet_service.cc:1511] Processing CreateTablet for tablet a05e50b2200d49cf872447367c25c31a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=7b30a0064e66449ba9d33983ff4bd21e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:09.092288 17176 tablet_service.cc:1511] Processing CreateTablet for tablet ed1a6f3249494de388db6ec618c972d6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=7b30a0064e66449ba9d33983ff4bd21e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:09.094195 17176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1a6f3249494de388db6ec618c972d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.095922 17177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a05e50b2200d49cf872447367c25c31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.121599 17376 tablet_bootstrap.cc:492] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930: Bootstrap starting.
I20260709 13:22:09.125097 17377 tablet_bootstrap.cc:492] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6: Bootstrap starting.
I20260709 13:22:09.132905 17377 tablet_bootstrap.cc:654] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.132997 17376 tablet_bootstrap.cc:654] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.133213 17378 tablet_bootstrap.cc:492] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5: Bootstrap starting.
I20260709 13:22:09.137967 17377 tablet_bootstrap.cc:492] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6: No bootstrap required, opened a new log
I20260709 13:22:09.138433 17377 ts_tablet_manager.cc:1403] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:22:09.139199 17376 tablet_bootstrap.cc:492] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930: No bootstrap required, opened a new log
I20260709 13:22:09.139786 17376 ts_tablet_manager.cc:1403] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.009s
I20260709 13:22:09.141273 17378 tablet_bootstrap.cc:654] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.141132 17377 raft_consensus.cc:359] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.141871 17377 raft_consensus.cc:385] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.142148 17377 raft_consensus.cc:740] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b04be8669b4f1eb9872e373d0eb1d6, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.142863 17377 consensus_queue.cc:260] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.143127 17376 raft_consensus.cc:359] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.145174 17361 heartbeater.cc:499] Master 127.5.72.190:41863 was elected leader, sending a full tablet report...
I20260709 13:22:09.145735 17377 ts_tablet_manager.cc:1434] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:09.146736 17377 tablet_bootstrap.cc:492] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6: Bootstrap starting.
I20260709 13:22:09.143970 17376 raft_consensus.cc:385] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.148834 17376 raft_consensus.cc:740] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47ea03cdec9482db614e98dc1930930, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.150952 17376 consensus_queue.cc:260] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.158067 17377 tablet_bootstrap.cc:654] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.166039 17376 ts_tablet_manager.cc:1434] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930: Time spent starting tablet: real 0.026s	user 0.014s	sys 0.003s
I20260709 13:22:09.171900 17287 heartbeater.cc:499] Master 127.5.72.190:41863 was elected leader, sending a full tablet report...
I20260709 13:22:09.176173 17377 tablet_bootstrap.cc:492] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6: No bootstrap required, opened a new log
I20260709 13:22:09.177680 17377 ts_tablet_manager.cc:1403] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.008s
I20260709 13:22:09.178350 17378 tablet_bootstrap.cc:492] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5: No bootstrap required, opened a new log
I20260709 13:22:09.179183 17378 ts_tablet_manager.cc:1403] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent bootstrapping tablet: real 0.046s	user 0.028s	sys 0.013s
I20260709 13:22:09.180845 17376 tablet_bootstrap.cc:492] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930: Bootstrap starting.
I20260709 13:22:09.182440 17377 raft_consensus.cc:359] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.183404 17377 raft_consensus.cc:385] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.183768 17377 raft_consensus.cc:740] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b04be8669b4f1eb9872e373d0eb1d6, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.184158 17378 raft_consensus.cc:359] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.185076 17378 raft_consensus.cc:385] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.184737 17377 consensus_queue.cc:260] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.185392 17378 raft_consensus.cc:740] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.186923 17378 consensus_queue.cc:260] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.187774 17377 ts_tablet_manager.cc:1434] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:22:09.190148 17376 tablet_bootstrap.cc:654] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.193928 17378 ts_tablet_manager.cc:1434] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.008s
I20260709 13:22:09.195402 17378 tablet_bootstrap.cc:492] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5: Bootstrap starting.
I20260709 13:22:09.193918 17212 heartbeater.cc:499] Master 127.5.72.190:41863 was elected leader, sending a full tablet report...
I20260709 13:22:09.196666 17376 tablet_bootstrap.cc:492] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930: No bootstrap required, opened a new log
I20260709 13:22:09.197238 17376 ts_tablet_manager.cc:1403] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.003s
I20260709 13:22:09.200331 17376 raft_consensus.cc:359] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.201145 17376 raft_consensus.cc:385] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.201472 17376 raft_consensus.cc:740] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47ea03cdec9482db614e98dc1930930, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.201658 17378 tablet_bootstrap.cc:654] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.202325 17376 consensus_queue.cc:260] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.207202 17376 ts_tablet_manager.cc:1434] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:09.211853 17378 tablet_bootstrap.cc:492] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5: No bootstrap required, opened a new log
I20260709 13:22:09.212438 17378 ts_tablet_manager.cc:1403] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:22:09.214885 17378 raft_consensus.cc:359] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.215444 17378 raft_consensus.cc:385] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.215685 17378 raft_consensus.cc:740] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.216315 17378 consensus_queue.cc:260] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.218181 17378 ts_tablet_manager.cc:1434] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:09.233692 17385 raft_consensus.cc:493] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.234300 17385 raft_consensus.cc:515] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.236513 17385 leader_election.cc:290] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:09.246371 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05e50b2200d49cf872447367c25c31a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930" is_pre_election: true
I20260709 13:22:09.247198 17262 raft_consensus.cc:2468] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 0.
I20260709 13:22:09.248152 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05e50b2200d49cf872447367c25c31a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" is_pre_election: true
I20260709 13:22:09.248418 17152 leader_election.cc:304] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:09.248894 17336 raft_consensus.cc:2468] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 0.
I20260709 13:22:09.249238 17385 raft_consensus.cc:2804] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:09.249663 17385 raft_consensus.cc:493] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:09.250029 17385 raft_consensus.cc:3060] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.255280 17385 raft_consensus.cc:515] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.256778 17385 leader_election.cc:290] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 election: Requested vote from peers f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:09.257656 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05e50b2200d49cf872447367c25c31a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930"
I20260709 13:22:09.257782 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a05e50b2200d49cf872447367c25c31a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
I20260709 13:22:09.258234 17262 raft_consensus.cc:3060] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.258363 17336 raft_consensus.cc:3060] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.263551 17262 raft_consensus.cc:2468] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 1.
I20260709 13:22:09.263550 17336 raft_consensus.cc:2468] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 1.
I20260709 13:22:09.264746 17152 leader_election.cc:304] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:09.265424 17385 raft_consensus.cc:2804] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:09.266476 17385 raft_consensus.cc:697] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Becoming Leader. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Running, Role: LEADER
I20260709 13:22:09.267213 17385 consensus_queue.cc:237] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.274915 17084 catalog_manager.cc:5697] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.352358 17385 raft_consensus.cc:493] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.353032 17385 raft_consensus.cc:515] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.354874 17385 leader_election.cc:290] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:09.355851 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930" is_pre_election: true
I20260709 13:22:09.356096 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" is_pre_election: true
I20260709 13:22:09.356524 17262 raft_consensus.cc:2468] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 0.
I20260709 13:22:09.356730 17336 raft_consensus.cc:2468] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 0.
I20260709 13:22:09.357681 17152 leader_election.cc:304] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:09.358409 17385 raft_consensus.cc:2804] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:09.358822 17385 raft_consensus.cc:493] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:09.359076 17385 raft_consensus.cc:3060] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.364219 17385 raft_consensus.cc:515] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.366225 17385 leader_election.cc:290] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 election: Requested vote from peers f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:09.367230 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
I20260709 13:22:09.367203 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930"
I20260709 13:22:09.368057 17336 raft_consensus.cc:3060] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.368095 17262 raft_consensus.cc:3060] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.375615 17262 raft_consensus.cc:2468] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 1.
I20260709 13:22:09.375615 17336 raft_consensus.cc:2468] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 1.
I20260709 13:22:09.377082 17152 leader_election.cc:304] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:09.377909 17385 raft_consensus.cc:2804] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:09.378364 17385 raft_consensus.cc:697] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Becoming Leader. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Running, Role: LEADER
I20260709 13:22:09.379168 17385 consensus_queue.cc:237] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.386080 17084 catalog_manager.cc:5697] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.480342 17084 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42758:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:09.482492 17084 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:09.506441 17177 tablet_service.cc:1511] Processing CreateTablet for tablet 6e79a6c30c8a426b9aeea398ba80e45c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=656f29790a2c40dfbf7948d0f33ee3f3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:09.507794 17177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e79a6c30c8a426b9aeea398ba80e45c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.507872 17252 tablet_service.cc:1511] Processing CreateTablet for tablet 6e79a6c30c8a426b9aeea398ba80e45c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=656f29790a2c40dfbf7948d0f33ee3f3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:09.509150 17252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e79a6c30c8a426b9aeea398ba80e45c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.510958 17326 tablet_service.cc:1511] Processing CreateTablet for tablet 6e79a6c30c8a426b9aeea398ba80e45c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=656f29790a2c40dfbf7948d0f33ee3f3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:09.512303 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e79a6c30c8a426b9aeea398ba80e45c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.512413 17176 tablet_service.cc:1511] Processing CreateTablet for tablet ec5f9465a4bc48209e1ab33b3384266a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=656f29790a2c40dfbf7948d0f33ee3f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:09.513684 17176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5f9465a4bc48209e1ab33b3384266a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.515543 17325 tablet_service.cc:1511] Processing CreateTablet for tablet ec5f9465a4bc48209e1ab33b3384266a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=656f29790a2c40dfbf7948d0f33ee3f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:09.516855 17251 tablet_service.cc:1511] Processing CreateTablet for tablet ec5f9465a4bc48209e1ab33b3384266a (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=656f29790a2c40dfbf7948d0f33ee3f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:09.516848 17325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5f9465a4bc48209e1ab33b3384266a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.518110 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec5f9465a4bc48209e1ab33b3384266a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:09.527171 17376 tablet_bootstrap.cc:492] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930: Bootstrap starting.
I20260709 13:22:09.531816 17377 tablet_bootstrap.cc:492] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6: Bootstrap starting.
I20260709 13:22:09.534791 17376 tablet_bootstrap.cc:654] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.538645 17377 tablet_bootstrap.cc:654] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.548506 17378 tablet_bootstrap.cc:492] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5: Bootstrap starting.
I20260709 13:22:09.552279 17377 tablet_bootstrap.cc:492] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6: No bootstrap required, opened a new log
I20260709 13:22:09.553063 17377 ts_tablet_manager.cc:1403] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 13:22:09.555968 17378 tablet_bootstrap.cc:654] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.557442 17377 raft_consensus.cc:359] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.558494 17377 raft_consensus.cc:385] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.558832 17377 raft_consensus.cc:740] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b04be8669b4f1eb9872e373d0eb1d6, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.559911 17377 consensus_queue.cc:260] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.561421 17376 tablet_bootstrap.cc:492] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930: No bootstrap required, opened a new log
I20260709 13:22:09.562151 17376 ts_tablet_manager.cc:1403] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.013s
I20260709 13:22:09.565888 17377 ts_tablet_manager.cc:1434] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:22:09.567137 17377 tablet_bootstrap.cc:492] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6: Bootstrap starting.
I20260709 13:22:09.570509 17378 tablet_bootstrap.cc:492] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5: No bootstrap required, opened a new log
I20260709 13:22:09.570441 17376 raft_consensus.cc:359] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.571290 17376 raft_consensus.cc:385] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.571309 17378 ts_tablet_manager.cc:1403] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:22:09.571641 17376 raft_consensus.cc:740] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47ea03cdec9482db614e98dc1930930, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.572561 17376 consensus_queue.cc:260] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.574409 17377 tablet_bootstrap.cc:654] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.575059 17378 raft_consensus.cc:359] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.575119 17376 ts_tablet_manager.cc:1434] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:22:09.575954 17378 raft_consensus.cc:385] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.576412 17378 raft_consensus.cc:740] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.576931 17376 tablet_bootstrap.cc:492] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: Bootstrap starting.
I20260709 13:22:09.577275 17378 consensus_queue.cc:260] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.580086 17378 ts_tablet_manager.cc:1434] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:09.581189 17378 tablet_bootstrap.cc:492] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: Bootstrap starting.
W20260709 13:22:09.581903 17133 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:09.588127 17378 tablet_bootstrap.cc:654] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.590338 17376 tablet_bootstrap.cc:654] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:09.597126 17377 tablet_bootstrap.cc:492] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6: No bootstrap required, opened a new log
I20260709 13:22:09.597820 17377 ts_tablet_manager.cc:1403] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.017s
I20260709 13:22:09.600441 17376 tablet_bootstrap.cc:492] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: No bootstrap required, opened a new log
I20260709 13:22:09.601015 17376 ts_tablet_manager.cc:1403] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:22:09.601727 17377 raft_consensus.cc:359] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.602792 17377 raft_consensus.cc:385] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.603209 17377 raft_consensus.cc:740] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b04be8669b4f1eb9872e373d0eb1d6, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.604094 17376 raft_consensus.cc:359] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.604908 17376 raft_consensus.cc:385] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.604214 17377 consensus_queue.cc:260] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.605252 17376 raft_consensus.cc:740] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47ea03cdec9482db614e98dc1930930, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.605419 17385 raft_consensus.cc:493] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.605943 17385 raft_consensus.cc:515] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.606109 17376 consensus_queue.cc:260] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.607802 17377 ts_tablet_manager.cc:1434] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:09.608152 17385 leader_election.cc:290] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:09.608470 17376 ts_tablet_manager.cc:1434] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:09.610049 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930" is_pre_election: true
I20260709 13:22:09.610737 17262 raft_consensus.cc:2468] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 0.
I20260709 13:22:09.611855 17378 tablet_bootstrap.cc:492] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: No bootstrap required, opened a new log
I20260709 13:22:09.612205 17152 leader_election.cc:304] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:09.613399 17385 raft_consensus.cc:2804] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:09.610674 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" is_pre_election: true
I20260709 13:22:09.613787 17385 raft_consensus.cc:493] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:09.614082 17336 raft_consensus.cc:2468] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 0.
I20260709 13:22:09.614180 17385 raft_consensus.cc:3060] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.612412 17378 ts_tablet_manager.cc:1403] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.004s
I20260709 13:22:09.621351 17378 raft_consensus.cc:359] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.621937 17378 raft_consensus.cc:385] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:09.621690 17385 raft_consensus.cc:515] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.622234 17378 raft_consensus.cc:740] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Initialized, Role: FOLLOWER
I20260709 13:22:09.622982 17378 consensus_queue.cc:260] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.623692 17385 leader_election.cc:290] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 election: Requested vote from peers f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:09.624430 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930"
I20260709 13:22:09.624822 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
I20260709 13:22:09.625686 17262 raft_consensus.cc:3060] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.624938 17378 ts_tablet_manager.cc:1434] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:09.626140 17336 raft_consensus.cc:3060] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.632529 17336 raft_consensus.cc:2468] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 1.
I20260709 13:22:09.632965 17262 raft_consensus.cc:2468] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 1.
I20260709 13:22:09.633813 17153 leader_election.cc:304] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, a9b04be8669b4f1eb9872e373d0eb1d6; no voters: 
I20260709 13:22:09.634843 17385 raft_consensus.cc:2804] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:09.635358 17385 raft_consensus.cc:697] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Becoming Leader. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Running, Role: LEADER
I20260709 13:22:09.636247 17385 consensus_queue.cc:237] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.644471 17085 catalog_manager.cc:5697] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.753867 17385 consensus_queue.cc:1048] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:09.757568 17385 consensus_queue.cc:1048] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:09.808300 17381 raft_consensus.cc:493] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.808817 17381 raft_consensus.cc:515] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.810952 17381 leader_election.cc:290] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129:35091), f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021)
I20260709 13:22:09.832144 17187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c" candidate_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" is_pre_election: true
I20260709 13:22:09.832158 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c" candidate_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930" is_pre_election: true
I20260709 13:22:09.833186 17187 raft_consensus.cc:2468] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b04be8669b4f1eb9872e373d0eb1d6 in term 0.
I20260709 13:22:09.833186 17262 raft_consensus.cc:2468] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b04be8669b4f1eb9872e373d0eb1d6 in term 0.
I20260709 13:22:09.834589 17299 leader_election.cc:304] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, a9b04be8669b4f1eb9872e373d0eb1d6; no voters: 
I20260709 13:22:09.835358 17381 raft_consensus.cc:2804] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:09.835706 17381 raft_consensus.cc:493] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:09.835978 17381 raft_consensus.cc:3060] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.841336 17381 raft_consensus.cc:515] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.843006 17381 leader_election.cc:290] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [CANDIDATE]: Term 1 election: Requested vote from peers 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129:35091), f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021)
I20260709 13:22:09.844033 17187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c" candidate_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
I20260709 13:22:09.844080 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c" candidate_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f47ea03cdec9482db614e98dc1930930"
I20260709 13:22:09.844667 17187 raft_consensus.cc:3060] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.844769 17262 raft_consensus.cc:3060] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:09.850042 17262 raft_consensus.cc:2468] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b04be8669b4f1eb9872e373d0eb1d6 in term 1.
I20260709 13:22:09.850147 17187 raft_consensus.cc:2468] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b04be8669b4f1eb9872e373d0eb1d6 in term 1.
I20260709 13:22:09.851387 17299 leader_election.cc:304] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, a9b04be8669b4f1eb9872e373d0eb1d6; no voters: 
I20260709 13:22:09.852137 17381 raft_consensus.cc:2804] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:09.853394 17381 raft_consensus.cc:697] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 LEADER]: Becoming Leader. State: Replica: a9b04be8669b4f1eb9872e373d0eb1d6, State: Running, Role: LEADER
I20260709 13:22:09.854208 17381 consensus_queue.cc:237] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:09.861649 17083 catalog_manager.cc:5697] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131). New cstate: current_term: 1 leader_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:09.881125 17385 consensus_queue.cc:1048] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:09.892170 17389 consensus_queue.cc:1048] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:09.942955 17187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6"
dest_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
mode: GRACEFUL
new_leader_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
 from {username='slave'} at 127.0.0.1:55644
I20260709 13:22:09.943645 17187 raft_consensus.cc:606] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Received request to transfer leadership to TS a9b04be8669b4f1eb9872e373d0eb1d6
I20260709 13:22:09.958699 17336 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c"
dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
mode: GRACEFUL
new_leader_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
 from {username='slave'} at 127.0.0.1:34662
I20260709 13:22:09.959275 17336 raft_consensus.cc:606] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 LEADER]: Received request to transfer leadership to TS 8aaeb9bbd9584ae09bd676685c2fafb5
I20260709 13:22:10.218237 17389 consensus_queue.cc:1048] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:10.231026 17409 consensus_queue.cc:1048] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:10.421763 17381 consensus_queue.cc:1048] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:10.434506 17425 consensus_queue.cc:1048] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:10.442948 17381 raft_consensus.cc:993] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6: : Instructing follower 8aaeb9bbd9584ae09bd676685c2fafb5 to start an election
I20260709 13:22:10.443410 17425 raft_consensus.cc:1081] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 LEADER]: Signalling peer 8aaeb9bbd9584ae09bd676685c2fafb5 to start an election
I20260709 13:22:10.444893 17187 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c"
dest_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
 from {username='slave'} at 127.0.0.1:55630
I20260709 13:22:10.445494 17187 raft_consensus.cc:493] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.445868 17187 raft_consensus.cc:3060] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.450971 17187 raft_consensus.cc:515] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.453336 17187 leader_election.cc:290] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 2 election: Requested vote from peers f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:10.453667 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f47ea03cdec9482db614e98dc1930930"
I20260709 13:22:10.453941 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e79a6c30c8a426b9aeea398ba80e45c" candidate_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
I20260709 13:22:10.454352 17262 raft_consensus.cc:3060] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.454491 17336 raft_consensus.cc:3055] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:10.454916 17336 raft_consensus.cc:740] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9b04be8669b4f1eb9872e373d0eb1d6, State: Running, Role: LEADER
I20260709 13:22:10.455736 17336 consensus_queue.cc:260] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.457098 17336 raft_consensus.cc:3060] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.462219 17262 raft_consensus.cc:2468] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 2.
I20260709 13:22:10.463618 17152 leader_election.cc:304] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8aaeb9bbd9584ae09bd676685c2fafb5, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:10.464322 17409 raft_consensus.cc:2804] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:10.464265 17336 raft_consensus.cc:2468] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8aaeb9bbd9584ae09bd676685c2fafb5 in term 2.
I20260709 13:22:10.465080 17409 raft_consensus.cc:697] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 LEADER]: Becoming Leader. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Running, Role: LEADER
I20260709 13:22:10.465811 17409 consensus_queue.cc:237] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.471663 17083 catalog_manager.cc:5697] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 reported cstate change: term changed from 1 to 2, leader changed from a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131) to 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.486739 17389 raft_consensus.cc:993] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5: : Instructing follower a9b04be8669b4f1eb9872e373d0eb1d6 to start an election
I20260709 13:22:10.487246 17409 raft_consensus.cc:1081] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Signalling peer a9b04be8669b4f1eb9872e373d0eb1d6 to start an election
I20260709 13:22:10.488929 17336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6"
dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
 from {username='slave'} at 127.0.0.1:34648
I20260709 13:22:10.489498 17336 raft_consensus.cc:493] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.489852 17336 raft_consensus.cc:3060] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.494997 17336 raft_consensus.cc:515] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.496510 17336 leader_election.cc:290] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [CANDIDATE]: Term 2 election: Requested vote from peers 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129:35091), f47ea03cdec9482db614e98dc1930930 (127.5.72.130:41021)
I20260709 13:22:10.497560 17187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6" candidate_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
I20260709 13:22:10.497659 17262 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1a6f3249494de388db6ec618c972d6" candidate_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f47ea03cdec9482db614e98dc1930930"
I20260709 13:22:10.498248 17187 raft_consensus.cc:3055] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:10.498286 17262 raft_consensus.cc:3060] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.498580 17187 raft_consensus.cc:740] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Running, Role: LEADER
I20260709 13:22:10.499372 17187 consensus_queue.cc:260] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.500337 17187 raft_consensus.cc:3060] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.504235 17262 raft_consensus.cc:2468] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b04be8669b4f1eb9872e373d0eb1d6 in term 2.
I20260709 13:22:10.505395 17301 leader_election.cc:304] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b04be8669b4f1eb9872e373d0eb1d6, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:10.505610 17187 raft_consensus.cc:2468] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b04be8669b4f1eb9872e373d0eb1d6 in term 2.
I20260709 13:22:10.506326 17425 raft_consensus.cc:2804] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:10.506879 17425 raft_consensus.cc:697] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 LEADER]: Becoming Leader. State: Replica: a9b04be8669b4f1eb9872e373d0eb1d6, State: Running, Role: LEADER
I20260709 13:22:10.507670 17425 consensus_queue.cc:237] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.514035 17083 catalog_manager.cc:5697] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 reported cstate change: term changed from 1 to 2, leader changed from 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129) to a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131). New cstate: current_term: 2 leader_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:10.725205  5410 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:22:10.726847  5410 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:22:10.727391  5410 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:22:10.739873 17187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a"
dest_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
mode: GRACEFUL
new_leader_uuid: "f47ea03cdec9482db614e98dc1930930"
 from {username='slave'} at 127.0.0.1:58362
I20260709 13:22:10.740383 17187 raft_consensus.cc:606] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Received request to transfer leadership to TS f47ea03cdec9482db614e98dc1930930
I20260709 13:22:10.741420  5410 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:22:10.849318 17389 raft_consensus.cc:993] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5: : Instructing follower f47ea03cdec9482db614e98dc1930930 to start an election
I20260709 13:22:10.849823 17389 raft_consensus.cc:1081] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Signalling peer f47ea03cdec9482db614e98dc1930930 to start an election
I20260709 13:22:10.851819 17262 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a"
dest_uuid: "f47ea03cdec9482db614e98dc1930930"
 from {username='slave'} at 127.0.0.1:42432
I20260709 13:22:10.852419 17262 raft_consensus.cc:493] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.852759 17262 raft_consensus.cc:3060] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.858289 17262 raft_consensus.cc:515] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.860535 17262 leader_election.cc:290] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [CANDIDATE]: Term 2 election: Requested vote from peers 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129:35091), a9b04be8669b4f1eb9872e373d0eb1d6 (127.5.72.131:40237)
I20260709 13:22:10.872046 17187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a" candidate_uuid: "f47ea03cdec9482db614e98dc1930930" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5"
I20260709 13:22:10.872726 17187 raft_consensus.cc:3055] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:10.873111 17187 raft_consensus.cc:740] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8aaeb9bbd9584ae09bd676685c2fafb5, State: Running, Role: LEADER
I20260709 13:22:10.873919 17187 consensus_queue.cc:260] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.874593 17336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec5f9465a4bc48209e1ab33b3384266a" candidate_uuid: "f47ea03cdec9482db614e98dc1930930" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6"
I20260709 13:22:10.875249 17187 raft_consensus.cc:3060] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.875276 17336 raft_consensus.cc:3060] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.881259 17336 raft_consensus.cc:2468] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f47ea03cdec9482db614e98dc1930930 in term 2.
I20260709 13:22:10.881400 17187 raft_consensus.cc:2468] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f47ea03cdec9482db614e98dc1930930 in term 2.
I20260709 13:22:10.882472 17228 leader_election.cc:304] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b04be8669b4f1eb9872e373d0eb1d6, f47ea03cdec9482db614e98dc1930930; no voters: 
I20260709 13:22:10.883272 17383 raft_consensus.cc:2804] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:10.884374 17383 raft_consensus.cc:697] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 2 LEADER]: Becoming Leader. State: Replica: f47ea03cdec9482db614e98dc1930930, State: Running, Role: LEADER
I20260709 13:22:10.885181 17383 consensus_queue.cc:237] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } }
I20260709 13:22:10.891880 17083 catalog_manager.cc:5697] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 reported cstate change: term changed from 1 to 2, leader changed from 8aaeb9bbd9584ae09bd676685c2fafb5 (127.5.72.129) to f47ea03cdec9482db614e98dc1930930 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "f47ea03cdec9482db614e98dc1930930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9b04be8669b4f1eb9872e373d0eb1d6" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 40237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.941152 17187 raft_consensus.cc:1275] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Refusing update from remote peer a9b04be8669b4f1eb9872e373d0eb1d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.942370 17425 consensus_queue.cc:1048] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aaeb9bbd9584ae09bd676685c2fafb5" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:10.954082  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
I20260709 13:22:10.958597 17262 raft_consensus.cc:1275] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Refusing update from remote peer a9b04be8669b4f1eb9872e373d0eb1d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.960330 17425 consensus_queue.cc:1048] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f47ea03cdec9482db614e98dc1930930" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:10.983516  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:10.984179  5410 tablet_replica.cc:333] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: stopping tablet replica
I20260709 13:22:10.984843  5410 raft_consensus.cc:2243] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:10.985632  5410 pending_rounds.cc:70] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: Trying to abort 1 pending ops.
I20260709 13:22:10.985862  5410 pending_rounds.cc:77] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305639241595203584 op_type: NO_OP noop_request { }
I20260709 13:22:10.986192  5410 raft_consensus.cc:2889] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:10.986500  5410 raft_consensus.cc:2272] T 6e79a6c30c8a426b9aeea398ba80e45c P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:10.988970  5410 tablet_replica.cc:333] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5: stopping tablet replica
I20260709 13:22:10.989562  5410 raft_consensus.cc:2243] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:10.989966  5410 raft_consensus.cc:2272] T ec5f9465a4bc48209e1ab33b3384266a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:10.992399  5410 tablet_replica.cc:333] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5: stopping tablet replica
I20260709 13:22:10.992924  5410 raft_consensus.cc:2243] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:10.993347  5410 raft_consensus.cc:2272] T ed1a6f3249494de388db6ec618c972d6 P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:10.995584  5410 tablet_replica.cc:333] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5: stopping tablet replica
I20260709 13:22:10.996078  5410 raft_consensus.cc:2243] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:10.996810  5410 raft_consensus.cc:2272] T a05e50b2200d49cf872447367c25c31a P 8aaeb9bbd9584ae09bd676685c2fafb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.020296  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
I20260709 13:22:11.032294  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
I20260709 13:22:11.058463  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:11.060001  5410 tablet_replica.cc:333] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: stopping tablet replica
I20260709 13:22:11.061372  5410 raft_consensus.cc:2243] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:11.063060  5410 pending_rounds.cc:70] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: Trying to abort 1 pending ops.
I20260709 13:22:11.063498  5410 pending_rounds.cc:77] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305639243314688000 op_type: NO_OP noop_request { }
I20260709 13:22:11.063944  5410 raft_consensus.cc:2889] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:11.064441  5410 raft_consensus.cc:2272] T ec5f9465a4bc48209e1ab33b3384266a P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.067337  5410 tablet_replica.cc:333] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930: stopping tablet replica
I20260709 13:22:11.067902  5410 raft_consensus.cc:2243] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:11.068332  5410 raft_consensus.cc:2272] T 6e79a6c30c8a426b9aeea398ba80e45c P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.070737  5410 tablet_replica.cc:333] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930: stopping tablet replica
I20260709 13:22:11.071269  5410 raft_consensus.cc:2243] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:11.071702  5410 raft_consensus.cc:2272] T ed1a6f3249494de388db6ec618c972d6 P f47ea03cdec9482db614e98dc1930930 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.074100  5410 tablet_replica.cc:333] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930: stopping tablet replica
I20260709 13:22:11.074662  5410 raft_consensus.cc:2243] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:11.075093  5410 raft_consensus.cc:2272] T a05e50b2200d49cf872447367c25c31a P f47ea03cdec9482db614e98dc1930930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.098765  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
I20260709 13:22:11.112061  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
I20260709 13:22:11.128846  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:11.129535  5410 tablet_replica.cc:333] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6: stopping tablet replica
I20260709 13:22:11.130091  5410 raft_consensus.cc:2243] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:11.130546  5410 raft_consensus.cc:2272] T ec5f9465a4bc48209e1ab33b3384266a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.132925  5410 tablet_replica.cc:333] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6: stopping tablet replica
I20260709 13:22:11.133625  5410 raft_consensus.cc:2243] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:11.134079  5410 raft_consensus.cc:2272] T 6e79a6c30c8a426b9aeea398ba80e45c P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.136662  5410 tablet_replica.cc:333] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6: stopping tablet replica
I20260709 13:22:11.137188  5410 raft_consensus.cc:2243] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:11.138028  5410 raft_consensus.cc:2272] T ed1a6f3249494de388db6ec618c972d6 P a9b04be8669b4f1eb9872e373d0eb1d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.140457  5410 tablet_replica.cc:333] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6: stopping tablet replica
I20260709 13:22:11.140946  5410 raft_consensus.cc:2243] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:11.141366  5410 raft_consensus.cc:2272] T a05e50b2200d49cf872447367c25c31a P a9b04be8669b4f1eb9872e373d0eb1d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.164526  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
I20260709 13:22:11.182235  5410 master.cc:562] Master@127.5.72.190:41863 shutting down...
I20260709 13:22:11.197582  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:11.198086  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:11.198344  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 61fe1f87e5da4bfc978ea55c1e2ba67f: stopping tablet replica
I20260709 13:22:11.217393  5410 master.cc:584] Master@127.5.72.190:41863 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2814 ms)
[----------] 6 tests from LeaderRebalancerTest (385372 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:22:11.245452  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:40487
I20260709 13:22:11.246531  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:11.251874 17442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:11.251844 17443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:11.253394 17445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:11.254851  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:11.255805  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:11.255978  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:11.256167  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603331256147 us; error 0 us; skew 500 ppm
I20260709 13:22:11.256848  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:11.259568  5410 webserver.cc:533] Webserver started at http://127.5.72.190:46163/ using document root <none> and password file <none>
I20260709 13:22:11.260090  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:11.260298  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:11.260562  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:11.261615  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "3134d8af1387460683b6fd7457c8a5f6"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.266108  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:22:11.269486 17450 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.270331  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:22:11.270699  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "3134d8af1387460683b6fd7457c8a5f6"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.271037  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:11.280762  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:11.281777  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:11.317667  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:40487
I20260709 13:22:11.317777 17501 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:40487 every 8 connection(s)
I20260709 13:22:11.322775 17502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.333711 17502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6: Bootstrap starting.
I20260709 13:22:11.338277 17502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:11.342927 17502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6: No bootstrap required, opened a new log
I20260709 13:22:11.345381 17502 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3134d8af1387460683b6fd7457c8a5f6" member_type: VOTER }
I20260709 13:22:11.345734 17502 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:11.345909 17502 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3134d8af1387460683b6fd7457c8a5f6, State: Initialized, Role: FOLLOWER
I20260709 13:22:11.346493 17502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3134d8af1387460683b6fd7457c8a5f6" member_type: VOTER }
I20260709 13:22:11.346956 17502 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:11.347153 17502 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:11.347357 17502 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:11.351897 17502 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3134d8af1387460683b6fd7457c8a5f6" member_type: VOTER }
I20260709 13:22:11.352372 17502 leader_election.cc:304] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3134d8af1387460683b6fd7457c8a5f6; no voters: 
I20260709 13:22:11.353427 17502 leader_election.cc:290] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:11.353765 17505 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:11.355188 17505 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 1 LEADER]: Becoming Leader. State: Replica: 3134d8af1387460683b6fd7457c8a5f6, State: Running, Role: LEADER
I20260709 13:22:11.355778 17505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3134d8af1387460683b6fd7457c8a5f6" member_type: VOTER }
I20260709 13:22:11.356374 17502 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:11.358431 17507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3134d8af1387460683b6fd7457c8a5f6. Latest consensus state: current_term: 1 leader_uuid: "3134d8af1387460683b6fd7457c8a5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3134d8af1387460683b6fd7457c8a5f6" member_type: VOTER } }
I20260709 13:22:11.358444 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3134d8af1387460683b6fd7457c8a5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3134d8af1387460683b6fd7457c8a5f6" member_type: VOTER } }
I20260709 13:22:11.359227 17507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:11.359336 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:11.362221 17510 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:11.370184 17510 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:11.371169  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:11.379598 17510 catalog_manager.cc:1383] Generated new cluster ID: 58df7ef3cb6c4f5eaa0dc66dd188d63b
I20260709 13:22:11.379886 17510 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:11.398953 17510 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:11.400252 17510 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:11.417337 17510 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6: Generated new TSK 0
I20260709 13:22:11.417953 17510 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:11.437733  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:11.443709 17523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:11.445271 17524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:11.448504 17526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:11.448563  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:11.449692  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:11.449927  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:11.450157  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603331450137 us; error 0 us; skew 500 ppm
I20260709 13:22:11.450795  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:11.453159  5410 webserver.cc:533] Webserver started at http://127.5.72.129:40633/ using document root <none> and password file <none>
I20260709 13:22:11.453658  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:11.453853  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:11.454139  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:11.455312  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "b7fc8565fb3b4e63b85f8f408089c384"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.459833  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:11.463207 17531 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.463973  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:22:11.464279  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "b7fc8565fb3b4e63b85f8f408089c384"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.464560  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:11.497814  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:11.499063  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:11.500605  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:11.502923  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:11.503172  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.503419  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:11.503625  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.542898  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:34037
I20260709 13:22:11.543032 17593 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:34037 every 8 connection(s)
I20260709 13:22:11.547937  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:11.555755 17598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:11.556700 17599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:11.560087 17601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:11.560635  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:11.560900 17594 heartbeater.cc:344] Connected to a master server at 127.5.72.190:40487
I20260709 13:22:11.561371 17594 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:11.562131 17594 heartbeater.cc:507] Master 127.5.72.190:40487 requested a full tablet report, sending...
I20260709 13:22:11.562513  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:11.562856  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:11.563061  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603331563043 us; error 0 us; skew 500 ppm
I20260709 13:22:11.563746  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:11.564343 17467 ts_manager.cc:194] Registered new tserver with Master: b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037)
I20260709 13:22:11.566658  5410 webserver.cc:533] Webserver started at http://127.5.72.130:43227/ using document root <none> and password file <none>
I20260709 13:22:11.566792 17467 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40870
I20260709 13:22:11.567269  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:11.567509  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:11.567775  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:11.568869  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "06d01143e7944606834891750e8d5b26"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.576020  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:11.584038 17606 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.586494  5410 fs_manager.cc:730] Time spent opening block manager: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:22:11.587416  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "06d01143e7944606834891750e8d5b26"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.588382  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:11.626883  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:11.627962  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:11.629386  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:11.631878  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:11.632077  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.632402  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:11.632567  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.673425  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:44043
I20260709 13:22:11.673624 17668 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:44043 every 8 connection(s)
I20260709 13:22:11.678246  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:11.685537 17672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:11.686226 17673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:11.689989  5410 server_base.cc:1061] running on GCE node
W20260709 13:22:11.689757 17675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:11.691891 17669 heartbeater.cc:344] Connected to a master server at 127.5.72.190:40487
I20260709 13:22:11.692126  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:22:11.692353 17669 heartbeater.cc:461] Registering TS with master...
W20260709 13:22:11.692456  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:11.692875  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603331692851 us; error 0 us; skew 500 ppm
I20260709 13:22:11.693348 17669 heartbeater.cc:507] Master 127.5.72.190:40487 requested a full tablet report, sending...
I20260709 13:22:11.693465  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:11.695760 17467 ts_manager.cc:194] Registered new tserver with Master: 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:11.696384  5410 webserver.cc:533] Webserver started at http://127.5.72.131:39993/ using document root <none> and password file <none>
I20260709 13:22:11.696941  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:11.697151  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:11.697414  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:11.697397 17467 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40878
I20260709 13:22:11.699033  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.703539  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:22:11.707161 17680 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.707949  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.003s
I20260709 13:22:11.708276  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
format_stamp: "Formatted at 2026-07-09 13:22:11 on dist-test-slave-n5kz"
I20260709 13:22:11.708603  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:11.730278  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:11.731460  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:11.732915  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:11.735535  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:11.735793  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.736047  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:11.736222  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:11.775702  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:45027
I20260709 13:22:11.775796 17742 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:45027 every 8 connection(s)
I20260709 13:22:11.789258 17743 heartbeater.cc:344] Connected to a master server at 127.5.72.190:40487
I20260709 13:22:11.789654 17743 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:11.790401 17743 heartbeater.cc:507] Master 127.5.72.190:40487 requested a full tablet report, sending...
I20260709 13:22:11.792378 17467 ts_manager.cc:194] Registered new tserver with Master: 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:11.792515  5410 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013669029s
I20260709 13:22:11.793740 17467 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40894
I20260709 13:22:11.817747 17467 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40902:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:11.820300 17467 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:11.883476 17558 tablet_service.cc:1511] Processing CreateTablet for tablet b2bfb78532214f52b1d57b1d55c26829 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:11.884894 17558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2bfb78532214f52b1d57b1d55c26829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.888355 17559 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6517d1e6eb4aa6855d0e9dab1268d0 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:11.890259 17559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6517d1e6eb4aa6855d0e9dab1268d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.895635 17557 tablet_service.cc:1511] Processing CreateTablet for tablet df281b5696ba48768bbe0dcc6d12911a (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:11.897212 17557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df281b5696ba48768bbe0dcc6d12911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.899752 17556 tablet_service.cc:1511] Processing CreateTablet for tablet a6fcdc3cf8e343849cda6f25a6a918be (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:11.901211 17556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6fcdc3cf8e343849cda6f25a6a918be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.915228 17633 tablet_service.cc:1511] Processing CreateTablet for tablet b2bfb78532214f52b1d57b1d55c26829 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:11.915516 17631 tablet_service.cc:1511] Processing CreateTablet for tablet a6fcdc3cf8e343849cda6f25a6a918be (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:11.916880 17633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2bfb78532214f52b1d57b1d55c26829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.916977 17707 tablet_service.cc:1511] Processing CreateTablet for tablet b2bfb78532214f52b1d57b1d55c26829 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:11.918365 17707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2bfb78532214f52b1d57b1d55c26829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.918853 17708 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6517d1e6eb4aa6855d0e9dab1268d0 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:11.919696 17632 tablet_service.cc:1511] Processing CreateTablet for tablet df281b5696ba48768bbe0dcc6d12911a (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:11.921623 17634 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6517d1e6eb4aa6855d0e9dab1268d0 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:11.915422 17706 tablet_service.cc:1511] Processing CreateTablet for tablet df281b5696ba48768bbe0dcc6d12911a (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:11.923717 17706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df281b5696ba48768bbe0dcc6d12911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.927017 17708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6517d1e6eb4aa6855d0e9dab1268d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.927937 17632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df281b5696ba48768bbe0dcc6d12911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.931885 17634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6517d1e6eb4aa6855d0e9dab1268d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.931517 17705 tablet_service.cc:1511] Processing CreateTablet for tablet a6fcdc3cf8e343849cda6f25a6a918be (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:11.932022 17758 tablet_bootstrap.cc:492] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:11.934180 17705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6fcdc3cf8e343849cda6f25a6a918be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.937100 17558 tablet_service.cc:1511] Processing CreateTablet for tablet dc0466a5a6054a089de335deb2609daf (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:11.938558 17558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc0466a5a6054a089de335deb2609daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.938642 17630 tablet_service.cc:1511] Processing CreateTablet for tablet dc0466a5a6054a089de335deb2609daf (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:11.939682 17631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6fcdc3cf8e343849cda6f25a6a918be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.940881 17704 tablet_service.cc:1511] Processing CreateTablet for tablet dc0466a5a6054a089de335deb2609daf (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:11.942243 17704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc0466a5a6054a089de335deb2609daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.942965 17630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc0466a5a6054a089de335deb2609daf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.943615 17559 tablet_service.cc:1511] Processing CreateTablet for tablet f2340c19eff7470d81b8f44b3e19a87e (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:11.944979 17559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2340c19eff7470d81b8f44b3e19a87e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.946552 17703 tablet_service.cc:1511] Processing CreateTablet for tablet f2340c19eff7470d81b8f44b3e19a87e (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:11.947948 17703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2340c19eff7470d81b8f44b3e19a87e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.948676 17629 tablet_service.cc:1511] Processing CreateTablet for tablet f2340c19eff7470d81b8f44b3e19a87e (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:11.950431 17629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2340c19eff7470d81b8f44b3e19a87e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.964553 17555 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0c1612e14045e184c6df7f45a0819a (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:11.966120 17555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0c1612e14045e184c6df7f45a0819a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.968665 17702 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0c1612e14045e184c6df7f45a0819a (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:11.970145 17702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0c1612e14045e184c6df7f45a0819a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.973050 17758 tablet_bootstrap.cc:654] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:11.987460 17557 tablet_service.cc:1511] Processing CreateTablet for tablet b2226106f1aa42a5ad476a88e16b485e (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:11.988999 17557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2226106f1aa42a5ad476a88e16b485e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.989343 17758 tablet_bootstrap.cc:492] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:11.989943 17758 ts_tablet_manager.cc:1403] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.058s	user 0.004s	sys 0.009s
I20260709 13:22:11.993091 17758 raft_consensus.cc:359] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:11.993885 17758 raft_consensus.cc:385] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:11.995983 17758 raft_consensus.cc:740] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:11.995747 17628 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0c1612e14045e184c6df7f45a0819a (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:11.997227 17628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0c1612e14045e184c6df7f45a0819a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:11.996973 17758 consensus_queue.cc:260] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.002211 17758 ts_tablet_manager.cc:1434] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:22:12.003197 17594 heartbeater.cc:499] Master 127.5.72.190:40487 was elected leader, sending a full tablet report...
I20260709 13:22:12.004055 17627 tablet_service.cc:1511] Processing CreateTablet for tablet b2226106f1aa42a5ad476a88e16b485e (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:12.005402 17627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2226106f1aa42a5ad476a88e16b485e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:12.006632 17758 tablet_bootstrap.cc:492] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.013195 17758 tablet_bootstrap.cc:654] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.047360 17701 tablet_service.cc:1511] Processing CreateTablet for tablet b2226106f1aa42a5ad476a88e16b485e (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:12.048749 17701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2226106f1aa42a5ad476a88e16b485e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:12.062153 17558 tablet_service.cc:1511] Processing CreateTablet for tablet 43e11d78cf48431e9a207fc2d5c7fae3 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:12.063643 17558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e11d78cf48431e9a207fc2d5c7fae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:12.063575 17626 tablet_service.cc:1511] Processing CreateTablet for tablet 43e11d78cf48431e9a207fc2d5c7fae3 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:12.064910 17626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e11d78cf48431e9a207fc2d5c7fae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:12.091266 17762 tablet_bootstrap.cc:492] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.096988 17763 tablet_bootstrap.cc:492] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.100080 17704 tablet_service.cc:1511] Processing CreateTablet for tablet 43e11d78cf48431e9a207fc2d5c7fae3 (DEFAULT_TABLE table=test-workload [id=d0a991f1ec5f4e5bb2b64f2f5bff5093]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:12.103044 17762 tablet_bootstrap.cc:654] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.104907 17704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e11d78cf48431e9a207fc2d5c7fae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:12.104878 17763 tablet_bootstrap.cc:654] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.116405 17758 tablet_bootstrap.cc:492] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.116963 17758 ts_tablet_manager.cc:1403] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.111s	user 0.028s	sys 0.032s
I20260709 13:22:12.121649 17758 raft_consensus.cc:359] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.122721 17758 raft_consensus.cc:385] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.123191 17758 raft_consensus.cc:740] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.127487 17758 consensus_queue.cc:260] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.127950 17762 tablet_bootstrap.cc:492] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.128952 17762 ts_tablet_manager.cc:1403] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.004s
I20260709 13:22:12.129922 17758 ts_tablet_manager.cc:1434] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:22:12.127959 17763 tablet_bootstrap.cc:492] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.131067 17758 tablet_bootstrap.cc:492] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.131601 17763 ts_tablet_manager.cc:1403] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.010s
I20260709 13:22:12.132227 17762 raft_consensus.cc:359] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.136023 17762 raft_consensus.cc:385] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.134693 17763 raft_consensus.cc:359] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.136392 17762 raft_consensus.cc:740] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.136554 17763 raft_consensus.cc:385] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.136973 17763 raft_consensus.cc:740] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.137398 17762 consensus_queue.cc:260] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.137897 17758 tablet_bootstrap.cc:654] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.137838 17763 consensus_queue.cc:260] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.146263 17743 heartbeater.cc:499] Master 127.5.72.190:40487 was elected leader, sending a full tablet report...
I20260709 13:22:12.147398 17762 ts_tablet_manager.cc:1434] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.004s
I20260709 13:22:12.148577 17762 tablet_bootstrap.cc:492] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.180470 17763 ts_tablet_manager.cc:1434] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.004s
I20260709 13:22:12.181857 17669 heartbeater.cc:499] Master 127.5.72.190:40487 was elected leader, sending a full tablet report...
I20260709 13:22:12.182253 17763 tablet_bootstrap.cc:492] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.187422 17758 tablet_bootstrap.cc:492] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.188081 17758 ts_tablet_manager.cc:1403] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.004s
I20260709 13:22:12.188280 17762 tablet_bootstrap.cc:654] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.190991 17758 raft_consensus.cc:359] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.191910 17758 raft_consensus.cc:385] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.192209 17758 raft_consensus.cc:740] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.193002 17758 consensus_queue.cc:260] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.195384 17758 ts_tablet_manager.cc:1434] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:12.196594 17758 tablet_bootstrap.cc:492] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.200206 17762 tablet_bootstrap.cc:492] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.200838 17762 ts_tablet_manager.cc:1403] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.021s
I20260709 13:22:12.202956 17758 tablet_bootstrap.cc:654] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.203706 17762 raft_consensus.cc:359] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.204481 17762 raft_consensus.cc:385] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.204780 17762 raft_consensus.cc:740] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.205542 17762 consensus_queue.cc:260] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.206574 17763 tablet_bootstrap.cc:654] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.213915 17762 ts_tablet_manager.cc:1434] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:22:12.215168 17762 tablet_bootstrap.cc:492] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.216873 17763 tablet_bootstrap.cc:492] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.217535 17763 ts_tablet_manager.cc:1403] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.000s
I20260709 13:22:12.220412 17763 raft_consensus.cc:359] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.221233 17762 tablet_bootstrap.cc:654] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.221282 17763 raft_consensus.cc:385] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.221448 17758 tablet_bootstrap.cc:492] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.221952 17763 raft_consensus.cc:740] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.222373 17758 ts_tablet_manager.cc:1403] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.004s
I20260709 13:22:12.223001 17763 consensus_queue.cc:260] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.226205 17763 ts_tablet_manager.cc:1434] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:12.226393 17758 raft_consensus.cc:359] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.227296 17758 raft_consensus.cc:385] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.227576 17763 tablet_bootstrap.cc:492] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.227662 17758 raft_consensus.cc:740] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.228956 17758 consensus_queue.cc:260] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.232234 17758 ts_tablet_manager.cc:1434] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:22:12.232841 17762 tablet_bootstrap.cc:492] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.233284 17758 tablet_bootstrap.cc:492] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.233450 17762 ts_tablet_manager.cc:1403] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:22:12.236560 17762 raft_consensus.cc:359] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.237439 17762 raft_consensus.cc:385] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.237789 17762 raft_consensus.cc:740] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.238763 17762 consensus_queue.cc:260] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.240036 17758 tablet_bootstrap.cc:654] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.241549 17763 tablet_bootstrap.cc:654] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.243893 17762 ts_tablet_manager.cc:1434] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:22:12.245314 17762 tablet_bootstrap.cc:492] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.250041 17763 tablet_bootstrap.cc:492] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.250720 17763 ts_tablet_manager.cc:1403] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.003s
I20260709 13:22:12.252130 17762 tablet_bootstrap.cc:654] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.253823 17763 raft_consensus.cc:359] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.254794 17763 raft_consensus.cc:385] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.255242 17763 raft_consensus.cc:740] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.256444 17763 consensus_queue.cc:260] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.261750 17758 tablet_bootstrap.cc:492] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.262346 17758 ts_tablet_manager.cc:1403] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.016s
I20260709 13:22:12.262650 17763 ts_tablet_manager.cc:1434] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:22:12.263834 17763 tablet_bootstrap.cc:492] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.263854 17762 tablet_bootstrap.cc:492] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.264679 17762 ts_tablet_manager.cc:1403] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:22:12.265559 17758 raft_consensus.cc:359] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.266485 17758 raft_consensus.cc:385] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.266851 17758 raft_consensus.cc:740] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.267843 17762 raft_consensus.cc:359] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.267791 17758 consensus_queue.cc:260] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.268659 17762 raft_consensus.cc:385] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.269042 17762 raft_consensus.cc:740] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.270012 17762 consensus_queue.cc:260] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.271587 17763 tablet_bootstrap.cc:654] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.272850 17758 ts_tablet_manager.cc:1434] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:22:12.275869 17758 tablet_bootstrap.cc:492] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.274165 17762 ts_tablet_manager.cc:1434] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:12.279989 17762 tablet_bootstrap.cc:492] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.283052 17763 tablet_bootstrap.cc:492] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.283707 17763 ts_tablet_manager.cc:1403] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.004s
I20260709 13:22:12.285626 17762 tablet_bootstrap.cc:654] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.286952 17763 raft_consensus.cc:359] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.287597 17763 raft_consensus.cc:385] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.287842 17763 raft_consensus.cc:740] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.288616 17763 consensus_queue.cc:260] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.290723 17763 ts_tablet_manager.cc:1434] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:12.291652 17758 tablet_bootstrap.cc:654] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.291913 17763 tablet_bootstrap.cc:492] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.299540 17763 tablet_bootstrap.cc:654] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.303192 17762 tablet_bootstrap.cc:492] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.304148 17762 ts_tablet_manager.cc:1403] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.006s
I20260709 13:22:12.306571 17758 tablet_bootstrap.cc:492] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.307240 17758 ts_tablet_manager.cc:1403] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.011s
I20260709 13:22:12.307116 17767 raft_consensus.cc:493] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.307461 17762 raft_consensus.cc:359] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.308418 17762 raft_consensus.cc:385] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.308109 17767 raft_consensus.cc:515] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.308725 17762 raft_consensus.cc:740] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.310218 17762 consensus_queue.cc:260] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.310935 17763 tablet_bootstrap.cc:492] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.310947 17758 raft_consensus.cc:359] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.311636 17763 ts_tablet_manager.cc:1403] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:22:12.311956 17758 raft_consensus.cc:385] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.312323 17758 raft_consensus.cc:740] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.313164 17767 leader_election.cc:290] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.313908 17758 consensus_queue.cc:260] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.315553 17762 ts_tablet_manager.cc:1434] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:22:12.316867 17763 raft_consensus.cc:359] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.317759 17760 raft_consensus.cc:493] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.319077 17767 raft_consensus.cc:493] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.319584 17762 tablet_bootstrap.cc:492] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.319365 17760 raft_consensus.cc:515] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.319552 17767 raft_consensus.cc:515] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.323354 17760 leader_election.cc:290] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.328956 17763 raft_consensus.cc:385] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.329813 17763 raft_consensus.cc:740] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.333415 17763 consensus_queue.cc:260] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.334740 17767 leader_election.cc:290] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.332159 17762 tablet_bootstrap.cc:654] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.354449 17758 ts_tablet_manager.cc:1434] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.046s	user 0.017s	sys 0.016s
I20260709 13:22:12.357388 17763 ts_tablet_manager.cc:1434] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 13:22:12.359915 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.360666 17644 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:12.361012 17763 tablet_bootstrap.cc:492] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.362151 17758 tablet_bootstrap.cc:492] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.362345 17786 raft_consensus.cc:493] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.363051 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.363804 17569 raft_consensus.cc:2468] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:12.365011 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.365705 17568 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:12.360823 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
W20260709 13:22:12.367655 17515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:12.369879 17763 tablet_bootstrap.cc:654] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.370049 17758 tablet_bootstrap.cc:654] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.370805 17610 leader_election.cc:304] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.374398 17767 raft_consensus.cc:2804] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.374570 17610 leader_election.cc:304] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.365851 17533 leader_election.cc:304] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.362877 17786 raft_consensus.cc:515] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.374840 17767 raft_consensus.cc:493] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.375605 17785 raft_consensus.cc:2804] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.359731 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
W20260709 13:22:12.375931 17607 leader_election.cc:343] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.376080 17785 raft_consensus.cc:493] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.376281 17717 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:12.376562 17785 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.377310 17786 leader_election.cc:290] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.375875 17767 raft_consensus.cc:3060] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.378232 17786 raft_consensus.cc:2804] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.378667 17786 raft_consensus.cc:493] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.379010 17786 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.382047 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:12.382762 17718 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:12.382973 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
W20260709 13:22:12.384655 17607 leader_election.cc:343] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.386335 17786 raft_consensus.cc:515] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.388526 17786 leader_election.cc:290] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.390630 17785 raft_consensus.cc:515] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.392102 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:12.392880 17717 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.395722 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:12.397102 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:12.397897 17644 raft_consensus.cc:2393] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b7fc8565fb3b4e63b85f8f408089c384 in current term 1: Already voted for candidate 06d01143e7944606834891750e8d5b26 in this term.
I20260709 13:22:12.398350 17785 leader_election.cc:290] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.399507 17758 tablet_bootstrap.cc:492] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.400092 17758 ts_tablet_manager.cc:1403] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.002s
I20260709 13:22:12.402925 17766 raft_consensus.cc:493] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.403187 17533 leader_election.cc:304] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:12.403728 17766 raft_consensus.cc:515] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.405376 17767 raft_consensus.cc:515] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.403244 17758 raft_consensus.cc:359] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.408190 17758 raft_consensus.cc:385] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.408497 17758 raft_consensus.cc:740] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.419689 17767 leader_election.cc:290] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.421801 17760 raft_consensus.cc:493] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.427533 17760 raft_consensus.cc:515] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.431386 17760 leader_election.cc:290] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.432125 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.432825 17644 raft_consensus.cc:2468] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:12.419896 17766 leader_election.cc:290] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.420506 17785 raft_consensus.cc:2749] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:12.423326 17758 consensus_queue.cc:260] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.441249 17533 leader_election.cc:304] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.421285 17717 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:12.432196 17760 raft_consensus.cc:493] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.441941 17762 tablet_bootstrap.cc:492] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.445396 17785 raft_consensus.cc:2804] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.444484 17758 ts_tablet_manager.cc:1434] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.044s	user 0.003s	sys 0.004s
I20260709 13:22:12.445912 17785 raft_consensus.cc:493] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.446265 17785 raft_consensus.cc:3060] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.446671 17758 tablet_bootstrap.cc:492] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.446936 17760 raft_consensus.cc:515] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.450466 17760 leader_election.cc:290] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.445621 17762 ts_tablet_manager.cc:1403] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.126s	user 0.022s	sys 0.028s
I20260709 13:22:12.451156 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
W20260709 13:22:12.453959 17610 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.72.129:34037, user_credentials={real_user=slave}} blocked reactor thread for 79470us
W20260709 13:22:12.454873 17533 leader_election.cc:343] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.455770 17607 leader_election.cc:304] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:12.456674 17767 raft_consensus.cc:2804] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.456908 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:12.457244 17767 raft_consensus.cc:697] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
W20260709 13:22:12.458393 17607 leader_election.cc:343] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.458256 17767 consensus_queue.cc:237] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.458407 17762 raft_consensus.cc:359] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.460494 17762 raft_consensus.cc:385] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.460839 17762 raft_consensus.cc:740] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.463853 17758 tablet_bootstrap.cc:654] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.466094 17763 tablet_bootstrap.cc:492] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.465880 17762 consensus_queue.cc:260] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.467033 17763 ts_tablet_manager.cc:1403] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.106s	user 0.016s	sys 0.009s
I20260709 13:22:12.466241 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:12.467857 17717 raft_consensus.cc:2468] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:12.468760 17762 ts_tablet_manager.cc:1434] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:22:12.469949 17762 tablet_bootstrap.cc:492] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.470643 17766 raft_consensus.cc:493] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.473340 17785 raft_consensus.cc:515] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.473763 17766 raft_consensus.cc:515] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.477022 17766 leader_election.cc:290] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.481317 17785 leader_election.cc:290] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.481974 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:12.482684 17644 raft_consensus.cc:3060] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.484020 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:12.485639 17643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.486135 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.487195 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.487711 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:12.488515 17568 raft_consensus.cc:2393] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 06d01143e7944606834891750e8d5b26 in current term 1: Already voted for candidate b7fc8565fb3b4e63b85f8f408089c384 in this term.
W20260709 13:22:12.489028 17682 leader_election.cc:343] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.489759 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:12.490309 17762 tablet_bootstrap.cc:654] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.490365 17568 raft_consensus.cc:3060] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.492390 17763 raft_consensus.cc:359] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.502007 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.503088 17569 raft_consensus.cc:2468] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
W20260709 13:22:12.503333 17610 leader_election.cc:343] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.504144 17610 leader_election.cc:304] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d01143e7944606834891750e8d5b26; no voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:12.504729 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.505417 17789 raft_consensus.cc:2749] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:12.505728 17567 raft_consensus.cc:2468] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:12.507779 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
W20260709 13:22:12.508097 17532 leader_election.cc:343] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.508638 17717 raft_consensus.cc:3060] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.508852 17532 leader_election.cc:304] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:12.498216 17763 raft_consensus.cc:385] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.510181 17763 raft_consensus.cc:740] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.511338 17763 consensus_queue.cc:260] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.512493 17684 leader_election.cc:304] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26
I20260709 13:22:12.513752 17766 raft_consensus.cc:2804] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.514137 17766 raft_consensus.cc:493] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.514328 17763 ts_tablet_manager.cc:1434] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.045s	user 0.001s	sys 0.005s
I20260709 13:22:12.514496 17766 raft_consensus.cc:3060] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.514603 17642 raft_consensus.cc:2468] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:12.516418 17785 raft_consensus.cc:2749] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:12.514390 17684 leader_election.cc:304] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.523553 17763 tablet_bootstrap.cc:492] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.499519 17644 raft_consensus.cc:2468] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:12.532727 17533 leader_election.cc:304] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.539711 17717 raft_consensus.cc:2468] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:12.544953 17785 raft_consensus.cc:2804] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.546442 17785 raft_consensus.cc:697] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:12.548194 17785 consensus_queue.cc:237] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.551891 17797 raft_consensus.cc:2804] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.552870 17797 raft_consensus.cc:493] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.553328 17797 raft_consensus.cc:3060] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.546257 17568 raft_consensus.cc:2468] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:12.526589 17766 raft_consensus.cc:515] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.561203 17763 tablet_bootstrap.cc:654] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.562743 17610 leader_election.cc:304] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:12.561610 17466 catalog_manager.cc:5697] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:12.564142 17767 raft_consensus.cc:2804] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.564318 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:12.564878 17767 raft_consensus.cc:697] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
W20260709 13:22:12.565815 17682 leader_election.cc:343] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:12.567067 17766 leader_election.cc:290] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.568812 17766 raft_consensus.cc:493] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.565825 17767 consensus_queue.cc:237] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.569336 17766 raft_consensus.cc:515] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.570752 17762 tablet_bootstrap.cc:492] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.571364 17762 ts_tablet_manager.cc:1403] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.102s	user 0.031s	sys 0.039s
I20260709 13:22:12.572965 17766 leader_election.cc:290] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.573217 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.573997 17758 tablet_bootstrap.cc:492] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.574600 17758 ts_tablet_manager.cc:1403] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.128s	user 0.011s	sys 0.014s
I20260709 13:22:12.574656 17762 raft_consensus.cc:359] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.575577 17762 raft_consensus.cc:385] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.575830 17762 raft_consensus.cc:740] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.576563 17762 consensus_queue.cc:260] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.578363 17762 ts_tablet_manager.cc:1434] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:22:12.578245 17758 raft_consensus.cc:359] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.579977 17758 raft_consensus.cc:385] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.580343 17758 raft_consensus.cc:740] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.581349 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:12.583424 17762 tablet_bootstrap.cc:492] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.582432 17568 raft_consensus.cc:3060] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.586668 17758 consensus_queue.cc:260] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.590497 17758 ts_tablet_manager.cc:1434] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:22:12.592315 17758 tablet_bootstrap.cc:492] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:12.593717 17766 raft_consensus.cc:493] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.594332 17766 raft_consensus.cc:515] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.597821 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.598472 17644 raft_consensus.cc:2468] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:12.603435 17682 leader_election.cc:304] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:12.605819 17797 raft_consensus.cc:515] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.599915 17465 catalog_manager.cc:5697] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:12.616487 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:12.617116 17644 raft_consensus.cc:3060] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.601903 17766 leader_election.cc:290] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.618078 17766 raft_consensus.cc:2804] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.618789 17766 raft_consensus.cc:493] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.622690 17766 raft_consensus.cc:3060] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.626178 17758 tablet_bootstrap.cc:654] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.625429 17797 leader_election.cc:290] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.627415 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.627873 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.628258 17567 raft_consensus.cc:2393] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1c63da9d803242a9b6d8edc7c590b6cd in current term 1: Already voted for candidate 06d01143e7944606834891750e8d5b26 in this term.
I20260709 13:22:12.628517 17569 raft_consensus.cc:2468] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:12.630823 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:12.631462 17569 raft_consensus.cc:3060] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.632196 17762 tablet_bootstrap.cc:654] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.634357 17644 raft_consensus.cc:2468] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:12.635768 17682 leader_election.cc:304] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:12.636586 17797 raft_consensus.cc:2804] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.637669 17797 raft_consensus.cc:697] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:12.638572 17797 consensus_queue.cc:237] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.640823 17763 tablet_bootstrap.cc:492] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.641615 17763 ts_tablet_manager.cc:1403] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.118s	user 0.019s	sys 0.016s
I20260709 13:22:12.643422 17684 leader_election.cc:304] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:12.654989 17763 raft_consensus.cc:359] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.655706 17762 tablet_bootstrap.cc:492] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.655800 17763 raft_consensus.cc:385] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.656201 17763 raft_consensus.cc:740] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.656294 17762 ts_tablet_manager.cc:1403] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.073s	user 0.025s	sys 0.005s
I20260709 13:22:12.657171 17801 raft_consensus.cc:3060] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.659384 17762 raft_consensus.cc:359] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.660130 17762 raft_consensus.cc:385] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.660406 17762 raft_consensus.cc:740] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.661248 17762 consensus_queue.cc:260] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.661850 17568 raft_consensus.cc:2468] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:12.669404 17801 raft_consensus.cc:2749] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:12.664575 17763 consensus_queue.cc:260] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.674270 17763 ts_tablet_manager.cc:1434] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:22:12.664248 17569 raft_consensus.cc:2468] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:12.677870 17684 leader_election.cc:304] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26
I20260709 13:22:12.678506 17763 tablet_bootstrap.cc:492] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.679337 17797 raft_consensus.cc:2804] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.680328 17797 raft_consensus.cc:697] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:12.681165 17797 consensus_queue.cc:237] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.681697 17766 raft_consensus.cc:515] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.688952 17763 tablet_bootstrap.cc:654] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.689559 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:12.690465 17644 raft_consensus.cc:3060] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.697395 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:12.698559 17568 raft_consensus.cc:3060] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.700693 17792 raft_consensus.cc:493] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.701227 17758 tablet_bootstrap.cc:492] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:12.701236 17792 raft_consensus.cc:515] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.702010 17758 ts_tablet_manager.cc:1403] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.110s	user 0.023s	sys 0.002s
I20260709 13:22:12.705137 17758 raft_consensus.cc:359] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.705881 17758 raft_consensus.cc:385] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.706019 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:12.706234 17758 raft_consensus.cc:740] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.706888 17717 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:12.707660 17758 consensus_queue.cc:260] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.708439 17792 leader_election.cc:290] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.709829 17532 leader_election.cc:304] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.707957 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.711071 17642 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:12.711856 17758 ts_tablet_manager.cc:1434] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:22:12.712473 17644 raft_consensus.cc:2468] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:12.713641 17792 raft_consensus.cc:2804] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.714712 17792 raft_consensus.cc:493] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.714484 17682 leader_election.cc:304] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:12.715373 17792 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.720013 17762 ts_tablet_manager.cc:1434] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.063s	user 0.004s	sys 0.010s
I20260709 13:22:12.721019 17762 tablet_bootstrap.cc:492] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:12.724843 17763 tablet_bootstrap.cc:492] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.725639 17763 ts_tablet_manager.cc:1403] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.009s
I20260709 13:22:12.726802 17762 tablet_bootstrap.cc:654] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.709883 17766 leader_election.cc:290] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.716442 17797 raft_consensus.cc:2804] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.727946 17797 raft_consensus.cc:697] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:12.727754 17467 catalog_manager.cc:5697] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 1 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:12.728861 17797 consensus_queue.cc:237] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.729255 17763 raft_consensus.cc:359] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.730469 17763 raft_consensus.cc:385] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.730952 17763 raft_consensus.cc:740] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.732430 17763 consensus_queue.cc:260] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.735256 17568 raft_consensus.cc:2468] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:12.726519 17792 raft_consensus.cc:515] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.735843 17763 ts_tablet_manager.cc:1434] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:22:12.738248 17763 tablet_bootstrap.cc:492] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:12.739193 17792 leader_election.cc:290] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.740193 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:12.741818 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:12.743415 17644 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.743511 17717 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.749854 17767 raft_consensus.cc:493] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.750756 17767 raft_consensus.cc:515] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.755690 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.756515 17568 raft_consensus.cc:2393] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06d01143e7944606834891750e8d5b26 in current term 1: Already voted for candidate 1c63da9d803242a9b6d8edc7c590b6cd in this term.
I20260709 13:22:12.756781 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:12.759239 17607 leader_election.cc:304] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d01143e7944606834891750e8d5b26; no voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:12.760572 17789 raft_consensus.cc:3060] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.766893 17717 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:12.768302 17532 leader_election.cc:304] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.768936 17767 leader_election.cc:290] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:12.769250 17792 raft_consensus.cc:2804] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.769886 17792 raft_consensus.cc:697] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:12.770795 17792 consensus_queue.cc:237] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.770856 17465 catalog_manager.cc:5697] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:12.778103 17789 raft_consensus.cc:2749] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:12.788683 17762 tablet_bootstrap.cc:492] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:12.786028 17763 tablet_bootstrap.cc:654] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.790952 17762 ts_tablet_manager.cc:1403] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.070s	user 0.022s	sys 0.045s
I20260709 13:22:12.794049 17762 raft_consensus.cc:359] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.794755 17762 raft_consensus.cc:385] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.795066 17762 raft_consensus.cc:740] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.795923 17762 consensus_queue.cc:260] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.795536 17644 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:12.798144 17762 ts_tablet_manager.cc:1434] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:22:12.802410 17763 tablet_bootstrap.cc:492] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:12.801767 17466 catalog_manager.cc:5697] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:12.803155 17763 ts_tablet_manager.cc:1403] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.000s
I20260709 13:22:12.804733 17467 catalog_manager.cc:5697] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 1 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:12.806541 17763 raft_consensus.cc:359] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.807394 17763 raft_consensus.cc:385] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.807693 17763 raft_consensus.cc:740] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.808377 17763 consensus_queue.cc:260] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.810523 17763 ts_tablet_manager.cc:1434] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:22:12.807037 17467 catalog_manager.cc:5697] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 1 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:12.821442 17797 raft_consensus.cc:493] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.822218 17797 raft_consensus.cc:515] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.825016 17797 leader_election.cc:290] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.825659 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:12.825701 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:12.826531 17568 raft_consensus.cc:2468] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:12.826658 17644 raft_consensus.cc:2468] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:12.828096 17684 leader_election.cc:304] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:12.828967 17797 raft_consensus.cc:2804] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:12.829355 17797 raft_consensus.cc:493] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:12.829690 17797 raft_consensus.cc:3060] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.835747 17797 raft_consensus.cc:515] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.837404 17797 leader_election.cc:290] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:12.838289 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:12.838416 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:12.839002 17568 raft_consensus.cc:3060] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.839083 17644 raft_consensus.cc:3060] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.844806 17644 raft_consensus.cc:2468] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:12.845058 17568 raft_consensus.cc:2468] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:12.845978 17682 leader_election.cc:304] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:12.846887 17797 raft_consensus.cc:2804] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.847275 17797 raft_consensus.cc:697] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:12.847994 17797 consensus_queue.cc:237] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:12.855695 17465 catalog_manager.cc:5697] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 1 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:12.880190 17789 consensus_queue.cc:1048] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:12.891495 17786 consensus_queue.cc:1048] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:12.953061 17786 consensus_queue.cc:1048] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:12.956462 17786 consensus_queue.cc:1048] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:12.982016 17792 consensus_queue.cc:1048] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:12.993527 17792 consensus_queue.cc:1048] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.028539 17797 consensus_queue.cc:1048] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.039294 17801 consensus_queue.cc:1048] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.053478 17789 raft_consensus.cc:493] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.054039 17789 raft_consensus.cc:515] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.055821 17789 leader_election.cc:290] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.056437 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:13.056807 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:13.057170 17568 raft_consensus.cc:2468] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:13.057451 17717 raft_consensus.cc:2468] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:13.058411 17610 leader_election.cc:304] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.059151 17789 raft_consensus.cc:2804] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:13.059512 17789 raft_consensus.cc:493] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.059733 17789 raft_consensus.cc:3060] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:13.064990 17789 raft_consensus.cc:515] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.066656 17789 leader_election.cc:290] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.067394 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:13.067698 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.068055 17568 raft_consensus.cc:3060] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:13.068259 17717 raft_consensus.cc:3060] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:13.075196 17568 raft_consensus.cc:2468] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:13.075199 17717 raft_consensus.cc:2468] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:13.076501 17607 leader_election.cc:304] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:13.077147 17789 raft_consensus.cc:2804] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:13.077577 17789 raft_consensus.cc:697] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.078150 17789 consensus_queue.cc:237] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.085350 17465 catalog_manager.cc:5697] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.188668 17801 consensus_queue.cc:1048] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:13.200121 17809 consensus_queue.cc:1048] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.206113 17792 consensus_queue.cc:1048] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.212574 17809 consensus_queue.cc:1048] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.221050 17785 consensus_queue.cc:1048] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:13.234668 17766 consensus_queue.cc:1048] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.309348 17644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0"
dest_uuid: "06d01143e7944606834891750e8d5b26"
mode: GRACEFUL
new_leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:42166
I20260709 13:22:13.309952 17644 raft_consensus.cc:606] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Received request to transfer leadership to TS b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:13.314918 17644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829"
dest_uuid: "06d01143e7944606834891750e8d5b26"
mode: GRACEFUL
new_leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:42166
I20260709 13:22:13.315503 17644 raft_consensus.cc:606] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Received request to transfer leadership to TS b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:13.318411 17766 consensus_queue.cc:1048] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.329710 17766 consensus_queue.cc:1048] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.339720 17717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
mode: GRACEFUL
new_leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:45750
I20260709 13:22:13.340256 17717 raft_consensus.cc:606] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Received request to transfer leadership to TS b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:13.345419 17717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
mode: GRACEFUL
new_leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:45750
I20260709 13:22:13.345922 17717 raft_consensus.cc:606] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Received request to transfer leadership to TS b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:13.358539 17568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
mode: GRACEFUL
new_leader_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:41396
I20260709 13:22:13.359145 17568 raft_consensus.cc:606] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Received request to transfer leadership to TS 06d01143e7944606834891750e8d5b26
I20260709 13:22:13.364130 17568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
mode: GRACEFUL
new_leader_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:41396
I20260709 13:22:13.364657 17568 raft_consensus.cc:606] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Received request to transfer leadership to TS 06d01143e7944606834891750e8d5b26
I20260709 13:22:13.369956 17717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
mode: GRACEFUL
new_leader_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:45750
I20260709 13:22:13.370672 17717 raft_consensus.cc:606] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Received request to transfer leadership to TS 06d01143e7944606834891750e8d5b26
I20260709 13:22:13.378147 17717 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
mode: GRACEFUL
new_leader_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:45750
I20260709 13:22:13.378871 17717 raft_consensus.cc:606] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Received request to transfer leadership to TS 06d01143e7944606834891750e8d5b26
I20260709 13:22:13.385494 17644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3"
dest_uuid: "06d01143e7944606834891750e8d5b26"
mode: GRACEFUL
new_leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:42166
I20260709 13:22:13.386139 17644 raft_consensus.cc:606] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Received request to transfer leadership to TS 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:13.449962 17767 raft_consensus.cc:993] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26: : Instructing follower b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.450470 17789 raft_consensus.cc:1081] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Signalling peer b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.451975 17568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:41376
I20260709 13:22:13.452442 17568 raft_consensus.cc:493] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.452831 17568 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.458971 17568 raft_consensus.cc:515] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.460722 17568 leader_election.cc:290] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.461380 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:13.461779 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.462121 17644 raft_consensus.cc:3055] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.462365 17717 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.462451 17644 raft_consensus.cc:740] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.463374 17644 consensus_queue.cc:260] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.464339 17644 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.467983 17717 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.469204 17532 leader_election.cc:304] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.469760 17644 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.470099 17833 raft_consensus.cc:2804] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.470721 17833 raft_consensus.cc:697] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:13.471572 17833 consensus_queue.cc:237] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.477715 17465 catalog_manager.cc:5697] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 1 to 2, leader changed from 06d01143e7944606834891750e8d5b26 (127.5.72.130) to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.481083 17785 raft_consensus.cc:993] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384: : Instructing follower 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.481628 17833 raft_consensus.cc:1081] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Signalling peer 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.483399 17644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf"
dest_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:42146
I20260709 13:22:13.483974 17644 raft_consensus.cc:493] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.484334 17644 raft_consensus.cc:3060] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.489923 17644 raft_consensus.cc:515] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.491911 17644 leader_election.cc:290] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.492583 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:13.493355 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc0466a5a6054a089de335deb2609daf" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.493899 17568 raft_consensus.cc:3055] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.494405 17717 raft_consensus.cc:3060] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.494585 17568 raft_consensus.cc:740] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:13.496249 17568 consensus_queue.cc:260] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.498076 17568 raft_consensus.cc:3060] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.507370 17717 raft_consensus.cc:2468] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.510478 17607 leader_election.cc:304] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:13.512148 17568 raft_consensus.cc:2468] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.512148 17789 raft_consensus.cc:2804] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.515246 17789 raft_consensus.cc:697] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.517001 17789 consensus_queue.cc:237] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.517310 17767 consensus_queue.cc:1048] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:13.533313 17786 raft_consensus.cc:993] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26: : Instructing follower 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:13.533861 17786 raft_consensus.cc:1081] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Signalling peer 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:13.535961 17789 consensus_queue.cc:1048] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:13.536108 17718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:45728
I20260709 13:22:13.532761 17465 catalog_manager.cc:5697] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 1 to 2, leader changed from b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129) to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.538360 17718 raft_consensus.cc:493] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.538792 17718 raft_consensus.cc:3060] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.547099 17718 raft_consensus.cc:515] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.553830 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:13.554390 17718 leader_election.cc:290] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:13.554523 17568 raft_consensus.cc:3060] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.554663 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e11d78cf48431e9a207fc2d5c7fae3" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:13.555472 17644 raft_consensus.cc:3055] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.555809 17644 raft_consensus.cc:740] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.556604 17644 consensus_queue.cc:260] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.557641 17644 raft_consensus.cc:3060] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.561916 17568 raft_consensus.cc:2468] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:13.563184 17684 leader_election.cc:304] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.563944 17766 raft_consensus.cc:2804] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.563892 17644 raft_consensus.cc:2468] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:13.564692 17766 raft_consensus.cc:697] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:13.565500 17766 consensus_queue.cc:237] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.571707 17467 catalog_manager.cc:5697] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 1 to 2, leader changed from 06d01143e7944606834891750e8d5b26 (127.5.72.130) to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 2 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:13.587067 17786 raft_consensus.cc:993] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26: : Instructing follower b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.587543 17789 raft_consensus.cc:1081] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Signalling peer b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.588972 17568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:41376
I20260709 13:22:13.589453 17568 raft_consensus.cc:493] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.589775 17568 raft_consensus.cc:3060] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.595619 17568 raft_consensus.cc:515] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.597337 17568 leader_election.cc:290] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.598395 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:13.598448 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d6517d1e6eb4aa6855d0e9dab1268d0" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.599076 17644 raft_consensus.cc:3055] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.599205 17718 raft_consensus.cc:3060] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.599395 17644 raft_consensus.cc:740] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.600150 17644 consensus_queue.cc:260] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.601092 17644 raft_consensus.cc:3060] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.604652 17718 raft_consensus.cc:2468] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.605752 17532 leader_election.cc:304] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.606169 17644 raft_consensus.cc:2468] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.606459 17833 raft_consensus.cc:2804] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.607074 17833 raft_consensus.cc:697] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:13.607892 17833 consensus_queue.cc:237] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.614560 17467 catalog_manager.cc:5697] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 1 to 2, leader changed from 06d01143e7944606834891750e8d5b26 (127.5.72.130) to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.667899 17809 raft_consensus.cc:993] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd: : Instructing follower b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.668429 17801 raft_consensus.cc:1081] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Signalling peer b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.670022 17568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:41386
I20260709 13:22:13.670306 17809 raft_consensus.cc:993] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd: : Instructing follower b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.670594 17568 raft_consensus.cc:493] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.670917 17809 raft_consensus.cc:1081] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Signalling peer b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:13.671096 17568 raft_consensus.cc:3060] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.672748 17569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:41386
I20260709 13:22:13.673328 17569 raft_consensus.cc:493] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.673688 17569 raft_consensus.cc:3060] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.678701 17568 raft_consensus.cc:515] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.680969 17568 leader_election.cc:290] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.680850 17569 raft_consensus.cc:515] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.681630 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:13.682160 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df281b5696ba48768bbe0dcc6d12911a" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.682289 17644 raft_consensus.cc:3060] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.682883 17718 raft_consensus.cc:3055] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.683228 17718 raft_consensus.cc:740] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:13.684284 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:13.684151 17718 consensus_queue.cc:260] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.684558 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6fcdc3cf8e343849cda6f25a6a918be" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.684867 17642 raft_consensus.cc:3060] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.685217 17717 raft_consensus.cc:3055] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.685441 17718 raft_consensus.cc:3060] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.685572 17717 raft_consensus.cc:740] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:13.686573 17717 consensus_queue.cc:260] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.687739 17717 raft_consensus.cc:3060] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.690176 17644 raft_consensus.cc:2468] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.691346 17533 leader_election.cc:304] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.692173 17833 raft_consensus.cc:2804] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.692641 17833 raft_consensus.cc:697] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:13.692936 17642 raft_consensus.cc:2468] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.693499 17833 consensus_queue.cc:237] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.694025 17533 leader_election.cc:304] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.695025 17785 raft_consensus.cc:2804] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.695178 17569 leader_election.cc:290] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.695071 17718 raft_consensus.cc:2468] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.695763 17785 raft_consensus.cc:697] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:13.696679 17785 consensus_queue.cc:237] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.694959 17717 raft_consensus.cc:2468] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:13.702378 17467 catalog_manager.cc:5697] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 1 to 2, leader changed from 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131) to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.714262 17467 catalog_manager.cc:5697] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 1 to 2, leader changed from 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131) to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.782663 17833 raft_consensus.cc:993] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384: : Instructing follower 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.783175 17785 raft_consensus.cc:1081] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Signalling peer 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.784638 17642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e"
dest_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:42146
I20260709 13:22:13.785099 17642 raft_consensus.cc:493] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.785342 17642 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.790318 17642 raft_consensus.cc:515] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.791976 17642 leader_election.cc:290] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.793143 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:13.793138 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.793947 17569 raft_consensus.cc:3055] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.794018 17717 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.794298 17569 raft_consensus.cc:740] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:13.795100 17569 consensus_queue.cc:260] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.796211 17569 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.799602 17717 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.800693 17607 leader_election.cc:304] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:13.801406 17789 raft_consensus.cc:2804] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.801818 17789 raft_consensus.cc:697] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.801851 17569 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.802671 17789 consensus_queue.cc:237] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.810871 17465 catalog_manager.cc:5697] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 1 to 2, leader changed from b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129) to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.824760 17809 raft_consensus.cc:993] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd: : Instructing follower 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.825304 17809 raft_consensus.cc:1081] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Signalling peer 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.826884 17642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a"
dest_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:42162
I20260709 13:22:13.827313 17642 raft_consensus.cc:493] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.827558 17642 raft_consensus.cc:3060] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.832402 17642 raft_consensus.cc:515] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.833940 17642 leader_election.cc:290] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.834797 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:13.835026 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0c1612e14045e184c6df7f45a0819a" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.835417 17569 raft_consensus.cc:3060] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.835619 17717 raft_consensus.cc:3055] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.835986 17717 raft_consensus.cc:740] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:13.836620 17717 consensus_queue.cc:260] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.837548 17717 raft_consensus.cc:3060] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.840718 17569 raft_consensus.cc:2468] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.841781 17610 leader_election.cc:304] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.842509 17789 raft_consensus.cc:2804] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.842918 17717 raft_consensus.cc:2468] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.842958 17789 raft_consensus.cc:697] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.844077 17789 consensus_queue.cc:237] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.849498 17809 raft_consensus.cc:993] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd: : Instructing follower 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.849965 17766 raft_consensus.cc:1081] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Signalling peer 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:13.851354 17642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e"
dest_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:42162
I20260709 13:22:13.851845 17642 raft_consensus.cc:493] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:13.852324 17642 raft_consensus.cc:3060] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.851776 17467 catalog_manager.cc:5697] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 1 to 2, leader changed from 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131) to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.858157 17642 raft_consensus.cc:515] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.860294 17642 leader_election.cc:290] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:13.861136 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:13.861585 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2226106f1aa42a5ad476a88e16b485e" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:13.861783 17569 raft_consensus.cc:3060] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.862232 17717 raft_consensus.cc:3055] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:13.862563 17717 raft_consensus.cc:740] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:13.863305 17717 consensus_queue.cc:260] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.864244 17717 raft_consensus.cc:3060] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:13.868054 17569 raft_consensus.cc:2468] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.869230 17610 leader_election.cc:304] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:13.869917 17717 raft_consensus.cc:2468] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:13.870088 17789 raft_consensus.cc:2804] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:13.870711 17789 raft_consensus.cc:697] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:13.871539 17789 consensus_queue.cc:237] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:13.878250 17467 catalog_manager.cc:5697] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 1 to 2, leader changed from 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131) to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:13.904976 17717 raft_consensus.cc:1275] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:13.906265 17785 consensus_queue.cc:1048] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:13.915333 17642 raft_consensus.cc:1275] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:13.917728 17833 consensus_queue.cc:1048] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:13.938694 17569 raft_consensus.cc:1275] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:13.940001 17789 consensus_queue.cc:1048] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:13.947336 17717 raft_consensus.cc:1275] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:13.949272 17786 consensus_queue.cc:1048] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.085282 17717 raft_consensus.cc:1275] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.086547 17833 consensus_queue.cc:1048] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.095237 17642 raft_consensus.cc:1275] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.096975 17825 consensus_queue.cc:1048] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.112260 17717 raft_consensus.cc:1275] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.113637 17825 consensus_queue.cc:1048] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.121572 17569 raft_consensus.cc:1275] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.123593 17766 consensus_queue.cc:1048] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.124006 17642 raft_consensus.cc:1275] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.125695 17825 consensus_queue.cc:1048] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.132891 17644 raft_consensus.cc:1275] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.134593 17766 consensus_queue.cc:1048] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.201824 17569 raft_consensus.cc:1275] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.201817 17642 raft_consensus.cc:1275] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.203222 17786 consensus_queue.cc:1048] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.203361 17849 consensus_queue.cc:1048] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.207082 17717 raft_consensus.cc:1275] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.208406 17785 consensus_queue.cc:1048] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.213722 17717 raft_consensus.cc:1275] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.215152 17786 consensus_queue.cc:1048] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.289115 17718 raft_consensus.cc:1275] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.290344 17767 consensus_queue.cc:1048] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.291899 17569 raft_consensus.cc:1275] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.293361 17767 consensus_queue.cc:1048] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.417229 17569 raft_consensus.cc:1275] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.418816 17767 consensus_queue.cc:1048] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:14.429342 17718 raft_consensus.cc:1275] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:14.430874 17767 consensus_queue.cc:1048] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:16.420994 17466 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40908:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:16.424676 17466 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:16.477025 17557 tablet_service.cc:1511] Processing CreateTablet for tablet ff85f4876cbf4cffa6daf36fa7c90d00 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:16.478482 17626 tablet_service.cc:1511] Processing CreateTablet for tablet ff85f4876cbf4cffa6daf36fa7c90d00 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:16.479075 17557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff85f4876cbf4cffa6daf36fa7c90d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.480206 17626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff85f4876cbf4cffa6daf36fa7c90d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.482987 17704 tablet_service.cc:1511] Processing CreateTablet for tablet ff85f4876cbf4cffa6daf36fa7c90d00 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:16.490557 17558 tablet_service.cc:1511] Processing CreateTablet for tablet 67c6843e80d24c7caec391c58dbf8a20 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:16.492123 17704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff85f4876cbf4cffa6daf36fa7c90d00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.494601 17628 tablet_service.cc:1511] Processing CreateTablet for tablet 305ec350e9334ed4ad2a3b2abaae106a (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:16.496110 17628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305ec350e9334ed4ad2a3b2abaae106a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.496833 17708 tablet_service.cc:1511] Processing CreateTablet for tablet 67c6843e80d24c7caec391c58dbf8a20 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:16.498234 17708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c6843e80d24c7caec391c58dbf8a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.483721 17555 tablet_service.cc:1511] Processing CreateTablet for tablet 305ec350e9334ed4ad2a3b2abaae106a (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:16.486435 17559 tablet_service.cc:1511] Processing CreateTablet for tablet 0226e7d22e9641768e53252a60d11a28 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:16.499899 17627 tablet_service.cc:1511] Processing CreateTablet for tablet 0226e7d22e9641768e53252a60d11a28 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:16.501286 17627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0226e7d22e9641768e53252a60d11a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.503940 17632 tablet_service.cc:1511] Processing CreateTablet for tablet 67c6843e80d24c7caec391c58dbf8a20 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:16.504238 17706 tablet_service.cc:1511] Processing CreateTablet for tablet 0226e7d22e9641768e53252a60d11a28 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:16.505439 17632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c6843e80d24c7caec391c58dbf8a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.505861 17706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0226e7d22e9641768e53252a60d11a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.507929 17701 tablet_service.cc:1511] Processing CreateTablet for tablet 305ec350e9334ed4ad2a3b2abaae106a (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:16.509382 17701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305ec350e9334ed4ad2a3b2abaae106a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.492152 17558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c6843e80d24c7caec391c58dbf8a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.511302 17555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305ec350e9334ed4ad2a3b2abaae106a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.516964 17559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0226e7d22e9641768e53252a60d11a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.528833 17556 tablet_service.cc:1511] Processing CreateTablet for tablet bb9d31ac10ca4ccf9ed455dc74dacb68 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:16.530306 17556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb9d31ac10ca4ccf9ed455dc74dacb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.530361 17629 tablet_service.cc:1511] Processing CreateTablet for tablet bb9d31ac10ca4ccf9ed455dc74dacb68 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:16.531953 17629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb9d31ac10ca4ccf9ed455dc74dacb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.532927 17554 tablet_service.cc:1511] Processing CreateTablet for tablet e22afaa8d14241c4aed5a24b4f5183dc (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:16.534384 17554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22afaa8d14241c4aed5a24b4f5183dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.536203 17703 tablet_service.cc:1511] Processing CreateTablet for tablet bb9d31ac10ca4ccf9ed455dc74dacb68 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:16.538166 17703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb9d31ac10ca4ccf9ed455dc74dacb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.558338 17702 tablet_service.cc:1511] Processing CreateTablet for tablet e22afaa8d14241c4aed5a24b4f5183dc (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:16.560058 17702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22afaa8d14241c4aed5a24b4f5183dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.560684 17553 tablet_service.cc:1511] Processing CreateTablet for tablet 82ad5ef915b64154b0e632093755fabe (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:16.562093 17553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ad5ef915b64154b0e632093755fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.564065 17634 tablet_service.cc:1511] Processing CreateTablet for tablet e22afaa8d14241c4aed5a24b4f5183dc (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:16.565708 17634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e22afaa8d14241c4aed5a24b4f5183dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.578339 17633 tablet_service.cc:1511] Processing CreateTablet for tablet 82ad5ef915b64154b0e632093755fabe (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:16.579865 17633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ad5ef915b64154b0e632093755fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.584911 17705 tablet_service.cc:1511] Processing CreateTablet for tablet 82ad5ef915b64154b0e632093755fabe (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:16.586349 17705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ad5ef915b64154b0e632093755fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.621949 17632 tablet_service.cc:1511] Processing CreateTablet for tablet dd975ee6998e455eb4b1a63fbb69b713 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:16.631378 17632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd975ee6998e455eb4b1a63fbb69b713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.621120 17558 tablet_service.cc:1511] Processing CreateTablet for tablet dd975ee6998e455eb4b1a63fbb69b713 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:16.635818 17558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd975ee6998e455eb4b1a63fbb69b713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.649891 17872 tablet_bootstrap.cc:492] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:16.652877 17871 tablet_bootstrap.cc:492] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:16.663723 17871 tablet_bootstrap.cc:654] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.664036 17704 tablet_service.cc:1511] Processing CreateTablet for tablet dd975ee6998e455eb4b1a63fbb69b713 (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:16.665490 17704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd975ee6998e455eb4b1a63fbb69b713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.671548 17873 tablet_bootstrap.cc:492] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:16.674829 17557 tablet_service.cc:1511] Processing CreateTablet for tablet 849994c508bb421eb00d7d8aecb5e09f (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:16.678942 17557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849994c508bb421eb00d7d8aecb5e09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.684481 17872 tablet_bootstrap.cc:654] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.699141 17628 tablet_service.cc:1511] Processing CreateTablet for tablet 849994c508bb421eb00d7d8aecb5e09f (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:16.701335 17628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849994c508bb421eb00d7d8aecb5e09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.710249 17871 tablet_bootstrap.cc:492] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:16.709838 17705 tablet_service.cc:1511] Processing CreateTablet for tablet 849994c508bb421eb00d7d8aecb5e09f (DEFAULT_TABLE table=soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:16.710903 17871 ts_tablet_manager.cc:1403] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.000s
I20260709 13:22:16.711454 17705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849994c508bb421eb00d7d8aecb5e09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:16.714349 17871 raft_consensus.cc:359] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.715425 17871 raft_consensus.cc:385] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.715755 17871 raft_consensus.cc:740] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.716585 17871 consensus_queue.cc:260] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.701386 17873 tablet_bootstrap.cc:654] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.726023 17871 ts_tablet_manager.cc:1434] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:22:16.727257 17872 tablet_bootstrap.cc:492] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:16.727891 17872 ts_tablet_manager.cc:1403] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.000s
I20260709 13:22:16.731165 17872 raft_consensus.cc:359] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.731967 17872 raft_consensus.cc:385] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.732260 17872 raft_consensus.cc:740] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.733009 17872 consensus_queue.cc:260] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.729249 17871 tablet_bootstrap.cc:492] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:16.739733 17872 ts_tablet_manager.cc:1434] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:22:16.740928 17872 tablet_bootstrap.cc:492] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:16.741712 17871 tablet_bootstrap.cc:654] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.749377 17872 tablet_bootstrap.cc:654] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.754598 17871 tablet_bootstrap.cc:492] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:16.755144 17871 ts_tablet_manager.cc:1403] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.013s
I20260709 13:22:16.760676 17873 tablet_bootstrap.cc:492] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:16.761559 17873 ts_tablet_manager.cc:1403] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.002s
I20260709 13:22:16.767622 17871 raft_consensus.cc:359] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.768746 17871 raft_consensus.cc:385] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.769140 17871 raft_consensus.cc:740] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.770150 17871 consensus_queue.cc:260] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.766000 17873 raft_consensus.cc:359] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.771370 17873 raft_consensus.cc:385] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.771720 17873 raft_consensus.cc:740] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.772504 17873 consensus_queue.cc:260] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.775238 17871 ts_tablet_manager.cc:1434] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 13:22:16.775314 17873 ts_tablet_manager.cc:1434] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.006s
I20260709 13:22:16.776271 17871 tablet_bootstrap.cc:492] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:16.776484 17873 tablet_bootstrap.cc:492] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:16.777143 17872 tablet_bootstrap.cc:492] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:16.777770 17872 ts_tablet_manager.cc:1403] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.012s
I20260709 13:22:16.781162 17872 raft_consensus.cc:359] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.782032 17872 raft_consensus.cc:385] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.782377 17872 raft_consensus.cc:740] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.783311 17872 consensus_queue.cc:260] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.784219 17873 tablet_bootstrap.cc:654] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.783806 17871 tablet_bootstrap.cc:654] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.788178 17872 ts_tablet_manager.cc:1434] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:22:16.790876 17872 tablet_bootstrap.cc:492] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:16.797741 17871 tablet_bootstrap.cc:492] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:16.798377 17871 ts_tablet_manager.cc:1403] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 13:22:16.799173 17872 tablet_bootstrap.cc:654] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.801669 17871 raft_consensus.cc:359] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.805145 17873 tablet_bootstrap.cc:492] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:16.805932 17873 ts_tablet_manager.cc:1403] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.008s
I20260709 13:22:16.807205 17871 raft_consensus.cc:385] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.807610 17871 raft_consensus.cc:740] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.808307 17871 consensus_queue.cc:260] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.808568 17872 tablet_bootstrap.cc:492] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:16.809219 17872 ts_tablet_manager.cc:1403] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 13:22:16.809188 17873 raft_consensus.cc:359] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.810163 17873 raft_consensus.cc:385] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.810487 17873 raft_consensus.cc:740] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.810887 17871 ts_tablet_manager.cc:1434] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:22:16.811295 17873 consensus_queue.cc:260] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.812029 17871 tablet_bootstrap.cc:492] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:16.813822 17873 ts_tablet_manager.cc:1434] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:22:16.814883 17873 tablet_bootstrap.cc:492] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:16.819537 17871 tablet_bootstrap.cc:654] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.822130 17873 tablet_bootstrap.cc:654] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.826063 17857 raft_consensus.cc:493] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.826821 17857 raft_consensus.cc:515] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.831240 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:16.832329 17569 raft_consensus.cc:2468] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:16.833710 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:16.835217 17857 leader_election.cc:290] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:16.835457 17684 leader_election.cc:304] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:16.824993 17872 raft_consensus.cc:359] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.837859 17872 raft_consensus.cc:385] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.838368 17872 raft_consensus.cc:740] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.839509 17872 consensus_queue.cc:260] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.836732 17857 raft_consensus.cc:2804] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.840857 17642 raft_consensus.cc:2468] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 0.
I20260709 13:22:16.841265 17857 raft_consensus.cc:493] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.844907 17857 raft_consensus.cc:3060] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.861234 17857 raft_consensus.cc:515] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.864679 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:16.865594 17642 raft_consensus.cc:3060] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.883790 17884 raft_consensus.cc:493] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.884783 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:16.852880 17872 ts_tablet_manager.cc:1434] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.003s
I20260709 13:22:16.885730 17569 raft_consensus.cc:3060] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.887447 17872 tablet_bootstrap.cc:492] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:16.884613 17884 raft_consensus.cc:515] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.893577 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
W20260709 13:22:16.895905 17684 leader_election.cc:343] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.899209 17857 leader_election.cc:290] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:16.899355 17884 leader_election.cc:290] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:16.899809 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:16.898522 17569 raft_consensus.cc:2468] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
W20260709 13:22:16.901856 17682 leader_election.cc:343] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.902431 17682 leader_election.cc:304] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:16.903149 17684 leader_election.cc:304] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:16.903479 17873 tablet_bootstrap.cc:492] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:16.904075 17873 ts_tablet_manager.cc:1403] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.089s	user 0.040s	sys 0.043s
I20260709 13:22:16.904136 17857 raft_consensus.cc:2749] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.904270 17884 raft_consensus.cc:2804] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.901579 17642 raft_consensus.cc:2468] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 1.
I20260709 13:22:16.905581 17871 tablet_bootstrap.cc:492] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:16.906387 17871 ts_tablet_manager.cc:1403] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.095s	user 0.033s	sys 0.055s
I20260709 13:22:16.907648 17873 raft_consensus.cc:359] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.905076 17884 raft_consensus.cc:697] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:16.908428 17873 raft_consensus.cc:385] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.908898 17873 raft_consensus.cc:740] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.909363 17871 raft_consensus.cc:359] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.910308 17871 raft_consensus.cc:385] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.910686 17871 raft_consensus.cc:740] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.909703 17873 consensus_queue.cc:260] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.911624 17871 consensus_queue.cc:260] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.913664 17873 ts_tablet_manager.cc:1434] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:16.913959 17871 ts_tablet_manager.cc:1434] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:22:16.914788 17873 tablet_bootstrap.cc:492] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:16.915056 17871 tablet_bootstrap.cc:492] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:16.922155 17871 tablet_bootstrap.cc:654] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.926599 17873 tablet_bootstrap.cc:654] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.930927 17825 raft_consensus.cc:493] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.931695 17825 raft_consensus.cc:515] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.909337 17884 consensus_queue.cc:237] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.933384 17860 raft_consensus.cc:493] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.933880 17860 raft_consensus.cc:515] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.936409 17860 leader_election.cc:290] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:16.939007 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:16.940376 17825 leader_election.cc:290] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
W20260709 13:22:16.942884 17610 leader_election.cc:343] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.943503 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:16.946662 17872 tablet_bootstrap.cc:654] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.947124 17871 tablet_bootstrap.cc:492] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:16.947631 17871 ts_tablet_manager.cc:1403] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.033s	user 0.005s	sys 0.013s
W20260709 13:22:16.948592 17532 leader_election.cc:343] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.953343 17871 raft_consensus.cc:359] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.955855 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
W20260709 13:22:16.957799 17607 leader_election.cc:343] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.958349 17607 leader_election.cc:304] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d01143e7944606834891750e8d5b26; no voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:16.959259 17860 raft_consensus.cc:2749] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.959883 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:16.960503 17873 tablet_bootstrap.cc:492] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:16.961069 17873 ts_tablet_manager.cc:1403] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.026s
W20260709 13:22:16.961397 17533 leader_election.cc:343] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.961943 17533 leader_election.cc:304] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:16.962911 17888 raft_consensus.cc:2749] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.954429 17871 raft_consensus.cc:385] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.964139 17871 raft_consensus.cc:740] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.964352 17873 raft_consensus.cc:359] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.965129 17873 raft_consensus.cc:385] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.965698 17873 raft_consensus.cc:740] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.965107 17871 consensus_queue.cc:260] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.966446 17873 consensus_queue.cc:260] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.968075 17872 tablet_bootstrap.cc:492] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:16.968092 17871 ts_tablet_manager.cc:1434] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:22:16.967939 17465 catalog_manager.cc:5697] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 1 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:16.968786 17873 ts_tablet_manager.cc:1434] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:16.969338 17871 tablet_bootstrap.cc:492] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:16.969880 17873 tablet_bootstrap.cc:492] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:16.968683 17872 ts_tablet_manager.cc:1403] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.001s
I20260709 13:22:16.974697 17825 raft_consensus.cc:493] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.975714 17872 raft_consensus.cc:359] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.976378 17825 raft_consensus.cc:515] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:16.979504 17825 leader_election.cc:290] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:16.979707 17871 tablet_bootstrap.cc:654] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.980674 17873 tablet_bootstrap.cc:654] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.982012 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ad5ef915b64154b0e632093755fabe" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:16.976691 17872 raft_consensus.cc:385] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.985008 17872 raft_consensus.cc:740] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.986117 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ad5ef915b64154b0e632093755fabe" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
W20260709 13:22:16.988312 17532 leader_election.cc:343] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.987409 17872 consensus_queue.cc:260] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
W20260709 13:22:16.992089 17533 leader_election.cc:343] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.992581 17533 leader_election.cc:304] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:16.994179 17825 raft_consensus.cc:2749] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.995182 17871 tablet_bootstrap.cc:492] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:16.995890 17872 ts_tablet_manager.cc:1434] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 13:22:16.995900 17871 ts_tablet_manager.cc:1403] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.007s
I20260709 13:22:16.996919 17872 tablet_bootstrap.cc:492] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:17.000180 17871 raft_consensus.cc:359] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.001011 17871 raft_consensus.cc:385] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.001312 17871 raft_consensus.cc:740] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.001871 17873 tablet_bootstrap.cc:492] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:17.002437 17873 ts_tablet_manager.cc:1403] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.033s	user 0.001s	sys 0.017s
I20260709 13:22:17.002182 17871 consensus_queue.cc:260] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.004104 17872 tablet_bootstrap.cc:654] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.005013 17871 ts_tablet_manager.cc:1434] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:22:17.006093 17871 tablet_bootstrap.cc:492] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:17.005674 17873 raft_consensus.cc:359] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.012951 17873 raft_consensus.cc:385] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.013302 17873 raft_consensus.cc:740] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.013726 17871 tablet_bootstrap.cc:654] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.014123 17873 consensus_queue.cc:260] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.023089 17873 ts_tablet_manager.cc:1434] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:22:17.024171 17872 tablet_bootstrap.cc:492] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:17.024762 17872 ts_tablet_manager.cc:1403] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.008s
I20260709 13:22:17.024782 17873 tablet_bootstrap.cc:492] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:17.025436 17871 tablet_bootstrap.cc:492] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:17.025964 17871 ts_tablet_manager.cc:1403] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260709 13:22:17.028251 17872 raft_consensus.cc:359] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.029229 17872 raft_consensus.cc:385] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.029009 17871 raft_consensus.cc:359] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.029668 17872 raft_consensus.cc:740] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.029836 17871 raft_consensus.cc:385] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.030321 17871 raft_consensus.cc:740] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.030947 17872 consensus_queue.cc:260] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.031140 17871 consensus_queue.cc:260] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.033501 17872 ts_tablet_manager.cc:1434] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:17.033787 17871 ts_tablet_manager.cc:1434] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:22:17.034271 17825 raft_consensus.cc:493] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.034713 17872 tablet_bootstrap.cc:492] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:17.034806 17825 raft_consensus.cc:515] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.036392 17871 tablet_bootstrap.cc:492] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:17.037106 17825 leader_election.cc:290] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.042654 17872 tablet_bootstrap.cc:654] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.043341 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
W20260709 13:22:17.044715 17533 leader_election.cc:343] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.044917 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.045228 17871 tablet_bootstrap.cc:654] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.045601 17717 raft_consensus.cc:2468] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:17.046944 17532 leader_election.cc:304] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26
I20260709 13:22:17.049026 17825 raft_consensus.cc:2804] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.049557 17825 raft_consensus.cc:493] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.049993 17825 raft_consensus.cc:3060] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.061259 17888 raft_consensus.cc:493] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.062048 17888 raft_consensus.cc:515] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.066773 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305ec350e9334ed4ad2a3b2abaae106a" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.067703 17717 raft_consensus.cc:2468] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:17.069360 17532 leader_election.cc:304] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.071712 17888 leader_election.cc:290] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.072847 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305ec350e9334ed4ad2a3b2abaae106a" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:17.065780 17873 tablet_bootstrap.cc:654] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.075448 17644 raft_consensus.cc:2468] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:17.076822 17888 raft_consensus.cc:2804] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.078325 17888 raft_consensus.cc:493] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.078791 17888 raft_consensus.cc:3060] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.082792 17825 raft_consensus.cc:515] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.086889 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
W20260709 13:22:17.088843 17533 leader_election.cc:343] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.090449 17899 raft_consensus.cc:493] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.091960 17899 raft_consensus.cc:515] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.094296 17872 tablet_bootstrap.cc:492] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:17.094970 17872 ts_tablet_manager.cc:1403] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.040s
I20260709 13:22:17.095784 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:17.096779 17644 raft_consensus.cc:2468] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:17.098388 17872 raft_consensus.cc:359] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.098579 17533 leader_election.cc:304] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.099305 17872 raft_consensus.cc:385] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.099792 17872 raft_consensus.cc:740] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.100633 17872 consensus_queue.cc:260] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.101382 17888 raft_consensus.cc:515] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.104570 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305ec350e9334ed4ad2a3b2abaae106a" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.105125 17717 raft_consensus.cc:3060] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.105911 17872 ts_tablet_manager.cc:1434] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:22:17.107111 17872 tablet_bootstrap.cc:492] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:17.113539 17717 raft_consensus.cc:2468] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:17.116710 17872 tablet_bootstrap.cc:654] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.120115 17871 tablet_bootstrap.cc:492] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:17.120615 17871 ts_tablet_manager.cc:1403] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.085s	user 0.041s	sys 0.032s
I20260709 13:22:17.122929 17532 leader_election.cc:304] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.123515 17871 raft_consensus.cc:359] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.124328 17871 raft_consensus.cc:385] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.124470 17901 raft_consensus.cc:2804] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.124676 17871 raft_consensus.cc:740] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.124969 17901 raft_consensus.cc:493] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.125020 17888 leader_election.cc:290] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.125478 17871 consensus_queue.cc:260] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.126813 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "305ec350e9334ed4ad2a3b2abaae106a" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:17.126554 17903 raft_consensus.cc:2804] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.127713 17899 leader_election.cc:290] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.129956 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.130286 17644 raft_consensus.cc:3060] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.127816 17860 raft_consensus.cc:493] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.130399 17903 raft_consensus.cc:697] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:17.130937 17860 raft_consensus.cc:515] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.131457 17903 consensus_queue.cc:237] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.133253 17860 leader_election.cc:290] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.133253 17873 tablet_bootstrap.cc:492] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:17.134524 17873 ts_tablet_manager.cc:1403] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.003s
I20260709 13:22:17.134604 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:17.135392 17569 raft_consensus.cc:2468] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:17.135720 17872 tablet_bootstrap.cc:492] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:17.136253 17872 ts_tablet_manager.cc:1403] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.011s
I20260709 13:22:17.136878 17610 leader_election.cc:304] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.137919 17873 raft_consensus.cc:359] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.129856 17901 raft_consensus.cc:3060] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.139071 17860 raft_consensus.cc:2804] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.139333 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.128857 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.139434 17872 raft_consensus.cc:359] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.140259 17716 raft_consensus.cc:2468] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:17.140358 17717 raft_consensus.cc:2468] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:17.139716 17718 raft_consensus.cc:3060] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.143028 17644 raft_consensus.cc:2468] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:17.145745 17871 ts_tablet_manager.cc:1434] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:22:17.146783 17871 tablet_bootstrap.cc:492] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384: Bootstrap starting.
I20260709 13:22:17.149729 17873 raft_consensus.cc:385] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.150235 17873 raft_consensus.cc:740] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.150334 17860 raft_consensus.cc:493] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.151192 17860 raft_consensus.cc:3060] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.152840 17871 tablet_bootstrap.cc:654] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.128718 17825 leader_election.cc:290] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.157073 17718 raft_consensus.cc:2468] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:17.140398 17872 raft_consensus.cc:385] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.158423 17872 raft_consensus.cc:740] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.158769 17860 raft_consensus.cc:515] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.159364 17872 consensus_queue.cc:260] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.159564 17532 leader_election.cc:304] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 06d01143e7944606834891750e8d5b26
I20260709 13:22:17.161688 17825 raft_consensus.cc:2804] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.162007 17872 ts_tablet_manager.cc:1434] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.004s
I20260709 13:22:17.162362 17825 raft_consensus.cc:697] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:17.163216 17872 tablet_bootstrap.cc:492] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:17.163646 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:17.163484 17825 consensus_queue.cc:237] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.164259 17569 raft_consensus.cc:3060] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.169137 17903 raft_consensus.cc:493] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.169816 17903 raft_consensus.cc:515] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.169646 17464 catalog_manager.cc:5697] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.175137 17569 raft_consensus.cc:2468] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:17.175891 17903 leader_election.cc:290] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.176276 17610 leader_election.cc:304] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.177321 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:17.177528 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.178054 17644 raft_consensus.cc:2468] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:17.179431 17533 leader_election.cc:304] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.151292 17873 consensus_queue.cc:260] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.177729 17859 raft_consensus.cc:2804] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.180207 17903 raft_consensus.cc:2804] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.180716 17903 raft_consensus.cc:493] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.181152 17903 raft_consensus.cc:3060] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.190866 17718 raft_consensus.cc:2468] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 0.
I20260709 13:22:17.190382 17872 tablet_bootstrap.cc:654] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.200593 17860 leader_election.cc:290] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.200796 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.201442 17718 raft_consensus.cc:3060] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.205823 17901 raft_consensus.cc:515] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.208166 17873 ts_tablet_manager.cc:1434] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.001s
I20260709 13:22:17.210577 17718 raft_consensus.cc:2468] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:17.213117 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:17.213797 17644 raft_consensus.cc:3060] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.215780 17859 raft_consensus.cc:697] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:17.216845 17859 consensus_queue.cc:237] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.220093 17871 tablet_bootstrap.cc:492] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384: No bootstrap required, opened a new log
I20260709 13:22:17.220636 17871 ts_tablet_manager.cc:1403] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384: Time spent bootstrapping tablet: real 0.074s	user 0.034s	sys 0.018s
I20260709 13:22:17.221623 17644 raft_consensus.cc:2468] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:17.223448 17533 leader_election.cc:304] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.224253 17873 tablet_bootstrap.cc:492] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:17.224432 17825 raft_consensus.cc:2804] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.225040 17825 raft_consensus.cc:697] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:17.225970 17825 consensus_queue.cc:237] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.225751 17901 leader_election.cc:290] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.229286 17872 tablet_bootstrap.cc:492] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:17.230594 17872 ts_tablet_manager.cc:1403] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.004s
I20260709 13:22:17.234534 17465 catalog_manager.cc:5697] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.230792 17903 raft_consensus.cc:515] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.223424 17871 raft_consensus.cc:359] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.242138 17871 raft_consensus.cc:385] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.231887 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.241209 17860 raft_consensus.cc:493] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.243722 17860 raft_consensus.cc:515] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.245970 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:17.247643 17644 raft_consensus.cc:3060] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.248471 17718 raft_consensus.cc:3060] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.252915 17903 leader_election.cc:290] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.254032 17860 leader_election.cc:290] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.254410 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:17.255131 17717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.255935 17717 raft_consensus.cc:2393] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06d01143e7944606834891750e8d5b26 in current term 1: Already voted for candidate b7fc8565fb3b4e63b85f8f408089c384 in this term.
I20260709 13:22:17.256587 17872 raft_consensus.cc:359] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.257324 17872 raft_consensus.cc:385] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.257601 17872 raft_consensus.cc:740] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.258649 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.259308 17716 raft_consensus.cc:3060] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.261723 17873 tablet_bootstrap.cc:654] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.263335 17610 leader_election.cc:304] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d01143e7944606834891750e8d5b26; no voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:17.263983 17464 catalog_manager.cc:5697] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.267310 17860 raft_consensus.cc:3060] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.242789 17871 raft_consensus.cc:740] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.273502 17871 consensus_queue.cc:260] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.275650 17871 ts_tablet_manager.cc:1434] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 13:22:17.276036 17644 raft_consensus.cc:2468] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:17.277376 17533 leader_election.cc:304] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.277318 17716 raft_consensus.cc:2468] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:17.278734 17901 raft_consensus.cc:2804] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.279244 17901 raft_consensus.cc:697] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:17.280071 17901 consensus_queue.cc:237] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.281250 17718 raft_consensus.cc:2468] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 1.
I20260709 13:22:17.282809 17872 consensus_queue.cc:260] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.288044 17872 ts_tablet_manager.cc:1434] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 13:22:17.288417 17859 raft_consensus.cc:493] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.289044 17872 tablet_bootstrap.cc:492] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: Bootstrap starting.
I20260709 13:22:17.293243 17859 raft_consensus.cc:515] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.289506 17860 raft_consensus.cc:2749] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.292088 17465 catalog_manager.cc:5697] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.299520 17872 tablet_bootstrap.cc:654] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.300570 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:17.301577 17569 raft_consensus.cc:2468] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:17.309288 17610 leader_election.cc:304] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.310576 17860 raft_consensus.cc:2804] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.311049 17860 raft_consensus.cc:493] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.311288 17860 raft_consensus.cc:3060] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.313611 17873 tablet_bootstrap.cc:492] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:17.314263 17873 ts_tablet_manager.cc:1403] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.090s	user 0.024s	sys 0.016s
I20260709 13:22:17.316013 17872 tablet_bootstrap.cc:492] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: No bootstrap required, opened a new log
I20260709 13:22:17.316537 17872 ts_tablet_manager.cc:1403] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.004s
I20260709 13:22:17.317489 17873 raft_consensus.cc:359] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.318166 17859 leader_election.cc:290] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.318307 17873 raft_consensus.cc:385] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.318967 17873 raft_consensus.cc:740] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.319469 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.319733 17873 consensus_queue.cc:260] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
W20260709 13:22:17.320972 17607 leader_election.cc:343] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.320025 17872 raft_consensus.cc:359] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.322207 17872 raft_consensus.cc:385] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.322296 17873 ts_tablet_manager.cc:1434] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:22:17.322495 17872 raft_consensus.cc:740] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.323532 17873 tablet_bootstrap.cc:492] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:17.324307 17872 consensus_queue.cc:260] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.325898 17907 raft_consensus.cc:493] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.326337 17907 raft_consensus.cc:515] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.328390 17907 leader_election.cc:290] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.329088 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:17.330005 17908 raft_consensus.cc:493] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.334842 17569 raft_consensus.cc:2468] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:17.334841 17908 raft_consensus.cc:515] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.336107 17872 ts_tablet_manager.cc:1434] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:22:17.336370 17610 leader_election.cc:304] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.330756 17873 tablet_bootstrap.cc:654] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.337235 17908 leader_election.cc:290] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.338182 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ad5ef915b64154b0e632093755fabe" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:17.330518 17860 raft_consensus.cc:515] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.338796 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ad5ef915b64154b0e632093755fabe" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.338910 17569 raft_consensus.cc:2468] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:17.339484 17716 raft_consensus.cc:2468] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 0.
I20260709 13:22:17.340621 17610 leader_election.cc:304] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.331729 17464 catalog_manager.cc:5697] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.330036 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" is_pre_election: true
I20260709 13:22:17.341809 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
W20260709 13:22:17.342648 17607 leader_election.cc:343] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.338845 17908 raft_consensus.cc:2804] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.342759 17569 raft_consensus.cc:3060] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.343271 17859 raft_consensus.cc:2804] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.343207 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
W20260709 13:22:17.344800 17607 leader_election.cc:343] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.345374 17873 tablet_bootstrap.cc:492] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:17.345331 17859 raft_consensus.cc:493] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.345839 17859 raft_consensus.cc:3060] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.345989 17873 ts_tablet_manager.cc:1403] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.004s
I20260709 13:22:17.341876 17860 leader_election.cc:290] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.343304 17908 raft_consensus.cc:493] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.347388 17908 raft_consensus.cc:3060] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.349593 17873 raft_consensus.cc:359] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.350358 17873 raft_consensus.cc:385] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.350653 17873 raft_consensus.cc:740] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.351468 17873 consensus_queue.cc:260] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.354058 17873 ts_tablet_manager.cc:1434] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:17.355050 17873 tablet_bootstrap.cc:492] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd: Bootstrap starting.
I20260709 13:22:17.355743 17884 consensus_queue.cc:1048] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.361296 17873 tablet_bootstrap.cc:654] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.362435 17908 raft_consensus.cc:515] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.366036 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:17.366676 17567 raft_consensus.cc:3060] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.374660 17567 raft_consensus.cc:2468] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:17.384253 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.386324 17718 raft_consensus.cc:3060] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.386610 17908 leader_election.cc:290] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.355683 17569 raft_consensus.cc:2468] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:17.386360 17610 leader_election.cc:304] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.392096 17610 leader_election.cc:304] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:17.392513 17873 tablet_bootstrap.cc:492] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd: No bootstrap required, opened a new log
I20260709 13:22:17.393085 17873 ts_tablet_manager.cc:1403] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent bootstrapping tablet: real 0.038s	user 0.028s	sys 0.006s
I20260709 13:22:17.393290 17908 raft_consensus.cc:2804] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.393836 17908 raft_consensus.cc:697] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:17.394668 17908 consensus_queue.cc:237] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.395273 17718 raft_consensus.cc:2468] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:17.396363 17873 raft_consensus.cc:359] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.397313 17873 raft_consensus.cc:385] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.397598 17873 raft_consensus.cc:740] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.398233 17907 raft_consensus.cc:2804] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.398417 17873 consensus_queue.cc:260] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.400228 17873 ts_tablet_manager.cc:1434] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:17.401890 17857 consensus_queue.cc:1048] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.410902 17859 raft_consensus.cc:515] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.415987 17568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ad5ef915b64154b0e632093755fabe" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:17.417039 17859 leader_election.cc:290] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.417888 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ad5ef915b64154b0e632093755fabe" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.418810 17718 raft_consensus.cc:3060] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.421089 17568 raft_consensus.cc:3060] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.427109 17907 raft_consensus.cc:697] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:17.428277 17907 consensus_queue.cc:237] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.431571 17568 raft_consensus.cc:2468] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:17.432906 17610 leader_election.cc:304] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
W20260709 13:22:17.436849 17515 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:17.438937 17920 raft_consensus.cc:2804] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.439554 17920 raft_consensus.cc:697] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:17.440352 17920 consensus_queue.cc:237] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.445912 17718 raft_consensus.cc:2468] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 1.
I20260709 13:22:17.446930 17466 catalog_manager.cc:5697] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.452139 17466 catalog_manager.cc:5697] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.472647 17464 catalog_manager.cc:5697] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.589927 17884 raft_consensus.cc:493] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.590571 17884 raft_consensus.cc:515] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.592624 17884 leader_election.cc:290] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:17.593665 17644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d01143e7944606834891750e8d5b26" is_pre_election: true
I20260709 13:22:17.593568 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384" is_pre_election: true
I20260709 13:22:17.594551 17569 raft_consensus.cc:2393] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1c63da9d803242a9b6d8edc7c590b6cd in current term 1: Already voted for candidate 06d01143e7944606834891750e8d5b26 in this term.
I20260709 13:22:17.595512 17684 leader_election.cc:304] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:17.596119 17884 raft_consensus.cc:3060] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.601326 17884 raft_consensus.cc:2749] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.606977 17920 consensus_queue.cc:1048] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.617082 17907 consensus_queue.cc:1048] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.638566 17901 consensus_queue.cc:1048] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.649279 17903 consensus_queue.cc:1048] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.656522 17903 consensus_queue.cc:1048] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.677536 17903 consensus_queue.cc:1048] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.687970 17642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28"
dest_uuid: "06d01143e7944606834891750e8d5b26"
mode: GRACEFUL
new_leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:42166
I20260709 13:22:17.688614 17642 raft_consensus.cc:606] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Received request to transfer leadership to TS b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:17.694324 17718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
mode: GRACEFUL
new_leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:45750
I20260709 13:22:17.694948 17718 raft_consensus.cc:606] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Received request to transfer leadership to TS b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:17.700840 17569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
mode: GRACEFUL
new_leader_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:41396
I20260709 13:22:17.701503 17569 raft_consensus.cc:606] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Received request to transfer leadership to TS 06d01143e7944606834891750e8d5b26
I20260709 13:22:17.701540 17903 consensus_queue.cc:1048] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.707069 17569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
mode: GRACEFUL
new_leader_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:41396
I20260709 13:22:17.707603 17569 raft_consensus.cc:606] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Received request to transfer leadership to TS 06d01143e7944606834891750e8d5b26
I20260709 13:22:17.715186 17901 consensus_queue.cc:1048] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.721900 17642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f"
dest_uuid: "06d01143e7944606834891750e8d5b26"
mode: GRACEFUL
new_leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:42166
I20260709 13:22:17.723552 17642 raft_consensus.cc:606] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Received request to transfer leadership to TS 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:17.789083 17884 raft_consensus.cc:993] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: : Instructing follower b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:17.789577 17857 raft_consensus.cc:1081] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Signalling peer b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:17.791188 17569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:41386
I20260709 13:22:17.791785 17569 raft_consensus.cc:493] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:17.792104 17569 raft_consensus.cc:3060] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.796700 17569 raft_consensus.cc:515] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.798209 17569 leader_election.cc:290] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.799075 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:17.799682 17642 raft_consensus.cc:3060] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.799594 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67c6843e80d24c7caec391c58dbf8a20" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.800348 17718 raft_consensus.cc:3055] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:17.800576 17718 raft_consensus.cc:740] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:17.801342 17718 consensus_queue.cc:260] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.802313 17718 raft_consensus.cc:3060] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.805352 17642 raft_consensus.cc:2468] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:17.806610 17533 leader_election.cc:304] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.807452 17901 raft_consensus.cc:2804] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:17.808166 17901 raft_consensus.cc:697] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:17.808681 17718 raft_consensus.cc:2468] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:17.809072 17901 consensus_queue.cc:237] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.816186 17465 catalog_manager.cc:5697] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 1 to 2, leader changed from 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131) to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.821462 17901 consensus_queue.cc:1048] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.825533 17907 consensus_queue.cc:1048] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.834136 17903 consensus_queue.cc:1048] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.847601 17903 raft_consensus.cc:993] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: : Instructing follower 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:17.848064 17903 raft_consensus.cc:1081] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Signalling peer 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:17.850508 17642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68"
dest_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:42146
I20260709 13:22:17.851541 17642 raft_consensus.cc:493] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:17.851383 17907 consensus_queue.cc:1048] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.852111 17642 raft_consensus.cc:3060] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.863775 17642 raft_consensus.cc:515] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.866940 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:17.867061 17718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb9d31ac10ca4ccf9ed455dc74dacb68" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:17.867595 17569 raft_consensus.cc:3055] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:17.867702 17718 raft_consensus.cc:3060] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.867910 17569 raft_consensus.cc:740] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:17.868749 17569 consensus_queue.cc:260] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.869598 17569 raft_consensus.cc:3060] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.871826 17642 leader_election.cc:290] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:17.876755 17569 raft_consensus.cc:2468] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:17.878129 17610 leader_election.cc:304] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.878871 17718 raft_consensus.cc:2468] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:17.879024 17907 raft_consensus.cc:2804] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:17.879640 17907 raft_consensus.cc:697] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:17.880491 17907 consensus_queue.cc:237] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.880976 17920 consensus_queue.cc:1048] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.888764 17920 raft_consensus.cc:993] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: : Instructing follower 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:17.889231 17908 raft_consensus.cc:1081] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Signalling peer 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:17.890476 17464 catalog_manager.cc:5697] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 1 to 2, leader changed from b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129) to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.891597 17716 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:45728
I20260709 13:22:17.895499 17716 raft_consensus.cc:493] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:17.895895 17716 raft_consensus.cc:3060] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.902359 17716 raft_consensus.cc:515] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.903211 17860 consensus_queue.cc:1048] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.904389 17716 leader_election.cc:290] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:17.905875 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:17.906572 17642 raft_consensus.cc:3055] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:17.906880 17642 raft_consensus.cc:740] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:17.907691 17642 consensus_queue.cc:260] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.908816 17642 raft_consensus.cc:3060] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.909577 17716 raft_consensus.cc:1240] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Rejecting Update request from peer 06d01143e7944606834891750e8d5b26 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:17.914189 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849994c508bb421eb00d7d8aecb5e09f" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:17.914862 17569 raft_consensus.cc:3060] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:17.916057 17908 consensus_queue.cc:1048] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.920703 17569 raft_consensus.cc:2468] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:17.922894 17684 leader_election.cc:304] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:17.924211 17857 raft_consensus.cc:2804] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:17.924635 17857 raft_consensus.cc:697] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:17.925195 17642 raft_consensus.cc:2468] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:17.925463 17857 consensus_queue.cc:237] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:17.931424 17908 consensus_queue.cc:1048] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.933620 17465 catalog_manager.cc:5697] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 1 to 2, leader changed from 06d01143e7944606834891750e8d5b26 (127.5.72.130) to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 2 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.136413 17825 raft_consensus.cc:993] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: : Instructing follower 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:18.136902 17888 raft_consensus.cc:1081] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Signalling peer 06d01143e7944606834891750e8d5b26 to start an election
I20260709 13:22:18.138319 17642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc"
dest_uuid: "06d01143e7944606834891750e8d5b26"
 from {username='slave'} at 127.0.0.1:42146
I20260709 13:22:18.138779 17642 raft_consensus.cc:493] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:18.139026 17642 raft_consensus.cc:3060] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:18.144757 17642 raft_consensus.cc:515] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:18.146335 17642 leader_election.cc:290] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:18.147095 17569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:18.147414 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e22afaa8d14241c4aed5a24b4f5183dc" candidate_uuid: "06d01143e7944606834891750e8d5b26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:18.147747 17569 raft_consensus.cc:3055] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:18.148110 17716 raft_consensus.cc:3060] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:18.148111 17569 raft_consensus.cc:740] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:18.149140 17569 consensus_queue.cc:260] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:18.150323 17569 raft_consensus.cc:3060] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:18.154573 17716 raft_consensus.cc:2468] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:18.156349 17607 leader_election.cc:304] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:18.157315 17908 raft_consensus.cc:2804] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:18.157702 17569 raft_consensus.cc:2468] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d01143e7944606834891750e8d5b26 in term 2.
I20260709 13:22:18.157820 17908 raft_consensus.cc:697] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Becoming Leader. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:18.158967 17908 consensus_queue.cc:237] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:18.166728 17465 catalog_manager.cc:5697] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 reported cstate change: term changed from 1 to 2, leader changed from b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129) to 06d01143e7944606834891750e8d5b26 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "06d01143e7944606834891750e8d5b26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.172267 17859 raft_consensus.cc:993] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: : Instructing follower b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:18.172703 17908 raft_consensus.cc:1081] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Signalling peer b7fc8565fb3b4e63b85f8f408089c384 to start an election
I20260709 13:22:18.174340 17569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
 from {username='slave'} at 127.0.0.1:41376
I20260709 13:22:18.174899 17569 raft_consensus.cc:493] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:18.175203 17569 raft_consensus.cc:3060] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:18.179554 17569 raft_consensus.cc:515] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:18.181133 17569 leader_election.cc:290] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Requested vote from peers 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043), 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131:45027)
I20260709 13:22:18.181913 17642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:18.182170 17716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0226e7d22e9641768e53252a60d11a28" candidate_uuid: "b7fc8565fb3b4e63b85f8f408089c384" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
I20260709 13:22:18.182557 17642 raft_consensus.cc:3055] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:18.182808 17716 raft_consensus.cc:3060] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:18.182965 17642 raft_consensus.cc:740] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:18.183718 17642 consensus_queue.cc:260] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:18.184582 17642 raft_consensus.cc:3060] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:18.189586 17716 raft_consensus.cc:2468] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:18.190603 17642 raft_consensus.cc:2468] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7fc8565fb3b4e63b85f8f408089c384 in term 2.
I20260709 13:22:18.190800 17532 leader_election.cc:304] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:18.191583 17888 raft_consensus.cc:2804] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:18.192085 17888 raft_consensus.cc:697] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Becoming Leader. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:18.192935 17888 consensus_queue.cc:237] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:18.200836 17465 catalog_manager.cc:5697] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 reported cstate change: term changed from 1 to 2, leader changed from 06d01143e7944606834891750e8d5b26 (127.5.72.130) to b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "b7fc8565fb3b4e63b85f8f408089c384" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.285297 17716 raft_consensus.cc:1275] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.286566 17908 consensus_queue.cc:1048] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.298792 17569 raft_consensus.cc:1275] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.300961 17860 consensus_queue.cc:1048] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.303259 17642 raft_consensus.cc:1275] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.305189 17888 consensus_queue.cc:1048] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.315671 17718 raft_consensus.cc:1275] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.316891 17903 consensus_queue.cc:1048] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.349246 17642 raft_consensus.cc:1275] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.350484 17857 consensus_queue.cc:1048] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.359320 17569 raft_consensus.cc:1275] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.361222 17884 consensus_queue.cc:1048] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.617234 17642 raft_consensus.cc:1275] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.618403 17901 consensus_queue.cc:1048] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.626428 17716 raft_consensus.cc:1275] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer b7fc8565fb3b4e63b85f8f408089c384: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.628206 17888 consensus_queue.cc:1048] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:18.690582 17716 raft_consensus.cc:1275] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.692158 17859 consensus_queue.cc:1048] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:18.700825 17569 raft_consensus.cc:1275] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 06d01143e7944606834891750e8d5b26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:18.702363 17860 consensus_queue.cc:1048] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:21.339527 17465 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40908:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:22:21.377508 17465 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40908:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:22:21.420181  5410 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:22:21.483603 17986 tablet.cc:1724] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.583909 18000 tablet.cc:1724] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:21.624965 17982 log.cc:927] Time spent T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:22:21.649091 18005 tablet.cc:1724] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.584270 18001 tablet.cc:1724] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:21.725986 17670 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:bb9d31ac10ca4ccf9ed455dc74dacb68)
W20260709 13:22:21.739538 17595 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:bb9d31ac10ca4ccf9ed455dc74dacb68)
W20260709 13:22:21.740410 17670 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:bb9d31ac10ca4ccf9ed455dc74dacb68)
I20260709 13:22:21.742974 18004 tablet.cc:1724] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:21.747660 17994 log.cc:927] Time spent T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: Append to log took a long time: real 0.076s	user 0.006s	sys 0.008s
W20260709 13:22:21.737509 17532 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.131:45027, user_credentials={real_user=slave}} blocked reactor thread for 74348.2us
I20260709 13:22:21.795022 18011 tablet.cc:1724] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:21.800133 17684 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:34037, user_credentials={real_user=slave}} blocked reactor thread for 63133.7us
I20260709 13:22:21.814043 18014 tablet.cc:1724] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.796556 17986 tablet.cc:1724] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.814414 18000 tablet.cc:1724] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.804661 18009 tablet.cc:1724] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.857165 17641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713"
dest_uuid: "06d01143e7944606834891750e8d5b26"
mode: GRACEFUL
new_leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:44864
I20260709 13:22:21.858249 17641 raft_consensus.cc:606] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Received request to transfer leadership to TS 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:21.827100 18001 tablet.cc:1724] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.864600 18010 tablet.cc:1724] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.881129 18016 tablet.cc:1724] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.903884 18005 tablet.cc:1724] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.907544 18011 tablet.cc:1724] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.898023 17986 tablet.cc:1724] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.922029 17465 catalog_manager.cc:5789] TS 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043) does not have the latest schema for tablet 82ad5ef915b64154b0e632093755fabe (table soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]). Expected version 1 got 0
I20260709 13:22:21.901939 18009 tablet.cc:1724] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.947075 18021 tablet.cc:1724] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.950042 18011 tablet.cc:1724] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.915555 18010 tablet.cc:1724] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.970713 17464 catalog_manager.cc:5789] TS b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037) does not have the latest schema for tablet 67c6843e80d24c7caec391c58dbf8a20 (table soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187]). Expected version 1 got 0
I20260709 13:22:21.975896 18005 tablet.cc:1724] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.976392 18014 tablet.cc:1724] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.978433 18016 tablet.cc:1724] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:21.979070 18020 tablet.cc:1724] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:22.001916 18001 tablet.cc:1724] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:22.011773 18004 tablet.cc:1724] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:22.014219 17565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
mode: GRACEFUL
new_leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:22:22.014941 17565 raft_consensus.cc:606] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Received request to transfer leadership to TS 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:22.016515  5410 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:22:22.016940 17957 raft_consensus.cc:993] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: : Instructing follower 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.017553 17954 raft_consensus.cc:1081] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Signalling peer 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.020270 17717 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:45728
I20260709 13:22:22.020784 17717 raft_consensus.cc:493] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:22.021064 17717 raft_consensus.cc:3060] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:22.027499 17951 raft_consensus.cc:993] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: : Instructing follower 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.028043 17951 raft_consensus.cc:1081] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Signalling peer 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.028878 17717 raft_consensus.cc:515] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.030820 17716 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:45742
I20260709 13:22:22.031497 17716 raft_consensus.cc:493] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:22.025879 18014 tablet.cc:1724] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:22.031947 17716 raft_consensus.cc:3060] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:22.034487 17643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:22.035140 17643 raft_consensus.cc:3055] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:22.035460 17643 raft_consensus.cc:740] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:22.036276 17643 consensus_queue.cc:260] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.037369 17643 raft_consensus.cc:3060] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:22.032182 17565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd975ee6998e455eb4b1a63fbb69b713" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:22.038903 17565 raft_consensus.cc:3060] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:22.040184  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:22.033066 17717 leader_election.cc:290] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:22.046711 17643 raft_consensus.cc:2468] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:22.048126 17682 leader_election.cc:304] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:22.049170 17948 raft_consensus.cc:2804] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:22.049661 17948 raft_consensus.cc:697] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:22.049767 17565 raft_consensus.cc:2468] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:22.050594 17948 consensus_queue.cc:237] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.054848 17716 raft_consensus.cc:515] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.056684 17716 leader_election.cc:290] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:22.057780 17643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:22.058466 17643 raft_consensus.cc:3060] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:22.058799 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829"
dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
mode: GRACEFUL
new_leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:55628
I20260709 13:22:22.059434 17567 raft_consensus.cc:606] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Received request to transfer leadership to TS 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:22.060756 17565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff85f4876cbf4cffa6daf36fa7c90d00" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:22.061371 17565 raft_consensus.cc:3055] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:22.061702 17565 raft_consensus.cc:740] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:22.062593 17641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e"
dest_uuid: "06d01143e7944606834891750e8d5b26"
mode: GRACEFUL
new_leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:44864
I20260709 13:22:22.063323 17641 raft_consensus.cc:606] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Received request to transfer leadership to TS 1c63da9d803242a9b6d8edc7c590b6cd
I20260709 13:22:22.064122 17565 consensus_queue.cc:260] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.064925  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:22.065490 17565 raft_consensus.cc:3060] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:22.066097  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:22.070000 17643 raft_consensus.cc:2468] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:22.071437 17682 leader_election.cc:304] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d01143e7944606834891750e8d5b26, 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 
I20260709 13:22:22.072268 17948 raft_consensus.cc:2804] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:22.072793 17948 raft_consensus.cc:697] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:22.073132 17565 raft_consensus.cc:2468] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 2.
I20260709 13:22:22.073882 17948 consensus_queue.cc:237] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.080387 17464 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6: soft_deleted_table [id=c6386dc9aa3c48aaab5084fb104b5187] alter complete (version 1)
I20260709 13:22:22.088908 17466 catalog_manager.cc:5697] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 1 to 2, leader changed from 06d01143e7944606834891750e8d5b26 (127.5.72.130) to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 2 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:22.090121 17466 catalog_manager.cc:5697] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 1 to 2, leader changed from b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129) to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 2 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:22.243443 17665 debug-util.cc:398] Leaking SignalData structure 0x7b08001f75c0 after lost signal to thread 5413
W20260709 13:22:22.244706 17665 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9180 after lost signal to thread 17501
W20260709 13:22:22.247409 17665 debug-util.cc:398] Leaking SignalData structure 0x7b0800172120 after lost signal to thread 17742
I20260709 13:22:22.293205 17954 raft_consensus.cc:993] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26: : Instructing follower 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.293825 17957 raft_consensus.cc:1081] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Signalling peer 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.295409 17716 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:45728
I20260709 13:22:22.295979 17716 raft_consensus.cc:493] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:22.297034 17716 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.302847 17716 raft_consensus.cc:515] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.304430 17716 leader_election.cc:290] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:22.305326 17565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:22.305627 17643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2340c19eff7470d81b8f44b3e19a87e" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:22.306025 17565 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.306200 17643 raft_consensus.cc:3055] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:22.306643 17643 raft_consensus.cc:740] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06d01143e7944606834891750e8d5b26, State: Running, Role: LEADER
I20260709 13:22:22.307524 17643 consensus_queue.cc:260] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.308796 17643 raft_consensus.cc:3060] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.313702 17565 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 3.
I20260709 13:22:22.315172 17684 leader_election.cc:304] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:22.316056 17948 raft_consensus.cc:2804] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:22.316170 17643 raft_consensus.cc:2468] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 3.
I20260709 13:22:22.316634 17948 raft_consensus.cc:697] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:22.317696 17948 consensus_queue.cc:237] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.327721 17466 catalog_manager.cc:5697] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 2 to 3, leader changed from 06d01143e7944606834891750e8d5b26 (127.5.72.130) to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 3 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:22.438669 17951 raft_consensus.cc:993] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384: : Instructing follower 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.439327 18018 raft_consensus.cc:1081] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Signalling peer 1c63da9d803242a9b6d8edc7c590b6cd to start an election
I20260709 13:22:22.440828 17716 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829"
dest_uuid: "1c63da9d803242a9b6d8edc7c590b6cd"
 from {username='slave'} at 127.0.0.1:45742
I20260709 13:22:22.441269 17716 raft_consensus.cc:493] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:22.441532 17716 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.446179 17716 raft_consensus.cc:515] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.447956 17716 leader_election.cc:290] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 election: Requested vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
I20260709 13:22:22.448963 17565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7fc8565fb3b4e63b85f8f408089c384"
I20260709 13:22:22.449647 17565 raft_consensus.cc:3055] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:22.450243 17565 raft_consensus.cc:740] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b7fc8565fb3b4e63b85f8f408089c384, State: Running, Role: LEADER
I20260709 13:22:22.449489 17643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2bfb78532214f52b1d57b1d55c26829" candidate_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06d01143e7944606834891750e8d5b26"
I20260709 13:22:22.450997 17643 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.451148 17565 consensus_queue.cc:260] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.452529 17565 raft_consensus.cc:3060] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.460310 17565 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 3.
I20260709 13:22:22.464044 17684 leader_election.cc:304] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd, b7fc8565fb3b4e63b85f8f408089c384; no voters: 
I20260709 13:22:22.465023 17948 raft_consensus.cc:2804] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:22.465415 17643 raft_consensus.cc:2468] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c63da9d803242a9b6d8edc7c590b6cd in term 3.
I20260709 13:22:22.465595 17948 raft_consensus.cc:697] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 LEADER]: Becoming Leader. State: Replica: 1c63da9d803242a9b6d8edc7c590b6cd, State: Running, Role: LEADER
I20260709 13:22:22.466676 17948 consensus_queue.cc:237] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:22.475598 17466 catalog_manager.cc:5697] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd reported cstate change: term changed from 2 to 3, leader changed from b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129) to 1c63da9d803242a9b6d8edc7c590b6cd (127.5.72.131). New cstate: current_term: 3 leader_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:22.502133 17565 raft_consensus.cc:1275] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:22.503541 17948 consensus_queue.cc:1048] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:22.512882 17643 raft_consensus.cc:1275] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:22.516053 18019 consensus_queue.cc:1048] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:22.532114 17565 raft_consensus.cc:1275] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:22.538797 18019 consensus_queue.cc:1048] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:22.549930 17643 raft_consensus.cc:1275] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:22.551524 17948 consensus_queue.cc:1048] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:22.735060 17565 raft_consensus.cc:1275] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:22.736354 18019 consensus_queue.cc:1048] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:22.745604 17641 raft_consensus.cc:1275] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:22.747191 17948 consensus_queue.cc:1048] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:23.024098 17565 raft_consensus.cc:1275] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:23.025524 17948 consensus_queue.cc:1048] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:23.038383 17641 raft_consensus.cc:1275] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Refusing update from remote peer 1c63da9d803242a9b6d8edc7c590b6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:23.042273 18019 consensus_queue.cc:1048] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:23.142867  5410 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:22:23.150858  5410 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:22:23.151437  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:23.156667  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:22:23.157085  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:24.189842  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
I20260709 13:22:24.234406  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:24.235194  5410 tablet_replica.cc:333] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.235827  5410 raft_consensus.cc:2243] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.236346  5410 raft_consensus.cc:2272] T 849994c508bb421eb00d7d8aecb5e09f P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.238281  5410 tablet_replica.cc:333] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.238950  5410 raft_consensus.cc:2243] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.239403  5410 raft_consensus.cc:2272] T dd975ee6998e455eb4b1a63fbb69b713 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.241413  5410 tablet_replica.cc:333] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.241953  5410 raft_consensus.cc:2243] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.242408  5410 raft_consensus.cc:2272] T bb9d31ac10ca4ccf9ed455dc74dacb68 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.244616  5410 tablet_replica.cc:333] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.245199  5410 raft_consensus.cc:2243] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.245713  5410 raft_consensus.cc:2272] T e22afaa8d14241c4aed5a24b4f5183dc P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.249295  5410 tablet_replica.cc:333] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.250203  5410 raft_consensus.cc:2243] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.250814  5410 raft_consensus.cc:2272] T 82ad5ef915b64154b0e632093755fabe P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.253005  5410 tablet_replica.cc:333] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.253705  5410 raft_consensus.cc:2243] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.254506  5410 raft_consensus.cc:2272] T 305ec350e9334ed4ad2a3b2abaae106a P b7fc8565fb3b4e63b85f8f408089c384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.256515  5410 tablet_replica.cc:333] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.257186  5410 raft_consensus.cc:2243] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.257768  5410 raft_consensus.cc:2272] T ff85f4876cbf4cffa6daf36fa7c90d00 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.259917  5410 tablet_replica.cc:333] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.260435  5410 raft_consensus.cc:2243] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.261191  5410 raft_consensus.cc:2272] T 67c6843e80d24c7caec391c58dbf8a20 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.263083  5410 tablet_replica.cc:333] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.263599  5410 raft_consensus.cc:2243] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.264309  5410 raft_consensus.cc:2272] T 0226e7d22e9641768e53252a60d11a28 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.266100  5410 tablet_replica.cc:333] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.266716  5410 raft_consensus.cc:2243] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.267460  5410 raft_consensus.cc:2272] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.269618  5410 tablet_replica.cc:333] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.270239  5410 raft_consensus.cc:2243] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.270746  5410 raft_consensus.cc:2272] T dc0466a5a6054a089de335deb2609daf P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.272943  5410 tablet_replica.cc:333] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.273535  5410 raft_consensus.cc:2243] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.274013  5410 raft_consensus.cc:2272] T 3d0c1612e14045e184c6df7f45a0819a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.276679  5410 tablet_replica.cc:333] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.277248  5410 raft_consensus.cc:2243] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.277704  5410 raft_consensus.cc:2272] T f2340c19eff7470d81b8f44b3e19a87e P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.279975  5410 tablet_replica.cc:333] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.280489  5410 raft_consensus.cc:2243] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.280905  5410 raft_consensus.cc:2272] T 43e11d78cf48431e9a207fc2d5c7fae3 P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.282948  5410 tablet_replica.cc:333] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.283461  5410 raft_consensus.cc:2243] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.283905  5410 raft_consensus.cc:2272] T b2226106f1aa42a5ad476a88e16b485e P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.285930  5410 tablet_replica.cc:333] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.286423  5410 raft_consensus.cc:2243] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.287189  5410 raft_consensus.cc:2272] T a6fcdc3cf8e343849cda6f25a6a918be P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.289281  5410 tablet_replica.cc:333] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.289798  5410 raft_consensus.cc:2243] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.290546  5410 raft_consensus.cc:2272] T df281b5696ba48768bbe0dcc6d12911a P b7fc8565fb3b4e63b85f8f408089c384 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.292972  5410 tablet_replica.cc:333] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384: stopping tablet replica
I20260709 13:22:24.293453  5410 raft_consensus.cc:2243] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.293850  5410 raft_consensus.cc:2272] T b2bfb78532214f52b1d57b1d55c26829 P b7fc8565fb3b4e63b85f8f408089c384 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:24.325898 17610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260709 13:22:24.327683  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
W20260709 13:22:24.332285 17610 consensus_peers.cc:597] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.357409 17610 consensus_peers.cc:597] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:24.365023  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
I20260709 13:22:24.401562  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:24.402539  5410 tablet_replica.cc:333] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.403501  5410 raft_consensus.cc:2243] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.404170  5410 raft_consensus.cc:2272] T 849994c508bb421eb00d7d8aecb5e09f P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.407086  5410 tablet_replica.cc:333] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.407923  5410 raft_consensus.cc:2243] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.408468  5410 raft_consensus.cc:2272] T dd975ee6998e455eb4b1a63fbb69b713 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.411163  5410 tablet_replica.cc:333] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.412230  5410 raft_consensus.cc:2243] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.413359  5410 raft_consensus.cc:2272] T 82ad5ef915b64154b0e632093755fabe P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.416124  5410 tablet_replica.cc:333] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.416965  5410 raft_consensus.cc:2243] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.418013  5410 raft_consensus.cc:2272] T e22afaa8d14241c4aed5a24b4f5183dc P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.420588  5410 tablet_replica.cc:333] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.421406  5410 raft_consensus.cc:2243] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.421988  5410 raft_consensus.cc:2272] T 305ec350e9334ed4ad2a3b2abaae106a P 06d01143e7944606834891750e8d5b26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.424623  5410 tablet_replica.cc:333] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.425439  5410 raft_consensus.cc:2243] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.426029  5410 raft_consensus.cc:2272] T 0226e7d22e9641768e53252a60d11a28 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.428575  5410 tablet_replica.cc:333] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.429364  5410 raft_consensus.cc:2243] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.429924  5410 raft_consensus.cc:2272] T 67c6843e80d24c7caec391c58dbf8a20 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.432530  5410 tablet_replica.cc:333] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.433346  5410 raft_consensus.cc:2243] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.434022  5410 raft_consensus.cc:2272] T ff85f4876cbf4cffa6daf36fa7c90d00 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.436838  5410 tablet_replica.cc:333] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.437687  5410 raft_consensus.cc:2243] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:24.438112 17610 consensus_peers.cc:597] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:24.438300  5410 raft_consensus.cc:2272] T f2340c19eff7470d81b8f44b3e19a87e P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.441597  5410 tablet_replica.cc:333] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.442448  5410 raft_consensus.cc:2243] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.443127  5410 raft_consensus.cc:2272] T 43e11d78cf48431e9a207fc2d5c7fae3 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.446533  5410 tablet_replica.cc:333] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.447475  5410 raft_consensus.cc:2243] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.448863  5410 raft_consensus.cc:2272] T dc0466a5a6054a089de335deb2609daf P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.452111  5410 tablet_replica.cc:333] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.452906  5410 raft_consensus.cc:2243] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.453500  5410 raft_consensus.cc:2272] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.456866  5410 tablet_replica.cc:333] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.457759  5410 raft_consensus.cc:2243] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.458988  5410 raft_consensus.cc:2272] T 3d0c1612e14045e184c6df7f45a0819a P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.462165  5410 tablet_replica.cc:333] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.463042  5410 raft_consensus.cc:2243] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.463657  5410 raft_consensus.cc:2272] T df281b5696ba48768bbe0dcc6d12911a P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.466718  5410 tablet_replica.cc:333] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.467484  5410 raft_consensus.cc:2243] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.468554  5410 raft_consensus.cc:2272] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.471563  5410 tablet_replica.cc:333] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.472419  5410 raft_consensus.cc:2243] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:24.473548  5410 raft_consensus.cc:2272] T b2226106f1aa42a5ad476a88e16b485e P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.476594  5410 tablet_replica.cc:333] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.477309  5410 raft_consensus.cc:2243] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.477861  5410 raft_consensus.cc:2272] T a6fcdc3cf8e343849cda6f25a6a918be P 06d01143e7944606834891750e8d5b26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:24.480885  5410 tablet_replica.cc:333] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26: stopping tablet replica
I20260709 13:22:24.481534  5410 raft_consensus.cc:2243] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:24.482088  5410 raft_consensus.cc:2272] T b2bfb78532214f52b1d57b1d55c26829 P 06d01143e7944606834891750e8d5b26 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:24.490483 17684 consensus_peers.cc:597] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.497212 17684 consensus_peers.cc:597] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.513931 17682 consensus_peers.cc:597] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Couldn't send request to peer 06d01143e7944606834891750e8d5b26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.523449 17682 consensus_peers.cc:597] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Couldn't send request to peer 06d01143e7944606834891750e8d5b26. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.572639 17682 consensus_peers.cc:597] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Couldn't send request to peer 06d01143e7944606834891750e8d5b26. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.605827 17682 consensus_peers.cc:597] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Couldn't send request to peer 06d01143e7944606834891750e8d5b26. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.622012 17684 consensus_peers.cc:597] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.644095 17682 consensus_peers.cc:597] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Couldn't send request to peer 06d01143e7944606834891750e8d5b26. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.647614 17682 consensus_peers.cc:597] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Couldn't send request to peer 06d01143e7944606834891750e8d5b26. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.649029 17684 consensus_peers.cc:597] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.674491 17684 consensus_peers.cc:597] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:24.765245 17684 consensus_peers.cc:597] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd -> Peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Couldn't send request to peer b7fc8565fb3b4e63b85f8f408089c384. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:25.374446 18053 raft_consensus.cc:493] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b7fc8565fb3b4e63b85f8f408089c384)
I20260709 13:22:25.374979 18053 raft_consensus.cc:515] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:25.376686 18053 leader_election.cc:290] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
W20260709 13:22:25.380913 17682 leader_election.cc:336] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111)
W20260709 13:22:25.381394 17684 leader_election.cc:336] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111)
I20260709 13:22:25.381748 17684 leader_election.cc:304] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:25.382292 18053 raft_consensus.cc:2749] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:25.415040 18053 raft_consensus.cc:493] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b7fc8565fb3b4e63b85f8f408089c384)
I20260709 13:22:25.415645 18053 raft_consensus.cc:515] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:25.419080 18053 leader_election.cc:290] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
W20260709 13:22:25.422191 17684 leader_election.cc:336] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111)
W20260709 13:22:25.423475 17682 leader_election.cc:336] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111)
I20260709 13:22:25.423851 17682 leader_election.cc:304] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:25.424495 18053 raft_consensus.cc:2749] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:25.476313  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
I20260709 13:22:25.507982 18053 raft_consensus.cc:493] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b7fc8565fb3b4e63b85f8f408089c384)
I20260709 13:22:25.508569 18053 raft_consensus.cc:515] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:25.510697  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
I20260709 13:22:25.510864 18053 leader_election.cc:290] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
W20260709 13:22:25.515480 17684 leader_election.cc:336] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111)
W20260709 13:22:25.517232 17682 leader_election.cc:336] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111)
I20260709 13:22:25.517719 17682 leader_election.cc:304] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:25.518525 18053 raft_consensus.cc:2749] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:25.533480 18053 raft_consensus.cc:493] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 06d01143e7944606834891750e8d5b26)
I20260709 13:22:25.534020 18053 raft_consensus.cc:515] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7fc8565fb3b4e63b85f8f408089c384" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 34037 } } peers { permanent_uuid: "06d01143e7944606834891750e8d5b26" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 44043 } } peers { permanent_uuid: "1c63da9d803242a9b6d8edc7c590b6cd" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 45027 } }
I20260709 13:22:25.537415 18053 leader_election.cc:290] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037), 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043)
W20260709 13:22:25.542927 17682 leader_election.cc:336] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 06d01143e7944606834891750e8d5b26 (127.5.72.130:44043): Network error: Client connection negotiation failed: client connection to 127.5.72.130:44043: connect: Connection refused (error 111)
I20260709 13:22:25.545080  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:22:25.545208 17684 leader_election.cc:336] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b7fc8565fb3b4e63b85f8f408089c384 (127.5.72.129:34037): Network error: Client connection negotiation failed: client connection to 127.5.72.129:34037: connect: Connection refused (error 111)
I20260709 13:22:25.545697 17684 leader_election.cc:304] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c63da9d803242a9b6d8edc7c590b6cd; no voters: 06d01143e7944606834891750e8d5b26, b7fc8565fb3b4e63b85f8f408089c384
I20260709 13:22:25.545924  5410 tablet_replica.cc:333] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.546681 18053 raft_consensus.cc:2749] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:25.546926  5410 raft_consensus.cc:2243] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:25.547917  5410 raft_consensus.cc:2272] T dd975ee6998e455eb4b1a63fbb69b713 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.549743  5410 tablet_replica.cc:333] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.550371  5410 raft_consensus.cc:2243] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:25.551146  5410 raft_consensus.cc:2272] T 849994c508bb421eb00d7d8aecb5e09f P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.552815  5410 tablet_replica.cc:333] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.553314  5410 raft_consensus.cc:2243] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.553697  5410 raft_consensus.cc:2272] T 82ad5ef915b64154b0e632093755fabe P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.555441  5410 tablet_replica.cc:333] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.555925  5410 raft_consensus.cc:2243] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.556283  5410 raft_consensus.cc:2272] T bb9d31ac10ca4ccf9ed455dc74dacb68 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.557914  5410 tablet_replica.cc:333] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.558406  5410 raft_consensus.cc:2243] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:25.559157  5410 raft_consensus.cc:2272] T ff85f4876cbf4cffa6daf36fa7c90d00 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.560890  5410 tablet_replica.cc:333] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.561411  5410 raft_consensus.cc:2243] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.561777  5410 raft_consensus.cc:2272] T 305ec350e9334ed4ad2a3b2abaae106a P 1c63da9d803242a9b6d8edc7c590b6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.563493  5410 tablet_replica.cc:333] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.563990  5410 raft_consensus.cc:2243] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.564344  5410 raft_consensus.cc:2272] T e22afaa8d14241c4aed5a24b4f5183dc P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.565976  5410 tablet_replica.cc:333] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.566454  5410 raft_consensus.cc:2243] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.566891  5410 raft_consensus.cc:2272] T 0226e7d22e9641768e53252a60d11a28 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.568548  5410 tablet_replica.cc:333] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.569020  5410 raft_consensus.cc:2243] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.569383  5410 raft_consensus.cc:2272] T 67c6843e80d24c7caec391c58dbf8a20 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.571066  5410 tablet_replica.cc:333] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.571704  5410 raft_consensus.cc:2243] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:25.572434  5410 raft_consensus.cc:2272] T f2340c19eff7470d81b8f44b3e19a87e P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.574270  5410 tablet_replica.cc:333] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.574795  5410 raft_consensus.cc:2243] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:25.575518  5410 raft_consensus.cc:2272] T 43e11d78cf48431e9a207fc2d5c7fae3 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.577481  5410 tablet_replica.cc:333] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.577939  5410 raft_consensus.cc:2243] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.578332  5410 raft_consensus.cc:2272] T 3d0c1612e14045e184c6df7f45a0819a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.580159  5410 tablet_replica.cc:333] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.580598  5410 raft_consensus.cc:2243] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.580969  5410 raft_consensus.cc:2272] T dc0466a5a6054a089de335deb2609daf P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.582787  5410 tablet_replica.cc:333] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.583237  5410 raft_consensus.cc:2243] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.583604  5410 raft_consensus.cc:2272] T 1d6517d1e6eb4aa6855d0e9dab1268d0 P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.585491  5410 tablet_replica.cc:333] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.585896  5410 raft_consensus.cc:2243] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.586241  5410 raft_consensus.cc:2272] T a6fcdc3cf8e343849cda6f25a6a918be P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.588042  5410 tablet_replica.cc:333] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.588454  5410 raft_consensus.cc:2243] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.588822  5410 raft_consensus.cc:2272] T b2226106f1aa42a5ad476a88e16b485e P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.590670  5410 tablet_replica.cc:333] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.591099  5410 raft_consensus.cc:2243] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:25.591876  5410 raft_consensus.cc:2272] T b2bfb78532214f52b1d57b1d55c26829 P 1c63da9d803242a9b6d8edc7c590b6cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.593712  5410 tablet_replica.cc:333] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd: stopping tablet replica
I20260709 13:22:25.594096  5410 raft_consensus.cc:2243] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:25.594455  5410 raft_consensus.cc:2272] T df281b5696ba48768bbe0dcc6d12911a P 1c63da9d803242a9b6d8edc7c590b6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.627663  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
I20260709 13:22:25.660683  5410 master.cc:562] Master@127.5.72.190:40487 shutting down...
I20260709 13:22:25.682042  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:25.682684  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:25.683053  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3134d8af1387460683b6fd7457c8a5f6: stopping tablet replica
I20260709 13:22:25.703552  5410 master.cc:584] Master@127.5.72.190:40487 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (14509 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:22:25.754902  5410 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.72.190:44229
I20260709 13:22:25.756111  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:25.761349 18056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:25.762573 18057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:25.763129 18059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:25.763387  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:25.764406  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:25.764587  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:25.764701  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603345764690 us; error 0 us; skew 500 ppm
I20260709 13:22:25.765136  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:25.767788  5410 webserver.cc:533] Webserver started at http://127.5.72.190:35623/ using document root <none> and password file <none>
I20260709 13:22:25.768225  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:25.768373  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:25.768572  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:25.769583  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/master-0-root/instance:
uuid: "3bacdc7796454bc687dcc4220430f64a"
format_stamp: "Formatted at 2026-07-09 13:22:25 on dist-test-slave-n5kz"
I20260709 13:22:25.773756  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:25.776911 18064 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:25.777626  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:22:25.777907  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/master-0-root
uuid: "3bacdc7796454bc687dcc4220430f64a"
format_stamp: "Formatted at 2026-07-09 13:22:25 on dist-test-slave-n5kz"
I20260709 13:22:25.778193  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:25.795058  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:25.796226  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:25.833204  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.190:44229
I20260709 13:22:25.833325 18115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.190:44229 every 8 connection(s)
I20260709 13:22:25.837028 18116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:25.848511 18116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a: Bootstrap starting.
I20260709 13:22:25.853226 18116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:25.857347 18116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a: No bootstrap required, opened a new log
I20260709 13:22:25.859544 18116 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bacdc7796454bc687dcc4220430f64a" member_type: VOTER }
I20260709 13:22:25.859911 18116 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:25.860088 18116 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bacdc7796454bc687dcc4220430f64a, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.860682 18116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bacdc7796454bc687dcc4220430f64a" member_type: VOTER }
I20260709 13:22:25.861109 18116 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:25.861301 18116 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:25.861529 18116 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.872438 18116 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bacdc7796454bc687dcc4220430f64a" member_type: VOTER }
I20260709 13:22:25.873446 18116 leader_election.cc:304] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bacdc7796454bc687dcc4220430f64a; no voters: 
I20260709 13:22:25.875233 18116 leader_election.cc:290] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:25.876226 18119 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:25.879050 18119 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 1 LEADER]: Becoming Leader. State: Replica: 3bacdc7796454bc687dcc4220430f64a, State: Running, Role: LEADER
I20260709 13:22:25.880124 18116 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:25.880587 18119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bacdc7796454bc687dcc4220430f64a" member_type: VOTER }
I20260709 13:22:25.887868 18120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bacdc7796454bc687dcc4220430f64a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bacdc7796454bc687dcc4220430f64a" member_type: VOTER } }
I20260709 13:22:25.887889 18121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bacdc7796454bc687dcc4220430f64a. Latest consensus state: current_term: 1 leader_uuid: "3bacdc7796454bc687dcc4220430f64a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bacdc7796454bc687dcc4220430f64a" member_type: VOTER } }
I20260709 13:22:25.888612 18120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:25.888641 18121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:25.891381 18126 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:25.899588 18126 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:25.927402  5410 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:25.938055 18126 catalog_manager.cc:1383] Generated new cluster ID: b2e338e067b54872abb414e3f24165f8
I20260709 13:22:25.938424 18126 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:25.961159 18126 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:25.962831 18126 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:25.978825 18126 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a: Generated new TSK 0
I20260709 13:22:25.979440 18126 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:25.994603  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:26.002146 18138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:26.003104 18139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:26.004634 18141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:26.005339  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:26.006328  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:26.006542  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:26.006763  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603346006743 us; error 0 us; skew 500 ppm
I20260709 13:22:26.007390  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:26.009987  5410 webserver.cc:533] Webserver started at http://127.5.72.129:44759/ using document root <none> and password file <none>
I20260709 13:22:26.010531  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:26.010782  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:26.011093  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:26.012509  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-0-root/instance:
uuid: "283925a3c2d24fb387b016ef4f9c5d69"
format_stamp: "Formatted at 2026-07-09 13:22:26 on dist-test-slave-n5kz"
I20260709 13:22:26.017264  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:22:26.020790 18146 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.021533  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:22:26.021809  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-0-root
uuid: "283925a3c2d24fb387b016ef4f9c5d69"
format_stamp: "Formatted at 2026-07-09 13:22:26 on dist-test-slave-n5kz"
I20260709 13:22:26.022109  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:26.048198  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:26.049584  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:26.051198  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:26.053639  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:26.053877  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.054126  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:26.054301  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.096318  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.129:39331
I20260709 13:22:26.096517 18208 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.129:39331 every 8 connection(s)
I20260709 13:22:26.102150  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:26.109850 18213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:26.110576 18214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:26.114506 18216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:26.115381  5410 server_base.cc:1061] running on GCE node
I20260709 13:22:26.116524 18209 heartbeater.cc:344] Connected to a master server at 127.5.72.190:44229
I20260709 13:22:26.116688  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:22:26.116953 18209 heartbeater.cc:461] Registering TS with master...
W20260709 13:22:26.117002  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:26.117329  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603346117312 us; error 0 us; skew 500 ppm
I20260709 13:22:26.117937  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:26.117882 18209 heartbeater.cc:507] Master 127.5.72.190:44229 requested a full tablet report, sending...
I20260709 13:22:26.120761  5410 webserver.cc:533] Webserver started at http://127.5.72.130:37797/ using document root <none> and password file <none>
I20260709 13:22:26.120707 18081 ts_manager.cc:194] Registered new tserver with Master: 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331)
I20260709 13:22:26.121515  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:26.121737  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:26.122011  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:26.123152 18081 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51182
I20260709 13:22:26.123476  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-1-root/instance:
uuid: "0dc67624917f441fbe92efac4c8d3934"
format_stamp: "Formatted at 2026-07-09 13:22:26 on dist-test-slave-n5kz"
I20260709 13:22:26.128111  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:26.131498 18221 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.132184  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:26.132475  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-1-root
uuid: "0dc67624917f441fbe92efac4c8d3934"
format_stamp: "Formatted at 2026-07-09 13:22:26 on dist-test-slave-n5kz"
I20260709 13:22:26.132764  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:26.147680  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:26.148836  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:26.150205  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:26.152455  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:26.152675  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.152912  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:26.153088  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.197336  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.130:40155
I20260709 13:22:26.197448 18283 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.130:40155 every 8 connection(s)
I20260709 13:22:26.202407  5410 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:26.209566 18287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:26.212107 18288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:26.214895 18284 heartbeater.cc:344] Connected to a master server at 127.5.72.190:44229
I20260709 13:22:26.215381 18284 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:26.216491  5410 server_base.cc:1061] running on GCE node
W20260709 13:22:26.216600 18290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:26.216650 18284 heartbeater.cc:507] Master 127.5.72.190:44229 requested a full tablet report, sending...
I20260709 13:22:26.217909  5410 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:26.218184  5410 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:26.218426  5410 hybrid_clock.cc:648] HybridClock initialized: now 1783603346218406 us; error 0 us; skew 500 ppm
I20260709 13:22:26.219087  5410 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:26.219149 18081 ts_manager.cc:194] Registered new tserver with Master: 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:26.220847 18081 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51198
I20260709 13:22:26.221715  5410 webserver.cc:533] Webserver started at http://127.5.72.131:40587/ using document root <none> and password file <none>
I20260709 13:22:26.222337  5410 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:26.222556  5410 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:26.222868  5410 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:26.224313  5410 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-2-root/instance:
uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
format_stamp: "Formatted at 2026-07-09 13:22:26 on dist-test-slave-n5kz"
I20260709 13:22:26.228837  5410 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:22:26.232249 18295 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.233111  5410 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:22:26.233457  5410 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-2-root
uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
format_stamp: "Formatted at 2026-07-09 13:22:26 on dist-test-slave-n5kz"
I20260709 13:22:26.233762  5410 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskn9FTie/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602945696884-5410-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:26.255386  5410 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:26.256589  5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:26.258072  5410 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:26.260421  5410 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:26.260672  5410 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.260929  5410 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:26.261101  5410 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.303454  5410 rpc_server.cc:307] RPC server started. Bound to: 127.5.72.131:38045
I20260709 13:22:26.303576 18357 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.72.131:38045 every 8 connection(s)
I20260709 13:22:26.317441 18358 heartbeater.cc:344] Connected to a master server at 127.5.72.190:44229
I20260709 13:22:26.317967 18358 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:26.318756 18358 heartbeater.cc:507] Master 127.5.72.190:44229 requested a full tablet report, sending...
I20260709 13:22:26.320670 18081 ts_manager.cc:194] Registered new tserver with Master: 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:26.321283  5410 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01444179s
I20260709 13:22:26.322113 18081 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51206
I20260709 13:22:26.347833 18081 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51222:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:26.350821 18081 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:26.427059 18249 tablet_service.cc:1511] Processing CreateTablet for tablet e32ac98de5674dacaef8332140cb0d2d (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:26.427278 18248 tablet_service.cc:1511] Processing CreateTablet for tablet b7ee78df24c040389e1bec48dcdfe6e0 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:26.429191 18249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e32ac98de5674dacaef8332140cb0d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.429962 18248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ee78df24c040389e1bec48dcdfe6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.439158 18323 tablet_service.cc:1511] Processing CreateTablet for tablet e32ac98de5674dacaef8332140cb0d2d (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:26.440759 18323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e32ac98de5674dacaef8332140cb0d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.439548 18322 tablet_service.cc:1511] Processing CreateTablet for tablet b7ee78df24c040389e1bec48dcdfe6e0 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:26.442332 18173 tablet_service.cc:1511] Processing CreateTablet for tablet b7ee78df24c040389e1bec48dcdfe6e0 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:26.443420 18322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ee78df24c040389e1bec48dcdfe6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.443931 18173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ee78df24c040389e1bec48dcdfe6e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.440759 18174 tablet_service.cc:1511] Processing CreateTablet for tablet e32ac98de5674dacaef8332140cb0d2d (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:26.447086 18174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e32ac98de5674dacaef8332140cb0d2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.452615 18172 tablet_service.cc:1511] Processing CreateTablet for tablet 4b1808a58b464d5d88dc40b33c33b464 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:26.454135 18172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b1808a58b464d5d88dc40b33c33b464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.454247 18247 tablet_service.cc:1511] Processing CreateTablet for tablet 4b1808a58b464d5d88dc40b33c33b464 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:26.455780 18247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b1808a58b464d5d88dc40b33c33b464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.460541 18321 tablet_service.cc:1511] Processing CreateTablet for tablet 4b1808a58b464d5d88dc40b33c33b464 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:26.462100 18321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b1808a58b464d5d88dc40b33c33b464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.469156 18171 tablet_service.cc:1511] Processing CreateTablet for tablet 3fbd032884b645ae846c6f3733dcc917 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:26.470698 18171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fbd032884b645ae846c6f3733dcc917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.470800 18246 tablet_service.cc:1511] Processing CreateTablet for tablet 3fbd032884b645ae846c6f3733dcc917 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:26.472280 18246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fbd032884b645ae846c6f3733dcc917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.482954 18320 tablet_service.cc:1511] Processing CreateTablet for tablet 3fbd032884b645ae846c6f3733dcc917 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:26.484789 18320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fbd032884b645ae846c6f3733dcc917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.492352 18170 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca3425970884557b2bd8c85a567d4bf (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:26.494179 18170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca3425970884557b2bd8c85a567d4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.495672 18245 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca3425970884557b2bd8c85a567d4bf (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:26.497155 18245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca3425970884557b2bd8c85a567d4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.504565 18319 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca3425970884557b2bd8c85a567d4bf (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:26.505988 18319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca3425970884557b2bd8c85a567d4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.510080 18169 tablet_service.cc:1511] Processing CreateTablet for tablet 854a26ffa5a447e48a8c9778e299eda7 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:26.511487 18169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854a26ffa5a447e48a8c9778e299eda7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.511955 18244 tablet_service.cc:1511] Processing CreateTablet for tablet 854a26ffa5a447e48a8c9778e299eda7 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:26.513823 18244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854a26ffa5a447e48a8c9778e299eda7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.515398 18168 tablet_service.cc:1511] Processing CreateTablet for tablet d4697fc0d4ce422c8129cb8d07900a5e (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:26.516947 18168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4697fc0d4ce422c8129cb8d07900a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.517077 18243 tablet_service.cc:1511] Processing CreateTablet for tablet d4697fc0d4ce422c8129cb8d07900a5e (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:26.518676 18243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4697fc0d4ce422c8129cb8d07900a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.564260 18323 tablet_service.cc:1511] Processing CreateTablet for tablet 854a26ffa5a447e48a8c9778e299eda7 (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:26.565660 18323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 854a26ffa5a447e48a8c9778e299eda7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.569761 18322 tablet_service.cc:1511] Processing CreateTablet for tablet d4697fc0d4ce422c8129cb8d07900a5e (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:26.571157 18322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4697fc0d4ce422c8129cb8d07900a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.575240 18374 tablet_bootstrap.cc:492] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:26.594655 18168 tablet_service.cc:1511] Processing CreateTablet for tablet 6d713a8451ee46c2be2578e9a0e3fcef (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:26.596078 18168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d713a8451ee46c2be2578e9a0e3fcef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.599983 18374 tablet_bootstrap.cc:654] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.616390 18243 tablet_service.cc:1511] Processing CreateTablet for tablet 6d713a8451ee46c2be2578e9a0e3fcef (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:26.618963 18243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d713a8451ee46c2be2578e9a0e3fcef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.624226 18373 tablet_bootstrap.cc:492] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:26.626504 18173 tablet_service.cc:1511] Processing CreateTablet for tablet 219dd503f7bd49aab20bc06b6c4779ff (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:26.629721 18173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219dd503f7bd49aab20bc06b6c4779ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.633539 18373 tablet_bootstrap.cc:654] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.653098 18374 tablet_bootstrap.cc:492] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:26.653635 18374 ts_tablet_manager.cc:1403] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.079s	user 0.010s	sys 0.004s
I20260709 13:22:26.656762 18374 raft_consensus.cc:359] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.656991 18248 tablet_service.cc:1511] Processing CreateTablet for tablet 219dd503f7bd49aab20bc06b6c4779ff (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:26.657758 18374 raft_consensus.cc:385] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.658090 18374 raft_consensus.cc:740] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.658460 18248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219dd503f7bd49aab20bc06b6c4779ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.658357 18320 tablet_service.cc:1511] Processing CreateTablet for tablet 219dd503f7bd49aab20bc06b6c4779ff (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:26.659504 18320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219dd503f7bd49aab20bc06b6c4779ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.659147 18374 consensus_queue.cc:260] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.671128 18319 tablet_service.cc:1511] Processing CreateTablet for tablet 6d713a8451ee46c2be2578e9a0e3fcef (DEFAULT_TABLE table=test-workload [id=d83334d4e6fa4807850cd1560caea141]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:26.672384 18373 tablet_bootstrap.cc:492] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:26.673036 18319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d713a8451ee46c2be2578e9a0e3fcef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:26.674070 18373 ts_tablet_manager.cc:1403] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.013s
I20260709 13:22:26.677171 18373 raft_consensus.cc:359] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.677749 18373 raft_consensus.cc:385] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.678009 18373 raft_consensus.cc:740] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.678762 18373 consensus_queue.cc:260] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.681222 18284 heartbeater.cc:499] Master 127.5.72.190:44229 was elected leader, sending a full tablet report...
I20260709 13:22:26.681911 18373 ts_tablet_manager.cc:1434] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:22:26.683094 18373 tablet_bootstrap.cc:492] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:26.683856 18377 tablet_bootstrap.cc:492] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:26.687798 18358 heartbeater.cc:499] Master 127.5.72.190:44229 was elected leader, sending a full tablet report...
I20260709 13:22:26.688359 18374 ts_tablet_manager.cc:1434] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.014s
I20260709 13:22:26.689426 18374 tablet_bootstrap.cc:492] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:26.690176 18373 tablet_bootstrap.cc:654] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.696089 18374 tablet_bootstrap.cc:654] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.728003 18377 tablet_bootstrap.cc:654] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.746438 18373 tablet_bootstrap.cc:492] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:26.747000 18373 ts_tablet_manager.cc:1403] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.064s	user 0.034s	sys 0.030s
I20260709 13:22:26.748903 18377 tablet_bootstrap.cc:492] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:26.749471 18377 ts_tablet_manager.cc:1403] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.004s
I20260709 13:22:26.749541 18373 raft_consensus.cc:359] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:26.750406 18373 raft_consensus.cc:385] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.750726 18373 raft_consensus.cc:740] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.751299 18374 tablet_bootstrap.cc:492] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:26.751888 18374 ts_tablet_manager.cc:1403] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.063s	user 0.026s	sys 0.024s
I20260709 13:22:26.751508 18373 consensus_queue.cc:260] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:26.752245 18377 raft_consensus.cc:359] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:26.752986 18377 raft_consensus.cc:385] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.753262 18377 raft_consensus.cc:740] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.753805 18373 ts_tablet_manager.cc:1434] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:22:26.754107 18377 consensus_queue.cc:260] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:26.754870 18373 tablet_bootstrap.cc:492] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:26.754966 18374 raft_consensus.cc:359] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.755756 18374 raft_consensus.cc:385] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.756033 18374 raft_consensus.cc:740] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.756922 18374 consensus_queue.cc:260] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.759191 18374 ts_tablet_manager.cc:1434] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:26.760227 18374 tablet_bootstrap.cc:492] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:26.762427 18373 tablet_bootstrap.cc:654] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.765803 18377 ts_tablet_manager.cc:1434] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.007s
I20260709 13:22:26.766842 18209 heartbeater.cc:499] Master 127.5.72.190:44229 was elected leader, sending a full tablet report...
I20260709 13:22:26.766981 18377 tablet_bootstrap.cc:492] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:26.767532 18374 tablet_bootstrap.cc:654] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.774260 18377 tablet_bootstrap.cc:654] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.806340 18373 tablet_bootstrap.cc:492] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:26.808671 18373 ts_tablet_manager.cc:1403] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.054s	user 0.025s	sys 0.026s
I20260709 13:22:26.809473 18374 tablet_bootstrap.cc:492] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:26.809908 18377 tablet_bootstrap.cc:492] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:26.810076 18374 ts_tablet_manager.cc:1403] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.024s
I20260709 13:22:26.810559 18377 ts_tablet_manager.cc:1403] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.044s	user 0.026s	sys 0.016s
I20260709 13:22:26.811668 18373 raft_consensus.cc:359] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.812484 18373 raft_consensus.cc:385] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.812763 18373 raft_consensus.cc:740] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.813216 18374 raft_consensus.cc:359] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:26.813524 18373 consensus_queue.cc:260] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.814003 18374 raft_consensus.cc:385] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.813828 18377 raft_consensus.cc:359] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.814399 18374 raft_consensus.cc:740] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.814698 18377 raft_consensus.cc:385] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.815012 18377 raft_consensus.cc:740] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.815254 18374 consensus_queue.cc:260] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:26.816005 18373 ts_tablet_manager.cc:1434] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:26.815830 18377 consensus_queue.cc:260] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.817451 18373 tablet_bootstrap.cc:492] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:26.819047 18377 ts_tablet_manager.cc:1434] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:22:26.820086 18374 ts_tablet_manager.cc:1434] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:22:26.820103 18377 tablet_bootstrap.cc:492] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:26.821341 18374 tablet_bootstrap.cc:492] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:26.824604 18373 tablet_bootstrap.cc:654] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.827265 18377 tablet_bootstrap.cc:654] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.828807 18374 tablet_bootstrap.cc:654] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.844918 18377 tablet_bootstrap.cc:492] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:26.845643 18377 ts_tablet_manager.cc:1403] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.008s
I20260709 13:22:26.849758 18377 raft_consensus.cc:359] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.850795 18377 raft_consensus.cc:385] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.853159 18383 raft_consensus.cc:493] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.854003 18383 raft_consensus.cc:515] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.864226 18383 leader_election.cc:290] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:26.864311 18377 raft_consensus.cc:740] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.865904 18377 consensus_queue.cc:260] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.871037 18377 ts_tablet_manager.cc:1434] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.003s
I20260709 13:22:26.872058 18377 tablet_bootstrap.cc:492] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:26.872134 18374 tablet_bootstrap.cc:492] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:26.872745 18374 ts_tablet_manager.cc:1403] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.031s
I20260709 13:22:26.872800 18373 tablet_bootstrap.cc:492] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:26.873401 18373 ts_tablet_manager.cc:1403] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.056s	user 0.028s	sys 0.024s
I20260709 13:22:26.875823 18374 raft_consensus.cc:359] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.876632 18374 raft_consensus.cc:385] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.876390 18373 raft_consensus.cc:359] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.876995 18374 raft_consensus.cc:740] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.877233 18373 raft_consensus.cc:385] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.877599 18373 raft_consensus.cc:740] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.877857 18374 consensus_queue.cc:260] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.879395 18377 tablet_bootstrap.cc:654] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.883214 18374 ts_tablet_manager.cc:1434] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:26.884348 18374 tablet_bootstrap.cc:492] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:26.890293 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
W20260709 13:22:26.892814 18149 leader_election.cc:343] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:26.894481 18129 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:26.896476 18373 consensus_queue.cc:260] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.897370 18374 tablet_bootstrap.cc:654] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.898927 18373 ts_tablet_manager.cc:1434] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.001s
I20260709 13:22:26.900000 18373 tablet_bootstrap.cc:492] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:26.900668 18377 tablet_bootstrap.cc:492] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:26.901201 18377 ts_tablet_manager.cc:1403] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.012s
I20260709 13:22:26.902856 18378 raft_consensus.cc:493] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.903371 18378 raft_consensus.cc:515] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.904407 18377 raft_consensus.cc:359] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.905202 18377 raft_consensus.cc:385] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.905490 18377 raft_consensus.cc:740] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.906672 18373 tablet_bootstrap.cc:654] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.910337 18378 leader_election.cc:290] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:26.914232 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:26.915303 18259 raft_consensus.cc:2468] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:26.917217 18148 leader_election.cc:304] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:26.918833 18383 raft_consensus.cc:2804] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:26.919320 18383 raft_consensus.cc:493] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:26.919773 18383 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:26.920583 18374 tablet_bootstrap.cc:492] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:26.921164 18374 ts_tablet_manager.cc:1403] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.000s
I20260709 13:22:26.924538 18374 raft_consensus.cc:359] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.925311 18374 raft_consensus.cc:385] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.925617 18374 raft_consensus.cc:740] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.926105 18378 raft_consensus.cc:493] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.926687 18378 raft_consensus.cc:515] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.929162 18378 leader_election.cc:290] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:26.926602 18374 consensus_queue.cc:260] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.931440 18383 raft_consensus.cc:515] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.906251 18377 consensus_queue.cc:260] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.934346 18374 ts_tablet_manager.cc:1434] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:22:26.937628 18374 tablet_bootstrap.cc:492] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:26.939735 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:26.940639 18259 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:26.945456 18374 tablet_bootstrap.cc:654] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.952099 18259 raft_consensus.cc:2468] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
W20260709 13:22:26.952518 18285 tablet.cc:2406] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:22:26.953496 18148 leader_election.cc:304] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:26.955608 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:26.956354 18183 raft_consensus.cc:2468] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:26.956460 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:26.958962 18297 leader_election.cc:304] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:26.960124 18378 raft_consensus.cc:2804] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:26.960608 18378 raft_consensus.cc:493] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:26.961001 18378 raft_consensus.cc:3060] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:26.965355 18374 tablet_bootstrap.cc:492] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:26.965694 18397 raft_consensus.cc:2804] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:26.965996 18374 ts_tablet_manager.cc:1403] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.003s
I20260709 13:22:26.968796 18377 ts_tablet_manager.cc:1434] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.067s	user 0.013s	sys 0.001s
I20260709 13:22:26.969473 18374 raft_consensus.cc:359] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.970230 18374 raft_consensus.cc:385] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:26.970525 18374 raft_consensus.cc:740] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
W20260709 13:22:26.970952 18297 leader_election.cc:343] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:26.975338 18383 leader_election.cc:290] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:26.971452 18374 consensus_queue.cc:260] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.973220 18377 tablet_bootstrap.cc:492] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:26.972962 18378 raft_consensus.cc:515] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.979724 18378 leader_election.cc:290] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:26.984303 18377 tablet_bootstrap.cc:654] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:26.985426 18258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:26.986173 18258 raft_consensus.cc:2468] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:26.987134 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:26.988159 18257 raft_consensus.cc:3060] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:26.992159 18374 ts_tablet_manager.cc:1434] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.006s
I20260709 13:22:26.993378 18374 tablet_bootstrap.cc:492] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:26.979813 18397 raft_consensus.cc:697] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:26.979725 18373 tablet_bootstrap.cc:492] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:26.982218 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:26.999457 18373 ts_tablet_manager.cc:1403] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.100s	user 0.032s	sys 0.045s
I20260709 13:22:26.999528 18184 raft_consensus.cc:3060] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.002533 18374 tablet_bootstrap.cc:654] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.003073 18373 raft_consensus.cc:359] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.003988 18373 raft_consensus.cc:385] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.004331 18373 raft_consensus.cc:740] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.005204 18373 consensus_queue.cc:260] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:26.982800 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:27.008296 18373 ts_tablet_manager.cc:1434] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
W20260709 13:22:27.008399 18149 leader_election.cc:343] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.009224 18401 raft_consensus.cc:493] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.009833 18401 raft_consensus.cc:515] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.013001 18401 leader_election.cc:290] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.011009 18373 tablet_bootstrap.cc:492] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:27.014506 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.016551 18258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
W20260709 13:22:27.016884 18297 leader_election.cc:343] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.017284 18258 raft_consensus.cc:2468] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:27.017697 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.019976 18397 consensus_queue.cc:237] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.022810 18297 leader_election.cc:304] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 283925a3c2d24fb387b016ef4f9c5d69
W20260709 13:22:27.024366 18297 leader_election.cc:343] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.024667 18373 tablet_bootstrap.cc:654] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.024844 18297 leader_election.cc:304] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:27.025736 18378 raft_consensus.cc:2804] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.026075 18401 raft_consensus.cc:2749] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:27.026124 18378 raft_consensus.cc:493] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.026748 18378 raft_consensus.cc:3060] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.028149 18184 raft_consensus.cc:2468] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:27.029903 18297 leader_election.cc:304] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:27.030982 18401 raft_consensus.cc:2804] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.033324 18257 raft_consensus.cc:2468] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:27.040310 18378 raft_consensus.cc:515] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.041862 18377 tablet_bootstrap.cc:492] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:27.042552 18377 ts_tablet_manager.cc:1403] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.070s	user 0.031s	sys 0.031s
I20260709 13:22:27.044409 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:27.045054 18378 leader_election.cc:290] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
W20260709 13:22:27.045683 18297 leader_election.cc:343] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.045248 18374 tablet_bootstrap.cc:492] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:27.046725 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:27.047134 18374 ts_tablet_manager.cc:1403] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.003s
I20260709 13:22:27.047367 18257 raft_consensus.cc:3060] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.049137 18080 catalog_manager.cc:5697] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:27.054030 18401 raft_consensus.cc:697] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:27.055115 18401 consensus_queue.cc:237] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.056887 18373 tablet_bootstrap.cc:492] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:27.057147 18257 raft_consensus.cc:2468] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:27.057495 18373 ts_tablet_manager.cc:1403] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.009s
I20260709 13:22:27.058897 18297 leader_election.cc:304] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:27.060055 18404 raft_consensus.cc:2804] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.060778 18404 raft_consensus.cc:697] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:27.061678 18404 consensus_queue.cc:237] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.064669 18374 raft_consensus.cc:359] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.064898 18377 raft_consensus.cc:359] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.065541 18374 raft_consensus.cc:385] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.065752 18377 raft_consensus.cc:385] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.065963 18374 raft_consensus.cc:740] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.066116 18377 raft_consensus.cc:740] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.067236 18373 raft_consensus.cc:359] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.067080 18377 consensus_queue.cc:260] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.067895 18373 raft_consensus.cc:385] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.068221 18373 raft_consensus.cc:740] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.068426 18374 consensus_queue.cc:260] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.069073 18373 consensus_queue.cc:260] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.070374 18377 ts_tablet_manager.cc:1434] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:22:27.071655 18377 tablet_bootstrap.cc:492] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:27.071689 18374 ts_tablet_manager.cc:1434] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 13:22:27.073225 18374 tablet_bootstrap.cc:492] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:27.075220 18373 ts_tablet_manager.cc:1434] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.002s
I20260709 13:22:27.076347 18373 tablet_bootstrap.cc:492] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:27.077764 18378 raft_consensus.cc:493] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.078356 18378 raft_consensus.cc:515] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.079497 18377 tablet_bootstrap.cc:654] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.082352 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.083284 18373 tablet_bootstrap.cc:654] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.085968 18378 leader_election.cc:290] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.086840 18378 raft_consensus.cc:493] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.087122 18374 tablet_bootstrap.cc:654] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.086884 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.087469 18378 raft_consensus.cc:515] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.088167 18257 raft_consensus.cc:2393] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b41bb7f2b7847c19b86e54214bf0f4e in current term 1: Already voted for candidate 283925a3c2d24fb387b016ef4f9c5d69 in this term.
I20260709 13:22:27.090166 18397 raft_consensus.cc:493] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.092047 18397 raft_consensus.cc:515] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.090822 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.096874 18184 raft_consensus.cc:2468] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:27.096448 18401 raft_consensus.cc:493] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.090287 18297 leader_election.cc:304] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:27.099330 18401 raft_consensus.cc:515] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.097822 18397 leader_election.cc:290] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.101554 18401 leader_election.cc:290] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.101639 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.102079 18401 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.089807 18378 leader_election.cc:290] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.102319 18297 leader_election.cc:304] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:27.102389 18259 raft_consensus.cc:2468] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:27.103369 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:27.104053 18333 raft_consensus.cc:2468] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:27.104678 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.105170 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.105271 18257 raft_consensus.cc:2468] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:27.105404 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.105993 18184 raft_consensus.cc:2468] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:27.106225 18259 raft_consensus.cc:2468] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:27.107147 18378 raft_consensus.cc:2804] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.107561 18148 leader_election.cc:304] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:27.108209 18297 leader_election.cc:304] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:27.111104 18378 raft_consensus.cc:493] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.111725 18378 raft_consensus.cc:3060] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.119613 18397 raft_consensus.cc:2804] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.123378 18397 raft_consensus.cc:493] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.123865 18397 raft_consensus.cc:3060] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.131166 18401 raft_consensus.cc:2749] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:27.134750 18374 tablet_bootstrap.cc:492] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:27.135272 18374 ts_tablet_manager.cc:1403] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.008s
I20260709 13:22:27.137707 18397 raft_consensus.cc:515] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.135914 18401 raft_consensus.cc:2804] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.140048 18401 raft_consensus.cc:493] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.140398 18401 raft_consensus.cc:3060] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.141469 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:27.142082 18259 raft_consensus.cc:3060] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.142270 18397 leader_election.cc:290] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.142297 18374 raft_consensus.cc:359] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.143555 18374 raft_consensus.cc:385] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.143816 18374 raft_consensus.cc:740] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.144536 18374 consensus_queue.cc:260] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.146535 18374 ts_tablet_manager.cc:1434] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.001s
I20260709 13:22:27.147137 18378 raft_consensus.cc:515] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.147423 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:27.148309 18333 raft_consensus.cc:2393] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 283925a3c2d24fb387b016ef4f9c5d69 in current term 1: Already voted for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in this term.
I20260709 13:22:27.149192 18259 raft_consensus.cc:2468] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
I20260709 13:22:27.149325 18378 leader_election.cc:290] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.149433 18373 tablet_bootstrap.cc:492] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:27.147492 18374 tablet_bootstrap.cc:492] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:27.150584 18373 ts_tablet_manager.cc:1403] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.074s	user 0.009s	sys 0.007s
I20260709 13:22:27.150532 18148 leader_election.cc:304] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:27.150230 18401 raft_consensus.cc:515] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.153849 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:27.154348 18401 leader_election.cc:290] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.154881 18373 raft_consensus.cc:359] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.155478 18373 raft_consensus.cc:385] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.155718 18373 raft_consensus.cc:740] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.155717 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:27.156469 18183 raft_consensus.cc:3060] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.156880 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:27.156613 18373 consensus_queue.cc:260] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.157480 18257 raft_consensus.cc:3060] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.158402 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:27.159287 18259 raft_consensus.cc:2393] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b41bb7f2b7847c19b86e54214bf0f4e in current term 1: Already voted for candidate 283925a3c2d24fb387b016ef4f9c5d69 in this term.
I20260709 13:22:27.162207 18397 raft_consensus.cc:2804] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.163007 18397 raft_consensus.cc:697] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:27.163998 18397 consensus_queue.cc:237] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.165293 18373 ts_tablet_manager.cc:1434] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:22:27.166391 18373 tablet_bootstrap.cc:492] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:27.169217 18374 tablet_bootstrap.cc:654] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.169662 18078 catalog_manager.cc:5697] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 1 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.172950 18373 tablet_bootstrap.cc:654] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.185853 18400 raft_consensus.cc:493] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.186609 18400 raft_consensus.cc:515] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:27.188122 18377 tablet_bootstrap.cc:492] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:27.188998 18377 ts_tablet_manager.cc:1403] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.118s	user 0.033s	sys 0.022s
I20260709 13:22:27.189555 18400 leader_election.cc:290] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.194020 18377 raft_consensus.cc:359] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.194975 18377 raft_consensus.cc:385] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.195317 18377 raft_consensus.cc:740] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.196107 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e32ac98de5674dacaef8332140cb0d2d" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.197062 18259 raft_consensus.cc:2468] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:27.198490 18257 raft_consensus.cc:2468] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:27.199028 18148 leader_election.cc:304] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:27.200201 18377 consensus_queue.cc:260] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.200974 18400 raft_consensus.cc:2804] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.201689 18400 raft_consensus.cc:493] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.204180 18400 raft_consensus.cc:3060] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.210389 18374 tablet_bootstrap.cc:492] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:27.210417 18377 ts_tablet_manager.cc:1434] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:22:27.211050 18374 ts_tablet_manager.cc:1403] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.064s	user 0.019s	sys 0.009s
I20260709 13:22:27.211709 18377 tablet_bootstrap.cc:492] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:27.211454 18297 leader_election.cc:304] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:27.217738 18377 tablet_bootstrap.cc:654] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.218026 18378 raft_consensus.cc:493] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.218566 18378 raft_consensus.cc:515] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.219926 18183 raft_consensus.cc:2468] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:27.220906 18378 leader_election.cc:290] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.221699 18378 raft_consensus.cc:2749] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:27.214102 18374 raft_consensus.cc:359] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.223748 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e32ac98de5674dacaef8332140cb0d2d" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:27.223846 18374 raft_consensus.cc:385] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.224282 18374 raft_consensus.cc:740] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.224416 18333 raft_consensus.cc:2468] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:27.225334 18297 leader_election.cc:304] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:27.227833 18184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219dd503f7bd49aab20bc06b6c4779ff" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.228688 18184 raft_consensus.cc:2468] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:27.228911 18078 catalog_manager.cc:5697] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:27.230244 18401 raft_consensus.cc:2804] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.234959 18401 raft_consensus.cc:697] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:27.236183 18401 consensus_queue.cc:237] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.225029 18374 consensus_queue.cc:260] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.243024 18374 ts_tablet_manager.cc:1434] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:22:27.243036 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219dd503f7bd49aab20bc06b6c4779ff" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.244225 18297 leader_election.cc:304] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:27.245205 18378 raft_consensus.cc:2804] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.245731 18378 raft_consensus.cc:493] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.246171 18378 raft_consensus.cc:3060] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.215987 18081 catalog_manager.cc:5697] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 1 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:27.255137 18297 leader_election.cc:343] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.261430 18400 raft_consensus.cc:515] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:27.266718 18378 raft_consensus.cc:515] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.271485 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219dd503f7bd49aab20bc06b6c4779ff" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:27.272202 18183 raft_consensus.cc:3060] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.273118 18373 tablet_bootstrap.cc:492] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:27.273734 18373 ts_tablet_manager.cc:1403] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.108s	user 0.036s	sys 0.069s
I20260709 13:22:27.273576 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219dd503f7bd49aab20bc06b6c4779ff" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
W20260709 13:22:27.275388 18297 leader_election.cc:343] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.275473 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e32ac98de5674dacaef8332140cb0d2d" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:27.276221 18333 raft_consensus.cc:3060] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.276314 18400 leader_election.cc:290] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.276219 18378 leader_election.cc:290] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:27.277318 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e32ac98de5674dacaef8332140cb0d2d" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:27.277393 18373 raft_consensus.cc:359] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.278177 18257 raft_consensus.cc:3060] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.278294 18373 raft_consensus.cc:385] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.278831 18373 raft_consensus.cc:740] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.279702 18373 consensus_queue.cc:260] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.282416 18373 ts_tablet_manager.cc:1434] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:22:27.283958 18373 tablet_bootstrap.cc:492] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:27.292843 18373 tablet_bootstrap.cc:654] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.289703 18257 raft_consensus.cc:2468] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
I20260709 13:22:27.322388 18080 catalog_manager.cc:5697] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 1 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.330464 18148 leader_election.cc:304] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:27.331120 18333 raft_consensus.cc:2468] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
I20260709 13:22:27.331610 18400 raft_consensus.cc:2804] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.332211 18400 raft_consensus.cc:697] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:27.333077 18400 consensus_queue.cc:237] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } }
I20260709 13:22:27.336007 18377 tablet_bootstrap.cc:492] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:27.336696 18377 ts_tablet_manager.cc:1403] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.125s	user 0.035s	sys 0.069s
I20260709 13:22:27.341218 18377 raft_consensus.cc:359] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.342247 18377 raft_consensus.cc:385] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.342839 18377 raft_consensus.cc:740] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.343909 18377 consensus_queue.cc:260] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.347334 18377 ts_tablet_manager.cc:1434] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 13:22:27.348757 18377 tablet_bootstrap.cc:492] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:27.351392 18183 raft_consensus.cc:2468] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:27.352723 18297 leader_election.cc:304] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 0dc67624917f441fbe92efac4c8d3934
I20260709 13:22:27.354039 18378 raft_consensus.cc:2804] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.354522 18378 raft_consensus.cc:697] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:27.355427 18373 tablet_bootstrap.cc:492] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:27.355309 18378 consensus_queue.cc:237] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.356000 18373 ts_tablet_manager.cc:1403] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.072s	user 0.037s	sys 0.029s
I20260709 13:22:27.356719 18377 tablet_bootstrap.cc:654] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.363315 18081 catalog_manager.cc:5697] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.364915 18377 tablet_bootstrap.cc:492] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:27.365420 18377 ts_tablet_manager.cc:1403] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:22:27.367720 18377 raft_consensus.cc:359] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.368242 18377 raft_consensus.cc:385] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.368462 18377 raft_consensus.cc:740] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.369221 18377 consensus_queue.cc:260] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.369133 18373 raft_consensus.cc:359] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.370289 18373 raft_consensus.cc:385] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.369125 18080 catalog_manager.cc:5697] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 1 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.370591 18373 raft_consensus.cc:740] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.371289 18373 consensus_queue.cc:260] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.373445 18373 ts_tablet_manager.cc:1434] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.002s
I20260709 13:22:27.371686 18377 ts_tablet_manager.cc:1434] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:22:27.374792 18377 tablet_bootstrap.cc:492] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:27.380353 18377 tablet_bootstrap.cc:654] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:27.385229 18377 tablet_bootstrap.cc:492] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:27.385789 18377 ts_tablet_manager.cc:1403] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:22:27.388247 18377 raft_consensus.cc:359] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.388720 18377 raft_consensus.cc:385] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:27.388887 18377 raft_consensus.cc:740] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.389338 18377 consensus_queue.cc:260] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.391153 18377 ts_tablet_manager.cc:1434] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.001s
I20260709 13:22:27.399672 18379 raft_consensus.cc:493] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.400300 18379 raft_consensus.cc:515] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.403993 18379 leader_election.cc:290] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.414000 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d713a8451ee46c2be2578e9a0e3fcef" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.414729 18183 raft_consensus.cc:2468] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:27.415515 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d713a8451ee46c2be2578e9a0e3fcef" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:27.415819 18223 leader_election.cc:304] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:27.416111 18333 raft_consensus.cc:2468] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:27.416676 18379 raft_consensus.cc:2804] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.417038 18379 raft_consensus.cc:493] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.417366 18379 raft_consensus.cc:3060] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.422556 18379 raft_consensus.cc:515] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.424381 18379 leader_election.cc:290] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.425161 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d713a8451ee46c2be2578e9a0e3fcef" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:27.425396 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d713a8451ee46c2be2578e9a0e3fcef" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:27.425769 18183 raft_consensus.cc:3060] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.426111 18333 raft_consensus.cc:3060] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.430828 18183 raft_consensus.cc:2468] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:27.431370 18333 raft_consensus.cc:2468] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:27.431910 18223 leader_election.cc:304] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:27.432674 18379 raft_consensus.cc:2804] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.433544 18379 raft_consensus.cc:697] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:27.434923 18379 consensus_queue.cc:237] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.442759 18080 catalog_manager.cc:5697] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:27.474267 18400 raft_consensus.cc:493] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.474900 18400 raft_consensus.cc:515] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.476553 18400 leader_election.cc:290] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.477501 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:27.477645 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:27.478325 18257 raft_consensus.cc:2393] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 283925a3c2d24fb387b016ef4f9c5d69 in current term 1: Already voted for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in this term.
I20260709 13:22:27.479344 18148 leader_election.cc:304] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69; no voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:27.480005 18400 raft_consensus.cc:3060] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.485146 18400 raft_consensus.cc:2749] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:27.494335 18400 consensus_queue.cc:1048] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.498895 18378 consensus_queue.cc:1048] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.517051 18397 consensus_queue.cc:1048] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.519421 18401 consensus_queue.cc:1048] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.540112 18379 raft_consensus.cc:493] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.540665 18379 raft_consensus.cc:515] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.542840 18379 leader_election.cc:290] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.543402 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.544124 18183 raft_consensus.cc:2468] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:27.544328 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:27.545014 18333 raft_consensus.cc:2468] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:27.545465 18223 leader_election.cc:304] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:27.546319 18379 raft_consensus.cc:2804] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:27.546702 18379 raft_consensus.cc:493] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.547035 18379 raft_consensus.cc:3060] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.552675 18379 raft_consensus.cc:515] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.554368 18379 leader_election.cc:290] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.555076 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:27.555433 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:27.555751 18183 raft_consensus.cc:3060] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.556039 18333 raft_consensus.cc:3060] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.563095 18333 raft_consensus.cc:2468] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:27.563095 18183 raft_consensus.cc:2468] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:27.564534 18224 leader_election.cc:304] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:27.565191 18379 raft_consensus.cc:2804] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:27.565620 18379 raft_consensus.cc:697] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:27.566334 18379 consensus_queue.cc:237] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.573907 18080 catalog_manager.cc:5697] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:27.589080 18379 raft_consensus.cc:493] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.589601 18379 raft_consensus.cc:515] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.591437 18379 leader_election.cc:290] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.592226 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219dd503f7bd49aab20bc06b6c4779ff" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:27.592464 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "219dd503f7bd49aab20bc06b6c4779ff" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:27.593027 18183 raft_consensus.cc:2393] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0dc67624917f441fbe92efac4c8d3934 in current term 1: Already voted for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in this term.
I20260709 13:22:27.594023 18223 leader_election.cc:304] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934; no voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:27.594673 18379 raft_consensus.cc:3060] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:27.599565 18379 raft_consensus.cc:2749] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:27.632846 18400 consensus_queue.cc:1048] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.644874 18400 consensus_queue.cc:1048] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.671103 18378 consensus_queue.cc:1048] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.681457 18378 consensus_queue.cc:1048] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.754073 18400 consensus_queue.cc:1048] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.764226 18383 consensus_queue.cc:1048] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.829746 18378 consensus_queue.cc:1048] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.835573 18378 consensus_queue.cc:1048] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:27.852046 18257 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464"
dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
mode: GRACEFUL
new_leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:36670
I20260709 13:22:27.852756 18257 raft_consensus.cc:606] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Received request to transfer leadership to TS 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:27.858038 18450 consensus_queue.cc:1048] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.872474 18333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917"
dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
mode: GRACEFUL
new_leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:59280
I20260709 13:22:27.873106 18333 raft_consensus.cc:606] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Received request to transfer leadership to TS 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:27.876145 18450 consensus_queue.cc:1048] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.879281 18378 raft_consensus.cc:993] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e: : Instructing follower 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:27.879724 18410 raft_consensus.cc:1081] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Signalling peer 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:27.882608 18182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917"
dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:33626
I20260709 13:22:27.883332 18182 raft_consensus.cc:493] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:27.883704 18182 raft_consensus.cc:3060] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.888201 18333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf"
dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
mode: GRACEFUL
new_leader_uuid: "0dc67624917f441fbe92efac4c8d3934"
 from {username='slave'} at 127.0.0.1:59280
I20260709 13:22:27.888860 18333 raft_consensus.cc:606] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Received request to transfer leadership to TS 0dc67624917f441fbe92efac4c8d3934
I20260709 13:22:27.892529 18182 raft_consensus.cc:515] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.898118 18333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7"
dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
mode: GRACEFUL
new_leader_uuid: "0dc67624917f441fbe92efac4c8d3934"
 from {username='slave'} at 127.0.0.1:59280
I20260709 13:22:27.898680 18183 raft_consensus.cc:1240] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Rejecting Update request from peer 5b41bb7f2b7847c19b86e54214bf0f4e for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:27.898876 18333 raft_consensus.cc:606] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Received request to transfer leadership to TS 0dc67624917f441fbe92efac4c8d3934
I20260709 13:22:27.903191 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:27.903863 18257 raft_consensus.cc:3060] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.904950 18410 consensus_queue.cc:1059] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.906163 18378 raft_consensus.cc:3055] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:27.906523 18378 raft_consensus.cc:740] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:27.907377 18378 consensus_queue.cc:260] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.911998 18378 raft_consensus.cc:3060] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.912433 18182 leader_election.cc:290] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:27.914647 18257 raft_consensus.cc:2468] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:27.915967 18148 leader_election.cc:304] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:27.916508 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fbd032884b645ae846c6f3733dcc917" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:27.916893 18383 raft_consensus.cc:2804] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:27.917466 18383 raft_consensus.cc:697] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:27.918292 18383 consensus_queue.cc:237] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:27.927778 18333 raft_consensus.cc:2468] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:27.929303 18182 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e"
dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
mode: GRACEFUL
new_leader_uuid: "0dc67624917f441fbe92efac4c8d3934"
 from {username='slave'} at 127.0.0.1:33648
I20260709 13:22:27.929996 18182 raft_consensus.cc:606] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Received request to transfer leadership to TS 0dc67624917f441fbe92efac4c8d3934
I20260709 13:22:27.929625 18077 catalog_manager.cc:5697] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 1 to 2, leader changed from 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131) to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:27.932178 18379 consensus_queue.cc:1048] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:27.948632 18455 consensus_queue.cc:1048] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:28.018115 18455 consensus_queue.cc:1048] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:28.025058 18429 raft_consensus.cc:993] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934: : Instructing follower 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:28.025569 18379 raft_consensus.cc:1081] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Signalling peer 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:28.027191 18182 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464"
dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:33640
I20260709 13:22:28.028914 18182 raft_consensus.cc:493] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.029335 18182 raft_consensus.cc:3060] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.031205 18379 consensus_queue.cc:1048] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:28.037237 18182 raft_consensus.cc:515] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.043291 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:28.043576 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b1808a58b464d5d88dc40b33c33b464" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:28.043967 18333 raft_consensus.cc:3060] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.044246 18257 raft_consensus.cc:3055] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:28.044703 18257 raft_consensus.cc:740] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:28.045516 18257 consensus_queue.cc:260] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.046334 18257 raft_consensus.cc:3060] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.047065 18182 leader_election.cc:290] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:28.050495 18333 raft_consensus.cc:2468] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:28.051748 18149 leader_election.cc:304] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:28.052270 18257 raft_consensus.cc:2468] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:28.052441 18383 raft_consensus.cc:2804] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.052989 18383 raft_consensus.cc:697] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:28.053879 18383 consensus_queue.cc:237] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.058214 18378 raft_consensus.cc:993] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e: : Instructing follower 0dc67624917f441fbe92efac4c8d3934 to start an election
I20260709 13:22:28.058821 18378 raft_consensus.cc:1081] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Signalling peer 0dc67624917f441fbe92efac4c8d3934 to start an election
I20260709 13:22:28.060448 18257 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf"
dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
 from {username='slave'} at 127.0.0.1:36668
I20260709 13:22:28.060933 18257 raft_consensus.cc:493] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.061237 18257 raft_consensus.cc:3060] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.061950 18077 catalog_manager.cc:5697] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 1 to 2, leader changed from 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130) to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.068140 18257 raft_consensus.cc:515] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.071117 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:28.071667 18257 leader_election.cc:290] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 2 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:28.071775 18182 raft_consensus.cc:3060] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.071872 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca3425970884557b2bd8c85a567d4bf" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:28.072714 18333 raft_consensus.cc:3055] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:28.073357 18333 raft_consensus.cc:740] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:28.074034 18333 consensus_queue.cc:260] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.075150 18333 raft_consensus.cc:3060] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.078203 18182 raft_consensus.cc:2468] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 2.
I20260709 13:22:28.079541 18223 leader_election.cc:304] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:28.080394 18379 raft_consensus.cc:2804] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.080716 18333 raft_consensus.cc:2468] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 2.
I20260709 13:22:28.080920 18379 raft_consensus.cc:697] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:28.081887 18379 consensus_queue.cc:237] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.088984 18077 catalog_manager.cc:5697] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 1 to 2, leader changed from 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131) to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.091109 18397 raft_consensus.cc:993] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69: : Instructing follower 0dc67624917f441fbe92efac4c8d3934 to start an election
I20260709 13:22:28.091632 18383 raft_consensus.cc:1081] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Signalling peer 0dc67624917f441fbe92efac4c8d3934 to start an election
I20260709 13:22:28.093243 18257 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e"
dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
 from {username='slave'} at 127.0.0.1:36662
I20260709 13:22:28.093819 18257 raft_consensus.cc:493] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.094177 18257 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.100422 18257 raft_consensus.cc:515] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.101346 18410 raft_consensus.cc:993] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e: : Instructing follower 0dc67624917f441fbe92efac4c8d3934 to start an election
I20260709 13:22:28.101984 18378 raft_consensus.cc:1081] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Signalling peer 0dc67624917f441fbe92efac4c8d3934 to start an election
I20260709 13:22:28.102672 18257 leader_election.cc:290] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 2 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:28.102994 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:28.103266 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:28.103693 18182 raft_consensus.cc:3055] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:28.103825 18333 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.103940 18257 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7"
dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
 from {username='slave'} at 127.0.0.1:36668
I20260709 13:22:28.104092 18182 raft_consensus.cc:740] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:28.104684 18257 raft_consensus.cc:493] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.105180 18257 raft_consensus.cc:3060] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.105406 18182 consensus_queue.cc:260] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.106788 18182 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.110184 18257 raft_consensus.cc:515] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.111331 18333 raft_consensus.cc:2468] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 2.
I20260709 13:22:28.112613 18224 leader_election.cc:304] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:28.112728 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:28.113328 18183 raft_consensus.cc:3060] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.113452 18379 raft_consensus.cc:2804] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.113837 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "854a26ffa5a447e48a8c9778e299eda7" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:28.114276 18379 raft_consensus.cc:697] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:28.114665 18332 raft_consensus.cc:3055] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:28.113874 18182 raft_consensus.cc:2468] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 2.
I20260709 13:22:28.115137 18332 raft_consensus.cc:740] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:28.115265 18379 consensus_queue.cc:237] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.116065 18332 consensus_queue.cc:260] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.117287 18332 raft_consensus.cc:3060] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.118754 18257 leader_election.cc:290] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 2 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:28.122421 18183 raft_consensus.cc:2468] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 2.
I20260709 13:22:28.124104 18223 leader_election.cc:304] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:28.124845 18379 raft_consensus.cc:2804] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.124850 18332 raft_consensus.cc:2468] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 2.
I20260709 13:22:28.125375 18379 raft_consensus.cc:697] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:28.126266 18379 consensus_queue.cc:237] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:28.125881 18077 catalog_manager.cc:5697] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 1 to 2, leader changed from 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129) to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.138032 18077 catalog_manager.cc:5697] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 1 to 2, leader changed from 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131) to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 2 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.300081 18257 raft_consensus.cc:1275] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.301555 18383 consensus_queue.cc:1048] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:28.311219 18332 raft_consensus.cc:1275] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.312747 18383 consensus_queue.cc:1048] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.456506 18332 raft_consensus.cc:1275] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.458609 18397 consensus_queue.cc:1048] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.466534 18257 raft_consensus.cc:1275] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.469178 18397 consensus_queue.cc:1048] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.507499 18333 raft_consensus.cc:1275] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Refusing update from remote peer 0dc67624917f441fbe92efac4c8d3934: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.508941 18379 consensus_queue.cc:1048] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.517406 18183 raft_consensus.cc:1275] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Refusing update from remote peer 0dc67624917f441fbe92efac4c8d3934: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.520659 18429 consensus_queue.cc:1048] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:28.545235 18183 raft_consensus.cc:1275] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Refusing update from remote peer 0dc67624917f441fbe92efac4c8d3934: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.549762 18429 consensus_queue.cc:1048] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:28.557807 18333 raft_consensus.cc:1275] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Refusing update from remote peer 0dc67624917f441fbe92efac4c8d3934: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.559533 18332 raft_consensus.cc:1275] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Refusing update from remote peer 0dc67624917f441fbe92efac4c8d3934: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.561028 18455 consensus_queue.cc:1048] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:28.561359 18429 consensus_queue.cc:1048] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:28.578830 18183 raft_consensus.cc:1275] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Refusing update from remote peer 0dc67624917f441fbe92efac4c8d3934: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.582469 18429 consensus_queue.cc:1048] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:30.968005 18077 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38436:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:30.971542 18077 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:31.024940 18174 tablet_service.cc:1511] Processing CreateTablet for tablet 964b1facbeb64496ab5ca09fb2baf747 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:31.026185 18174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964b1facbeb64496ab5ca09fb2baf747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.025895 18248 tablet_service.cc:1511] Processing CreateTablet for tablet 964b1facbeb64496ab5ca09fb2baf747 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:31.027230 18248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964b1facbeb64496ab5ca09fb2baf747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.027149 18319 tablet_service.cc:1511] Processing CreateTablet for tablet 964b1facbeb64496ab5ca09fb2baf747 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:31.028474 18319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964b1facbeb64496ab5ca09fb2baf747. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.031328 18320 tablet_service.cc:1511] Processing CreateTablet for tablet 68bbc1fb66db4f4791fe746af459e9fb (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:31.032621 18320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68bbc1fb66db4f4791fe746af459e9fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.034142 18168 tablet_service.cc:1511] Processing CreateTablet for tablet 68bbc1fb66db4f4791fe746af459e9fb (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:31.034592 18246 tablet_service.cc:1511] Processing CreateTablet for tablet 68bbc1fb66db4f4791fe746af459e9fb (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:31.035604 18168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68bbc1fb66db4f4791fe746af459e9fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.036041 18246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68bbc1fb66db4f4791fe746af459e9fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.039718 18172 tablet_service.cc:1511] Processing CreateTablet for tablet 24f07396f12743a89602f33e458543c3 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:31.041007 18172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f07396f12743a89602f33e458543c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.049350 18247 tablet_service.cc:1511] Processing CreateTablet for tablet 24f07396f12743a89602f33e458543c3 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:31.050752 18247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f07396f12743a89602f33e458543c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.055176 18323 tablet_service.cc:1511] Processing CreateTablet for tablet aed0eb62ebee4d8ea4e76318af5574d0 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:31.055881 18243 tablet_service.cc:1511] Processing CreateTablet for tablet aed0eb62ebee4d8ea4e76318af5574d0 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:31.056543 18323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed0eb62ebee4d8ea4e76318af5574d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.057160 18243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed0eb62ebee4d8ea4e76318af5574d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.054352 18322 tablet_service.cc:1511] Processing CreateTablet for tablet 24f07396f12743a89602f33e458543c3 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:31.059603 18322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f07396f12743a89602f33e458543c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.059652 18169 tablet_service.cc:1511] Processing CreateTablet for tablet aed0eb62ebee4d8ea4e76318af5574d0 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:31.061059 18169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aed0eb62ebee4d8ea4e76318af5574d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.071656 18173 tablet_service.cc:1511] Processing CreateTablet for tablet 7dc3d7cd8b134d5fb6d9144e947d2598 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:31.073166 18173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dc3d7cd8b134d5fb6d9144e947d2598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.077346 18244 tablet_service.cc:1511] Processing CreateTablet for tablet 7dc3d7cd8b134d5fb6d9144e947d2598 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:31.078755 18244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dc3d7cd8b134d5fb6d9144e947d2598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.079123 18321 tablet_service.cc:1511] Processing CreateTablet for tablet 7dc3d7cd8b134d5fb6d9144e947d2598 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:31.080438 18321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dc3d7cd8b134d5fb6d9144e947d2598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.084168 18249 tablet_service.cc:1511] Processing CreateTablet for tablet 07bc88e5b51646d7877a5faf05e53a17 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:31.084690 18170 tablet_service.cc:1511] Processing CreateTablet for tablet 07bc88e5b51646d7877a5faf05e53a17 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:31.085556 18249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07bc88e5b51646d7877a5faf05e53a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.085953 18170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07bc88e5b51646d7877a5faf05e53a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.087702 18318 tablet_service.cc:1511] Processing CreateTablet for tablet 07bc88e5b51646d7877a5faf05e53a17 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:31.089042 18318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07bc88e5b51646d7877a5faf05e53a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.090649 18171 tablet_service.cc:1511] Processing CreateTablet for tablet 489df491873f484589e87f29b84fc0e6 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:31.091146 18245 tablet_service.cc:1511] Processing CreateTablet for tablet 489df491873f484589e87f29b84fc0e6 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:31.091965 18171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489df491873f484589e87f29b84fc0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.091652 18317 tablet_service.cc:1511] Processing CreateTablet for tablet 489df491873f484589e87f29b84fc0e6 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:31.092465 18245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489df491873f484589e87f29b84fc0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.093052 18317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489df491873f484589e87f29b84fc0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.100337 18167 tablet_service.cc:1511] Processing CreateTablet for tablet 72f6d0c0e5d748c8aa14bf23103600d1 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:31.101761 18167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f6d0c0e5d748c8aa14bf23103600d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.132627 18248 tablet_service.cc:1511] Processing CreateTablet for tablet 72f6d0c0e5d748c8aa14bf23103600d1 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:31.134595 18248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f6d0c0e5d748c8aa14bf23103600d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.160319 18477 tablet_bootstrap.cc:492] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.144760 18166 tablet_service.cc:1511] Processing CreateTablet for tablet 5eaa4568017f4ed28ce3d64303829b20 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:31.174754 18166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eaa4568017f4ed28ce3d64303829b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.137318 18316 tablet_service.cc:1511] Processing CreateTablet for tablet 72f6d0c0e5d748c8aa14bf23103600d1 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:31.178460 18316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f6d0c0e5d748c8aa14bf23103600d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.164463 18476 tablet_bootstrap.cc:492] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.172241 18475 tablet_bootstrap.cc:492] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.209347 18249 tablet_service.cc:1511] Processing CreateTablet for tablet 5eaa4568017f4ed28ce3d64303829b20 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:31.210783 18249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eaa4568017f4ed28ce3d64303829b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.212891 18320 tablet_service.cc:1511] Processing CreateTablet for tablet 5eaa4568017f4ed28ce3d64303829b20 (DEFAULT_TABLE table=soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:31.214293 18320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eaa4568017f4ed28ce3d64303829b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:31.228832 18476 tablet_bootstrap.cc:654] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.212844 18477 tablet_bootstrap.cc:654] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.236989 18475 tablet_bootstrap.cc:654] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.251634 18476 tablet_bootstrap.cc:492] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.252468 18476 ts_tablet_manager.cc:1403] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.008s
I20260709 13:22:31.261184 18476 raft_consensus.cc:359] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.262380 18476 raft_consensus.cc:385] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.262869 18476 raft_consensus.cc:740] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.264050 18476 consensus_queue.cc:260] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.266083 18475 tablet_bootstrap.cc:492] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.266680 18475 ts_tablet_manager.cc:1403] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.095s	user 0.014s	sys 0.011s
I20260709 13:22:31.266682 18476 ts_tablet_manager.cc:1434] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:22:31.267868 18476 tablet_bootstrap.cc:492] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.269189 18475 raft_consensus.cc:359] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.269856 18475 raft_consensus.cc:385] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.270105 18475 raft_consensus.cc:740] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.270797 18475 consensus_queue.cc:260] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.273056 18475 ts_tablet_manager.cc:1434] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:31.274147 18475 tablet_bootstrap.cc:492] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.275637 18476 tablet_bootstrap.cc:654] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.280627 18475 tablet_bootstrap.cc:654] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.289301 18476 tablet_bootstrap.cc:492] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.289927 18476 ts_tablet_manager.cc:1403] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:22:31.293262 18476 raft_consensus.cc:359] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.294100 18476 raft_consensus.cc:385] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.294442 18476 raft_consensus.cc:740] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.295363 18476 consensus_queue.cc:260] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.296969 18475 tablet_bootstrap.cc:492] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.297803 18476 ts_tablet_manager.cc:1434] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:31.298034 18475 ts_tablet_manager.cc:1403] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.012s
I20260709 13:22:31.298949 18476 tablet_bootstrap.cc:492] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.300796 18475 raft_consensus.cc:359] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.301337 18475 raft_consensus.cc:385] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.301584 18475 raft_consensus.cc:740] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.302201 18475 consensus_queue.cc:260] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.304327 18475 ts_tablet_manager.cc:1434] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260709 13:22:31.305771 18475 tablet_bootstrap.cc:492] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.305800 18476 tablet_bootstrap.cc:654] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.313230 18475 tablet_bootstrap.cc:654] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.316094 18477 tablet_bootstrap.cc:492] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.316680 18477 ts_tablet_manager.cc:1403] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.157s	user 0.011s	sys 0.010s
I20260709 13:22:31.317291 18476 tablet_bootstrap.cc:492] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.317874 18476 ts_tablet_manager.cc:1403] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:22:31.318917 18475 tablet_bootstrap.cc:492] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.319476 18475 ts_tablet_manager.cc:1403] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.014s	user 0.004s	sys 0.009s
I20260709 13:22:31.320999 18476 raft_consensus.cc:359] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.321861 18476 raft_consensus.cc:385] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.322162 18476 raft_consensus.cc:740] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.322933 18476 consensus_queue.cc:260] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.323271 18475 raft_consensus.cc:359] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.324157 18475 raft_consensus.cc:385] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.324527 18475 raft_consensus.cc:740] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.325274 18475 consensus_queue.cc:260] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.327850 18475 ts_tablet_manager.cc:1434] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:22:31.328912 18475 tablet_bootstrap.cc:492] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.322901 18477 raft_consensus.cc:359] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.335908 18475 tablet_bootstrap.cc:654] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.335757 18477 raft_consensus.cc:385] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.336462 18477 raft_consensus.cc:740] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.337567 18477 consensus_queue.cc:260] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.345506 18475 tablet_bootstrap.cc:492] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.345741 18477 ts_tablet_manager.cc:1434] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:22:31.347079 18477 tablet_bootstrap.cc:492] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.351135 18475 ts_tablet_manager.cc:1403] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:22:31.354286 18477 tablet_bootstrap.cc:654] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.356060 18475 raft_consensus.cc:359] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.357192 18475 raft_consensus.cc:385] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.357527 18475 raft_consensus.cc:740] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.358421 18475 consensus_queue.cc:260] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.360311 18476 ts_tablet_manager.cc:1434] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.042s	user 0.031s	sys 0.010s
I20260709 13:22:31.360868 18475 ts_tablet_manager.cc:1434] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:22:31.361651 18476 tablet_bootstrap.cc:492] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.362022 18475 tablet_bootstrap.cc:492] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.368680 18476 tablet_bootstrap.cc:654] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.376281 18477 tablet_bootstrap.cc:492] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.377034 18477 ts_tablet_manager.cc:1403] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.009s
I20260709 13:22:31.381003 18477 raft_consensus.cc:359] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.381711 18477 raft_consensus.cc:385] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.382020 18477 raft_consensus.cc:740] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.382858 18477 consensus_queue.cc:260] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.384899 18475 tablet_bootstrap.cc:654] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.385413 18476 tablet_bootstrap.cc:492] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.385664 18477 ts_tablet_manager.cc:1434] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 13:22:31.386101 18476 ts_tablet_manager.cc:1403] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.012s
I20260709 13:22:31.386783 18477 tablet_bootstrap.cc:492] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.389242 18476 raft_consensus.cc:359] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.390053 18476 raft_consensus.cc:385] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.390396 18476 raft_consensus.cc:740] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.391855 18476 consensus_queue.cc:260] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.395637 18476 ts_tablet_manager.cc:1434] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:22:31.397125 18476 tablet_bootstrap.cc:492] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.395572 18477 tablet_bootstrap.cc:654] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.404795 18475 tablet_bootstrap.cc:492] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.405359 18475 ts_tablet_manager.cc:1403] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.007s
I20260709 13:22:31.406028 18476 tablet_bootstrap.cc:654] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.408507 18475 raft_consensus.cc:359] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.409490 18475 raft_consensus.cc:385] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.409929 18475 raft_consensus.cc:740] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.411305 18475 consensus_queue.cc:260] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.414599 18475 ts_tablet_manager.cc:1434] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:22:31.415993 18475 tablet_bootstrap.cc:492] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.426824 18476 tablet_bootstrap.cc:492] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.427421 18475 tablet_bootstrap.cc:654] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.427548 18477 tablet_bootstrap.cc:492] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.428442 18477 ts_tablet_manager.cc:1403] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.042s	user 0.024s	sys 0.007s
I20260709 13:22:31.427541 18476 ts_tablet_manager.cc:1403] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.008s
I20260709 13:22:31.431762 18476 raft_consensus.cc:359] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.432487 18476 raft_consensus.cc:385] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.432783 18476 raft_consensus.cc:740] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.432827 18477 raft_consensus.cc:359] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.434039 18477 raft_consensus.cc:385] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.433585 18476 consensus_queue.cc:260] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.434588 18477 raft_consensus.cc:740] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.435907 18477 consensus_queue.cc:260] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.438692 18476 ts_tablet_manager.cc:1434] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:22:31.438872 18477 ts_tablet_manager.cc:1434] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:22:31.439796 18476 tablet_bootstrap.cc:492] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.440053 18477 tablet_bootstrap.cc:492] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.442603 18475 tablet_bootstrap.cc:492] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.447330 18476 tablet_bootstrap.cc:654] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.447814 18477 tablet_bootstrap.cc:654] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.448998 18475 ts_tablet_manager.cc:1403] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.009s
I20260709 13:22:31.453248 18475 raft_consensus.cc:359] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.454493 18475 raft_consensus.cc:385] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.454921 18475 raft_consensus.cc:740] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.456147 18475 consensus_queue.cc:260] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.459188 18476 tablet_bootstrap.cc:492] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.460186 18476 ts_tablet_manager.cc:1403] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:22:31.459263 18475 ts_tablet_manager.cc:1434] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:31.462881 18475 tablet_bootstrap.cc:492] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.463621 18476 raft_consensus.cc:359] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.464468 18476 raft_consensus.cc:385] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.464810 18476 raft_consensus.cc:740] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.465616 18476 consensus_queue.cc:260] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.468156 18476 ts_tablet_manager.cc:1434] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:31.469172 18476 tablet_bootstrap.cc:492] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.473387 18475 tablet_bootstrap.cc:654] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.475875 18476 tablet_bootstrap.cc:654] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.477128 18477 tablet_bootstrap.cc:492] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.477667 18477 ts_tablet_manager.cc:1403] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.038s	user 0.030s	sys 0.003s
I20260709 13:22:31.480806 18477 raft_consensus.cc:359] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.481504 18477 raft_consensus.cc:385] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.481803 18477 raft_consensus.cc:740] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.482735 18477 consensus_queue.cc:260] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.485148 18477 ts_tablet_manager.cc:1434] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:31.486133 18477 tablet_bootstrap.cc:492] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.493280 18475 tablet_bootstrap.cc:492] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.493832 18475 ts_tablet_manager.cc:1403] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.019s
I20260709 13:22:31.496829 18475 raft_consensus.cc:359] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.497531 18475 raft_consensus.cc:385] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.497836 18475 raft_consensus.cc:740] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.498518 18475 consensus_queue.cc:260] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.499603 18477 tablet_bootstrap.cc:654] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.500874 18475 ts_tablet_manager.cc:1434] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:31.501932 18475 tablet_bootstrap.cc:492] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.505388 18476 tablet_bootstrap.cc:492] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.506009 18476 ts_tablet_manager.cc:1403] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.015s
I20260709 13:22:31.508275 18475 tablet_bootstrap.cc:654] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.508824 18477 tablet_bootstrap.cc:492] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.509388 18477 ts_tablet_manager.cc:1403] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.002s
I20260709 13:22:31.509110 18476 raft_consensus.cc:359] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.509893 18476 raft_consensus.cc:385] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.510214 18476 raft_consensus.cc:740] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.510993 18476 consensus_queue.cc:260] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.513172 18476 ts_tablet_manager.cc:1434] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:31.514207 18477 raft_consensus.cc:359] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.514838 18485 raft_consensus.cc:493] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.515269 18477 raft_consensus.cc:385] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.515679 18477 raft_consensus.cc:740] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.514214 18476 tablet_bootstrap.cc:492] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.515354 18485 raft_consensus.cc:515] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.516585 18477 consensus_queue.cc:260] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.519567 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68bbc1fb66db4f4791fe746af459e9fb" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:31.520263 18257 raft_consensus.cc:2468] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:31.519591 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68bbc1fb66db4f4791fe746af459e9fb" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.521767 18332 raft_consensus.cc:2468] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:31.522478 18148 leader_election.cc:304] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.523510 18397 raft_consensus.cc:2804] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.523569 18477 ts_tablet_manager.cc:1434] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:22:31.523974 18397 raft_consensus.cc:493] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.524348 18397 raft_consensus.cc:3060] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.524976 18477 tablet_bootstrap.cc:492] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.530722 18485 leader_election.cc:290] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.532621 18477 tablet_bootstrap.cc:654] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.538815 18397 raft_consensus.cc:515] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.542579 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68bbc1fb66db4f4791fe746af459e9fb" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:31.542703 18378 raft_consensus.cc:493] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.543223 18257 raft_consensus.cc:3060] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.543385 18378 raft_consensus.cc:515] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.545523 18378 leader_election.cc:290] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:31.547353 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f07396f12743a89602f33e458543c3" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.547813 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f07396f12743a89602f33e458543c3" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:31.548071 18183 raft_consensus.cc:2468] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:31.548473 18259 raft_consensus.cc:2468] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:31.549657 18297 leader_election.cc:304] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:31.550999 18378 raft_consensus.cc:2804] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.551357 18378 raft_consensus.cc:493] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.551682 18378 raft_consensus.cc:3060] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.558019 18257 raft_consensus.cc:2468] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
I20260709 13:22:31.559566 18148 leader_election.cc:304] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.559814 18378 raft_consensus.cc:515] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.561005 18485 raft_consensus.cc:2804] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.561061 18475 tablet_bootstrap.cc:492] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.561739 18475 ts_tablet_manager.cc:1403] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.060s	user 0.021s	sys 0.029s
I20260709 13:22:31.561897 18477 tablet_bootstrap.cc:492] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.562031 18378 leader_election.cc:290] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:31.562527 18477 ts_tablet_manager.cc:1403] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.001s
I20260709 13:22:31.563020 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68bbc1fb66db4f4791fe746af459e9fb" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:31.563632 18332 raft_consensus.cc:3060] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.563778 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f07396f12743a89602f33e458543c3" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:31.564424 18257 raft_consensus.cc:3060] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.565006 18475 raft_consensus.cc:359] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.565771 18475 raft_consensus.cc:385] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.566181 18475 raft_consensus.cc:740] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.565816 18477 raft_consensus.cc:359] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.566685 18477 raft_consensus.cc:385] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.567436 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f07396f12743a89602f33e458543c3" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.566985 18475 consensus_queue.cc:260] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.568030 18183 raft_consensus.cc:3060] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.570113 18475 ts_tablet_manager.cc:1434] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:31.567025 18477 raft_consensus.cc:740] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.572566 18257 raft_consensus.cc:2468] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:31.559603 18397 leader_election.cc:290] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.572890 18475 tablet_bootstrap.cc:492] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: Bootstrap starting.
I20260709 13:22:31.573797 18297 leader_election.cc:304] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:31.572942 18476 tablet_bootstrap.cc:654] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.575920 18464 raft_consensus.cc:493] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.576396 18464 raft_consensus.cc:515] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.578418 18464 leader_election.cc:290] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.572582 18477 consensus_queue.cc:260] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.580447 18485 raft_consensus.cc:697] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:31.580749 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.581435 18182 raft_consensus.cc:2468] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.581897 18485 consensus_queue.cc:237] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.582862 18223 leader_election.cc:304] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.584781 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.585456 18333 raft_consensus.cc:2468] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.587046 18475 tablet_bootstrap.cc:654] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.587201 18332 raft_consensus.cc:2468] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
I20260709 13:22:31.589952 18378 raft_consensus.cc:2804] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.586186 18183 raft_consensus.cc:2468] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 1.
I20260709 13:22:31.588232 18464 raft_consensus.cc:2804] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.593228 18464 raft_consensus.cc:493] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.593583 18464 raft_consensus.cc:3060] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.601732 18477 ts_tablet_manager.cc:1434] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.039s	user 0.019s	sys 0.004s
I20260709 13:22:31.603004 18477 tablet_bootstrap.cc:492] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.602985 18464 raft_consensus.cc:515] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.605273 18485 raft_consensus.cc:493] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.605816 18485 raft_consensus.cc:515] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.607889 18485 leader_election.cc:290] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.607472 18077 catalog_manager.cc:5697] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.611371 18477 tablet_bootstrap.cc:654] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.613060 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964b1facbeb64496ab5ca09fb2baf747" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.613916 18332 raft_consensus.cc:2468] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:31.615561 18149 leader_election.cc:304] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:31.616665 18485 raft_consensus.cc:2804] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.617123 18485 raft_consensus.cc:493] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.617527 18485 raft_consensus.cc:3060] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.620517 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964b1facbeb64496ab5ca09fb2baf747" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:31.621352 18257 raft_consensus.cc:2468] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 0.
I20260709 13:22:31.626161 18503 raft_consensus.cc:493] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.640194 18476 tablet_bootstrap.cc:492] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.641009 18476 ts_tablet_manager.cc:1403] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.127s	user 0.011s	sys 0.013s
I20260709 13:22:31.627035 18503 raft_consensus.cc:515] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.639113 18378 raft_consensus.cc:697] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:31.643354 18378 consensus_queue.cc:237] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.645692 18503 leader_election.cc:290] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.645701 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bc88e5b51646d7877a5faf05e53a17" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.652184 18476 raft_consensus.cc:359] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
W20260709 13:22:31.653069 18223 leader_election.cc:343] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.653587 18476 raft_consensus.cc:385] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.654040 18476 raft_consensus.cc:740] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.655424 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bc88e5b51646d7877a5faf05e53a17" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.656152 18333 raft_consensus.cc:2468] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.655488 18476 consensus_queue.cc:260] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.657883 18224 leader_election.cc:304] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:31.659735 18508 raft_consensus.cc:2804] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.659847 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.660185 18508 raft_consensus.cc:493] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.660574 18183 raft_consensus.cc:3060] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.660602 18508 raft_consensus.cc:3060] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.662153 18476 ts_tablet_manager.cc:1434] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:22:31.663261 18476 tablet_bootstrap.cc:492] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69: Bootstrap starting.
I20260709 13:22:31.664678 18464 leader_election.cc:290] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.666476 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:31.668040 18332 raft_consensus.cc:3060] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.670033 18183 raft_consensus.cc:2468] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.662181 18485 raft_consensus.cc:515] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.671439 18078 catalog_manager.cc:5697] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 1 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:31.672653 18476 tablet_bootstrap.cc:654] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.673357 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964b1facbeb64496ab5ca09fb2baf747" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:31.673961 18257 raft_consensus.cc:3060] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.676329 18332 raft_consensus.cc:2468] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.679046 18464 raft_consensus.cc:493] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.684350 18500 raft_consensus.cc:493] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.685454 18500 raft_consensus.cc:515] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.686594 18224 leader_election.cc:304] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:31.688691 18510 raft_consensus.cc:2804] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.689553 18510 raft_consensus.cc:697] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:31.690991 18510 consensus_queue.cc:237] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.691294 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
W20260709 13:22:31.694550 18148 leader_election.cc:343] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.679639 18464 raft_consensus.cc:515] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.698726 18464 leader_election.cc:290] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.700330 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc3d7cd8b134d5fb6d9144e947d2598" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.701086 18500 leader_election.cc:290] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.701761 18182 raft_consensus.cc:2468] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.702284 18257 raft_consensus.cc:2468] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
I20260709 13:22:31.703444 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.707266 18148 leader_election.cc:304] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.707839 18223 leader_election.cc:304] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.708647 18508 raft_consensus.cc:515] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.709537 18464 raft_consensus.cc:2804] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.709995 18464 raft_consensus.cc:493] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.710352 18464 raft_consensus.cc:3060] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:31.709019 18149 leader_election.cc:343] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.711076 18485 leader_election.cc:290] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.711262 18149 leader_election.cc:304] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69; no voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:31.711946 18477 tablet_bootstrap.cc:492] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.712306 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bc88e5b51646d7877a5faf05e53a17" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:31.713414 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bc88e5b51646d7877a5faf05e53a17" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.713066 18333 raft_consensus.cc:3060] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.714551 18331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964b1facbeb64496ab5ca09fb2baf747" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
W20260709 13:22:31.714955 18223 leader_election.cc:343] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.715497 18331 raft_consensus.cc:3060] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.709040 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc3d7cd8b134d5fb6d9144e947d2598" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
W20260709 13:22:31.717394 18224 leader_election.cc:343] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.718437 18485 raft_consensus.cc:2749] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:31.709019 18500 raft_consensus.cc:2804] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.719359 18500 raft_consensus.cc:697] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:31.720211 18500 consensus_queue.cc:237] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.712373 18475 tablet_bootstrap.cc:492] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: No bootstrap required, opened a new log
I20260709 13:22:31.722343 18475 ts_tablet_manager.cc:1403] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: Time spent bootstrapping tablet: real 0.150s	user 0.026s	sys 0.007s
I20260709 13:22:31.725150 18464 raft_consensus.cc:515] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.713066 18477 ts_tablet_manager.cc:1403] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.110s	user 0.032s	sys 0.028s
I20260709 13:22:31.727854 18475 raft_consensus.cc:359] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.728777 18475 raft_consensus.cc:385] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.729121 18475 raft_consensus.cc:740] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.733848 18477 raft_consensus.cc:359] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.734679 18477 raft_consensus.cc:385] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.735260 18477 raft_consensus.cc:740] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.734575 18503 raft_consensus.cc:493] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.736110 18477 consensus_queue.cc:260] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.736431 18503 raft_consensus.cc:515] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.730490 18475 consensus_queue.cc:260] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.727353 18508 leader_election.cc:290] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.737138 18333 raft_consensus.cc:2468] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.738533 18503 leader_election.cc:290] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.744385 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.744781 18475 ts_tablet_manager.cc:1434] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:22:31.744616 18224 leader_election.cc:304] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:31.745096 18182 raft_consensus.cc:2468] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.745500 18503 raft_consensus.cc:2804] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.745980 18503 raft_consensus.cc:697] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:31.746131 18331 raft_consensus.cc:2468] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 1.
I20260709 13:22:31.746475 18223 leader_election.cc:304] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.747466 18503 consensus_queue.cc:237] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.748458 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc3d7cd8b134d5fb6d9144e947d2598" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.749063 18182 raft_consensus.cc:3060] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.749419 18510 raft_consensus.cc:2804] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.750010 18510 raft_consensus.cc:493] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.750486 18510 raft_consensus.cc:3060] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.751226 18477 ts_tablet_manager.cc:1434] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:22:31.754048 18476 tablet_bootstrap.cc:492] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69: No bootstrap required, opened a new log
I20260709 13:22:31.754856 18476 ts_tablet_manager.cc:1403] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.016s
I20260709 13:22:31.756273 18508 raft_consensus.cc:493] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.756901 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.770939 18477 tablet_bootstrap.cc:492] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.773087 18331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dc3d7cd8b134d5fb6d9144e947d2598" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:31.773779 18331 raft_consensus.cc:3060] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.773262 18080 catalog_manager.cc:5697] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.775055 18506 raft_consensus.cc:493] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.775570 18506 raft_consensus.cc:515] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.778343 18476 raft_consensus.cc:359] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.779130 18476 raft_consensus.cc:385] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.779417 18476 raft_consensus.cc:740] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.777674 18506 leader_election.cc:290] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:31.780138 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:31.778525 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.780112 18476 consensus_queue.cc:260] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.780982 18183 raft_consensus.cc:2468] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 0.
I20260709 13:22:31.782226 18476 ts_tablet_manager.cc:1434] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:22:31.782228 18297 leader_election.cc:304] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:31.770175 18508 raft_consensus.cc:515] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.791189 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.791872 18183 raft_consensus.cc:2468] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.767726 18464 leader_election.cc:290] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.785043 18333 raft_consensus.cc:2468] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.793042 18223 leader_election.cc:304] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.793697 18506 raft_consensus.cc:2804] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.793902 18464 raft_consensus.cc:2804] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.798728 18506 raft_consensus.cc:493] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.799238 18506 raft_consensus.cc:3060] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.797282 18464 raft_consensus.cc:493] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.796921 18503 raft_consensus.cc:493] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.803867 18331 raft_consensus.cc:2468] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.804350 18508 leader_election.cc:290] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.810580 18224 leader_election.cc:304] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:31.804875 18078 catalog_manager.cc:5697] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 1 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.805012 18464 raft_consensus.cc:3060] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.811678 18508 raft_consensus.cc:2804] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.812350 18508 raft_consensus.cc:697] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:31.804466 18510 raft_consensus.cc:515] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.814407 18257 raft_consensus.cc:2393] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b41bb7f2b7847c19b86e54214bf0f4e in current term 1: Already voted for candidate 0dc67624917f441fbe92efac4c8d3934 in this term.
I20260709 13:22:31.805488 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.807811 18506 raft_consensus.cc:515] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.816380 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.807910 18182 raft_consensus.cc:2468] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.815650 18508 consensus_queue.cc:237] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.816999 18183 raft_consensus.cc:3060] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.819197 18510 leader_election.cc:290] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.826296 18464 raft_consensus.cc:515] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.827526 18506 leader_election.cc:290] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
W20260709 13:22:31.828804 18224 leader_election.cc:343] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.831936 18477 tablet_bootstrap.cc:654] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.830523 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.838181 18464 leader_election.cc:290] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.838711 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:31.839406 18257 raft_consensus.cc:2393] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b41bb7f2b7847c19b86e54214bf0f4e in current term 1: Already voted for candidate 0dc67624917f441fbe92efac4c8d3934 in this term.
I20260709 13:22:31.808147 18503 raft_consensus.cc:515] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.840657 18331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489df491873f484589e87f29b84fc0e6" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:31.840768 18183 raft_consensus.cc:2468] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.841430 18331 raft_consensus.cc:2393] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0dc67624917f441fbe92efac4c8d3934 in current term 1: Already voted for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in this term.
I20260709 13:22:31.842494 18503 leader_election.cc:290] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.843698 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72f6d0c0e5d748c8aa14bf23103600d1" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:31.844010 18297 leader_election.cc:304] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:31.843870 18080 catalog_manager.cc:5697] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.845489 18506 raft_consensus.cc:2749] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:22:31.846398 18224 leader_election.cc:343] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.847016 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.847651 18183 raft_consensus.cc:3060] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.849017 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
W20260709 13:22:31.850308 18224 leader_election.cc:343] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.852043 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72f6d0c0e5d748c8aa14bf23103600d1" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:31.852699 18182 raft_consensus.cc:2468] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 0.
I20260709 13:22:31.853996 18223 leader_election.cc:304] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:31.855306 18223 leader_election.cc:304] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:31.856074 18464 raft_consensus.cc:2804] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.856666 18464 raft_consensus.cc:697] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:31.857208 18510 raft_consensus.cc:2804] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.857849 18510 raft_consensus.cc:493] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.857510 18464 consensus_queue.cc:237] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.858317 18510 raft_consensus.cc:3060] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.863711 18183 raft_consensus.cc:2468] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.864812 18477 tablet_bootstrap.cc:492] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.865552 18477 ts_tablet_manager.cc:1403] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.095s	user 0.015s	sys 0.003s
I20260709 13:22:31.865397 18223 leader_election.cc:304] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:31.866684 18503 raft_consensus.cc:2804] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.867312 18503 raft_consensus.cc:697] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:31.868887 18503 consensus_queue.cc:237] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.876327 18477 raft_consensus.cc:359] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.877416 18477 raft_consensus.cc:385] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.877681 18477 raft_consensus.cc:740] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.884809 18510 raft_consensus.cc:515] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.878578 18477 consensus_queue.cc:260] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.887871 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72f6d0c0e5d748c8aa14bf23103600d1" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:31.888134 18510 leader_election.cc:290] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:31.888435 18183 raft_consensus.cc:3060] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.889171 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72f6d0c0e5d748c8aa14bf23103600d1" candidate_uuid: "0dc67624917f441fbe92efac4c8d3934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:31.889789 18333 raft_consensus.cc:3060] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.893899 18477 ts_tablet_manager.cc:1434] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.008s
I20260709 13:22:31.894287 18183 raft_consensus.cc:2468] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.895253 18477 tablet_bootstrap.cc:492] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e: Bootstrap starting.
I20260709 13:22:31.895484 18223 leader_election.cc:304] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69; no voters: 
I20260709 13:22:31.896548 18510 raft_consensus.cc:2804] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.897013 18510 raft_consensus.cc:697] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:31.897035 18333 raft_consensus.cc:2468] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc67624917f441fbe92efac4c8d3934 in term 1.
I20260709 13:22:31.898078 18510 consensus_queue.cc:237] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.900964 18477 tablet_bootstrap.cc:654] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:31.908083 18477 tablet_bootstrap.cc:492] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e: No bootstrap required, opened a new log
I20260709 13:22:31.908483 18477 ts_tablet_manager.cc:1403] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:22:31.911196 18477 raft_consensus.cc:359] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.911892 18477 raft_consensus.cc:385] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:31.912163 18477 raft_consensus.cc:740] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.912146 18080 catalog_manager.cc:5697] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.913265 18477 consensus_queue.cc:260] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:31.914125 18080 catalog_manager.cc:5697] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.915339 18477 ts_tablet_manager.cc:1434] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:31.915679 18080 catalog_manager.cc:5697] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.930819 18078 catalog_manager.cc:5697] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130). New cstate: current_term: 1 leader_uuid: "0dc67624917f441fbe92efac4c8d3934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.093564 18333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24f07396f12743a89602f33e458543c3"
dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
mode: GRACEFUL
new_leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:59280
I20260709 13:22:32.094121 18333 raft_consensus.cc:606] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Received request to transfer leadership to TS 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:32.099195 18257 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0"
dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
mode: GRACEFUL
new_leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:36670
I20260709 13:22:32.099828 18257 raft_consensus.cc:606] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Received request to transfer leadership to TS 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:32.101995 18510 consensus_queue.cc:1048] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.121706 18510 raft_consensus.cc:993] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: : Instructing follower 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:32.121673 18500 consensus_queue.cc:1048] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.122390 18518 raft_consensus.cc:1081] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Signalling peer 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:32.125397 18506 consensus_queue.cc:1048] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.128058 18506 raft_consensus.cc:493] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.128399 18257 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20"
dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
mode: GRACEFUL
new_leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
 from {username='slave'} at 127.0.0.1:36670
I20260709 13:22:32.128516 18506 raft_consensus.cc:515] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.129112 18257 raft_consensus.cc:606] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Received request to transfer leadership to TS 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:32.131556 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69" is_pre_election: true
I20260709 13:22:32.132359 18182 raft_consensus.cc:2393] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b41bb7f2b7847c19b86e54214bf0f4e in current term 1: Already voted for candidate 0dc67624917f441fbe92efac4c8d3934 in this term.
I20260709 13:22:32.133529 18184 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0"
dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:33640
I20260709 13:22:32.135202 18502 raft_consensus.cc:993] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: : Instructing follower 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:32.135819 18502 raft_consensus.cc:1081] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Signalling peer 283925a3c2d24fb387b016ef4f9c5d69 to start an election
I20260709 13:22:32.139009 18502 consensus_queue.cc:1048] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.139220 18181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24f07396f12743a89602f33e458543c3"
dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
 from {username='slave'} at 127.0.0.1:33626
I20260709 13:22:32.142908 18508 consensus_queue.cc:1048] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.152272 18506 leader_election.cc:290] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:32.167738 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:32.173959 18297 leader_election.cc:304] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 0dc67624917f441fbe92efac4c8d3934, 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:32.174463 18485 raft_consensus.cc:493] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.175195 18485 raft_consensus.cc:515] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.176179 18464 consensus_queue.cc:1048] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.181581 18378 raft_consensus.cc:3060] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.184147 18485 leader_election.cc:290] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:32.193787 18332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bc88e5b51646d7877a5faf05e53a17" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" is_pre_election: true
I20260709 13:22:32.196664 18258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bc88e5b51646d7877a5faf05e53a17" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc67624917f441fbe92efac4c8d3934" is_pre_election: true
I20260709 13:22:32.199373 18485 consensus_queue.cc:1048] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.214190 18485 consensus_queue.cc:1048] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.235755 18180 raft_consensus.cc:3060] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:32.239108 18148 leader_election.cc:304] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69; no voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:32.243292 18181 raft_consensus.cc:493] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:32.243724 18181 raft_consensus.cc:3060] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.251839 18508 consensus_queue.cc:1048] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.260489 18181 raft_consensus.cc:515] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.268020 18537 raft_consensus.cc:2749] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:32.274884 18378 raft_consensus.cc:2749] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:32.279491 18464 consensus_queue.cc:1048] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.291271 18184 raft_consensus.cc:493] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:32.291827 18184 raft_consensus.cc:3060] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.299047 18485 consensus_queue.cc:1048] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.299615 18184 raft_consensus.cc:515] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.316946 18181 leader_election.cc:290] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:32.320034 18333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f07396f12743a89602f33e458543c3" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:32.320796 18333 raft_consensus.cc:3055] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:32.321215 18333 raft_consensus.cc:740] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:32.322279 18333 consensus_queue.cc:260] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.325132 18259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:32.325902 18259 raft_consensus.cc:3055] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:32.326303 18259 raft_consensus.cc:740] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:32.344364 18333 raft_consensus.cc:3060] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.303378 18258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f07396f12743a89602f33e458543c3" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:32.345075 18184 leader_election.cc:290] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Requested vote from peers 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155), 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131:38045)
I20260709 13:22:32.345197 18258 raft_consensus.cc:3060] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.345821 18330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aed0eb62ebee4d8ea4e76318af5574d0" candidate_uuid: "283925a3c2d24fb387b016ef4f9c5d69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
I20260709 13:22:32.346447 18330 raft_consensus.cc:3060] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.349659 18508 consensus_queue.cc:1048] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.350548 18259 consensus_queue.cc:260] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.351980 18259 raft_consensus.cc:3060] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.348392 18464 consensus_queue.cc:1048] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.353960 18330 raft_consensus.cc:2468] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:32.355165 18149 leader_election.cc:304] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:32.356421 18543 raft_consensus.cc:2804] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:32.356878 18543 raft_consensus.cc:697] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:32.357913 18333 raft_consensus.cc:2468] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:32.359380 18149 leader_election.cc:304] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:32.360278 18500 raft_consensus.cc:2804] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:32.360756 18500 raft_consensus.cc:697] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Becoming Leader. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:32.358017 18543 consensus_queue.cc:237] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.363529 18500 consensus_queue.cc:237] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.366348 18510 consensus_queue.cc:1048] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.370769 18464 consensus_queue.cc:1048] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.372670 18259 raft_consensus.cc:2468] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:32.374123 18518 raft_consensus.cc:993] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: : Instructing follower 5b41bb7f2b7847c19b86e54214bf0f4e to start an election
I20260709 13:22:32.374754 18508 raft_consensus.cc:1081] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Signalling peer 5b41bb7f2b7847c19b86e54214bf0f4e to start an election
I20260709 13:22:32.388281 18258 raft_consensus.cc:2468] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283925a3c2d24fb387b016ef4f9c5d69 in term 2.
I20260709 13:22:32.390179 18330 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20"
dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
 from {username='slave'} at 127.0.0.1:59272
I20260709 13:22:32.390898 18330 raft_consensus.cc:493] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:32.391274 18330 raft_consensus.cc:3060] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.394707 18508 consensus_queue.cc:1048] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.396695 18464 consensus_queue.cc:1048] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:32.419970 18330 raft_consensus.cc:515] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.422046 18330 leader_election.cc:290] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 2 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:32.422708 18182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:32.425604 18258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa4568017f4ed28ce3d64303829b20" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:32.426221 18258 raft_consensus.cc:3055] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:32.426575 18258 raft_consensus.cc:740] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:32.428421 18258 consensus_queue.cc:260] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.429425 18258 raft_consensus.cc:3060] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:32.429917 18508 consensus_queue.cc:1175] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "283925a3c2d24fb387b016ef4f9c5d69" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:22:32.435596 18077 catalog_manager.cc:5697] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 1 to 2, leader changed from 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130) to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.437302 18077 catalog_manager.cc:5697] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 reported cstate change: term changed from 1 to 2, leader changed from 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131) to 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129). New cstate: current_term: 2 leader_uuid: "283925a3c2d24fb387b016ef4f9c5d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.438402 18510 consensus_queue.cc:1048] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:32.444000 18258 raft_consensus.cc:2468] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 2.
I20260709 13:22:32.445233 18297 leader_election.cc:304] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dc67624917f441fbe92efac4c8d3934, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 283925a3c2d24fb387b016ef4f9c5d69
I20260709 13:22:32.446004 18378 raft_consensus.cc:2804] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:32.446453 18378 raft_consensus.cc:697] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:32.447184 18378 consensus_queue.cc:237] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:32.471962 18077 catalog_manager.cc:5697] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 1 to 2, leader changed from 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130) to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 2 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:32.744534 18329 raft_consensus.cc:1275] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.745828 18500 consensus_queue.cc:1048] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:32.754340 18258 raft_consensus.cc:1275] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.756712 18500 consensus_queue.cc:1048] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:32.832253 18329 raft_consensus.cc:1275] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.833509 18500 consensus_queue.cc:1048] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:32.841102 18258 raft_consensus.cc:1275] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Refusing update from remote peer 283925a3c2d24fb387b016ef4f9c5d69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.843639 18500 consensus_queue.cc:1048] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:32.863734 18258 raft_consensus.cc:1275] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Refusing update from remote peer 5b41bb7f2b7847c19b86e54214bf0f4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.864938 18506 consensus_queue.cc:1048] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:32.872543 18181 raft_consensus.cc:3060] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.880666 18181 raft_consensus.cc:1275] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Refusing update from remote peer 5b41bb7f2b7847c19b86e54214bf0f4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.882167 18555 consensus_queue.cc:1048] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.206753 18078 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38436:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:22:35.226774 18078 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38436:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:22:35.316551 18584 tablet.cc:1724] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.401695  5410 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:35.430192 18593 tablet.cc:1724] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:35.421684 18597 log.cc:927] Time spent T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: Append to log took a long time: real 0.069s	user 0.002s	sys 0.000s
W20260709 13:22:35.466864 18587 log.cc:927] Time spent T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:22:35.588653 18600 log.cc:927] Time spent T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:22:35.589084 18593 tablet.cc:1724] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.576118 18584 tablet.cc:1724] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.562132 18612 tablet.cc:1724] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.580868 18616 tablet.cc:1724] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:35.643677 18605 log.cc:927] Time spent T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:22:35.643226 18618 tablet.cc:1724] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:35.644352 18611 log.cc:927] Time spent T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:22:35.643640 18623 tablet.cc:1724] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.646600 18615 tablet.cc:1724] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:35.635028 18223 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.129:39331, user_credentials={real_user=slave}} blocked reactor thread for 77703.5us
I20260709 13:22:35.654436 18614 tablet.cc:1724] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.667652 18620 tablet.cc:1724] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.681161 18619 tablet.cc:1724] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:35.682579 18148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.72.130:40155, user_credentials={real_user=slave}} blocked reactor thread for 105631us
I20260709 13:22:35.695012 18618 tablet.cc:1724] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.695501 18623 tablet.cc:1724] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.700001 18593 tablet.cc:1724] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.708808 18184 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0"
dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
mode: GRACEFUL
new_leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
 from {username='slave'} at 127.0.0.1:56434
I20260709 13:22:35.709461 18184 raft_consensus.cc:606] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Received request to transfer leadership to TS 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:35.717629 18618 tablet.cc:1724] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.727525 18630 tablet.cc:1724] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.718341 18616 tablet.cc:1724] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.718922 18612 tablet.cc:1724] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.763479 18629 tablet.cc:1724] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:35.766839 18625 log.cc:927] Time spent T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:22:35.772949 18623 tablet.cc:1724] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.756206 18584 tablet.cc:1724] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.787811 18612 tablet.cc:1724] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.799907 18485 raft_consensus.cc:993] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69: : Instructing follower 5b41bb7f2b7847c19b86e54214bf0f4e to start an election
I20260709 13:22:35.800495 18485 raft_consensus.cc:1081] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Signalling peer 5b41bb7f2b7847c19b86e54214bf0f4e to start an election
I20260709 13:22:35.802341 18332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0"
dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
 from {username='slave'} at 127.0.0.1:59256
I20260709 13:22:35.802939 18332 raft_consensus.cc:493] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.803314 18332 raft_consensus.cc:3060] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.810285 18332 raft_consensus.cc:515] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:35.814496 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:35.815101 18183 raft_consensus.cc:3055] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.815384 18183 raft_consensus.cc:740] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 283925a3c2d24fb387b016ef4f9c5d69, State: Running, Role: LEADER
I20260709 13:22:35.815773 18258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee78df24c040389e1bec48dcdfe6e0" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:35.816423 18258 raft_consensus.cc:3060] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.816155 18183 consensus_queue.cc:260] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:35.817126 18183 raft_consensus.cc:3060] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.821547 18593 tablet.cc:1724] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.827924 18183 raft_consensus.cc:2468] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 2.
I20260709 13:22:35.818523 18332 leader_election.cc:290] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 2 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:35.837824 18297 leader_election.cc:304] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:35.839244 18555 raft_consensus.cc:2804] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.839741 18555 raft_consensus.cc:697] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:35.840131 18614 tablet.cc:1724] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.840534 18555 consensus_queue.cc:237] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:35.843691 18258 raft_consensus.cc:2468] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 2.
I20260709 13:22:35.835424 18615 tablet.cc:1724] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.858693 18258 raft_consensus.cc:1240] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Rejecting Update request from peer 283925a3c2d24fb387b016ef4f9c5d69 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:35.864570 18629 tablet.cc:1724] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:35.867507 18257 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e"
dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
mode: GRACEFUL
new_leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
 from {username='slave'} at 127.0.0.1:58266
I20260709 13:22:35.868160 18257 raft_consensus.cc:606] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Received request to transfer leadership to TS 5b41bb7f2b7847c19b86e54214bf0f4e
I20260709 13:22:35.871872  5410 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:35.872522  5410 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:35.878221 18080 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a: soft_deleted_table [id=6663017faf6e4b768c0c7ee436c9c65d] alter complete (version 1)
I20260709 13:22:35.887549 18080 catalog_manager.cc:5697] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 1 to 2, leader changed from 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129) to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 2 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:36.228802 18257 raft_consensus.cc:1275] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Refusing update from remote peer 5b41bb7f2b7847c19b86e54214bf0f4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.231048 18555 consensus_queue.cc:1048] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:36.248487 18183 raft_consensus.cc:1275] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Refusing update from remote peer 5b41bb7f2b7847c19b86e54214bf0f4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.250941 18588 raft_consensus.cc:993] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934: : Instructing follower 5b41bb7f2b7847c19b86e54214bf0f4e to start an election
I20260709 13:22:36.251108 18638 consensus_queue.cc:1048] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:36.251844 18588 raft_consensus.cc:1081] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Signalling peer 5b41bb7f2b7847c19b86e54214bf0f4e to start an election
I20260709 13:22:36.253866 18332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e"
dest_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e"
 from {username='slave'} at 127.0.0.1:59272
I20260709 13:22:36.254855 18332 raft_consensus.cc:493] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:36.255450 18332 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:36.274072 18332 raft_consensus.cc:515] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:36.283906 18332 leader_election.cc:290] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 3 election: Requested vote from peers 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331), 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155)
I20260709 13:22:36.286145 18257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0dc67624917f441fbe92efac4c8d3934"
I20260709 13:22:36.285678 18183 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4697fc0d4ce422c8129cb8d07900a5e" candidate_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "283925a3c2d24fb387b016ef4f9c5d69"
I20260709 13:22:36.289427 18257 raft_consensus.cc:3055] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:36.290100 18183 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:36.290179 18257 raft_consensus.cc:740] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0dc67624917f441fbe92efac4c8d3934, State: Running, Role: LEADER
I20260709 13:22:36.294591 18257 consensus_queue.cc:260] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:36.298264 18257 raft_consensus.cc:3060] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:36.308125 18183 raft_consensus.cc:2468] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 3.
I20260709 13:22:36.309614 18297 leader_election.cc:304] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283925a3c2d24fb387b016ef4f9c5d69, 5b41bb7f2b7847c19b86e54214bf0f4e; no voters: 
I20260709 13:22:36.310568 18638 raft_consensus.cc:2804] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:36.311070 18638 raft_consensus.cc:697] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 3 LEADER]: Becoming Leader. State: Replica: 5b41bb7f2b7847c19b86e54214bf0f4e, State: Running, Role: LEADER
I20260709 13:22:36.312047 18638 consensus_queue.cc:237] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } }
I20260709 13:22:36.315663 18257 raft_consensus.cc:2468] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b41bb7f2b7847c19b86e54214bf0f4e in term 3.
I20260709 13:22:36.324134 18080 catalog_manager.cc:5697] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e reported cstate change: term changed from 2 to 3, leader changed from 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130) to 5b41bb7f2b7847c19b86e54214bf0f4e (127.5.72.131). New cstate: current_term: 3 leader_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b41bb7f2b7847c19b86e54214bf0f4e" member_type: VOTER last_known_addr { host: "127.5.72.131" port: 38045 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:36.747627 18184 raft_consensus.cc:1275] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 3 FOLLOWER]: Refusing update from remote peer 5b41bb7f2b7847c19b86e54214bf0f4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:36.748962 18638 consensus_queue.cc:1048] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "283925a3c2d24fb387b016ef4f9c5d69" member_type: VOTER last_known_addr { host: "127.5.72.129" port: 39331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:36.760746 18257 raft_consensus.cc:1275] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 3 FOLLOWER]: Refusing update from remote peer 5b41bb7f2b7847c19b86e54214bf0f4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:36.763607 18555 consensus_queue.cc:1048] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc67624917f441fbe92efac4c8d3934" member_type: VOTER last_known_addr { host: "127.5.72.130" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:36.930325  5410 tablet_server.cc:179] TabletServer@127.5.72.129:0 shutting down...
I20260709 13:22:36.957450  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:36.958230  5410 tablet_replica.cc:333] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.959245  5410 raft_consensus.cc:2243] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:36.959860  5410 raft_consensus.cc:2272] T 07bc88e5b51646d7877a5faf05e53a17 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.962518  5410 tablet_replica.cc:333] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.963363  5410 raft_consensus.cc:2243] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:36.963897  5410 raft_consensus.cc:2272] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.966423  5410 tablet_replica.cc:333] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.967103  5410 raft_consensus.cc:2243] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:36.967623  5410 raft_consensus.cc:2272] T 72f6d0c0e5d748c8aa14bf23103600d1 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.969878  5410 tablet_replica.cc:333] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.970652  5410 raft_consensus.cc:2243] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:36.971525  5410 raft_consensus.cc:2272] T aed0eb62ebee4d8ea4e76318af5574d0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.974025  5410 tablet_replica.cc:333] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.974728  5410 raft_consensus.cc:2243] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:36.975255  5410 raft_consensus.cc:2272] T 489df491873f484589e87f29b84fc0e6 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.977640  5410 tablet_replica.cc:333] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.978264  5410 raft_consensus.cc:2243] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:36.979151  5410 raft_consensus.cc:2272] T 24f07396f12743a89602f33e458543c3 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.981748  5410 tablet_replica.cc:333] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.982600  5410 raft_consensus.cc:2243] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:36.983175  5410 raft_consensus.cc:2272] T b7ee78df24c040389e1bec48dcdfe6e0 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.986811  5410 tablet_replica.cc:333] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.987394  5410 raft_consensus.cc:2243] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:36.988304  5410 raft_consensus.cc:2272] T 4b1808a58b464d5d88dc40b33c33b464 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.990607  5410 tablet_replica.cc:333] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.991326  5410 raft_consensus.cc:2243] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:36.991858  5410 raft_consensus.cc:2272] T 854a26ffa5a447e48a8c9778e299eda7 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.994191  5410 tablet_replica.cc:333] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.994823  5410 raft_consensus.cc:2243] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:36.995622  5410 raft_consensus.cc:2272] T 964b1facbeb64496ab5ca09fb2baf747 P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:36.997954  5410 tablet_replica.cc:333] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:36.998585  5410 raft_consensus.cc:2243] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:36.999485  5410 raft_consensus.cc:2272] T 68bbc1fb66db4f4791fe746af459e9fb P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.001940  5410 tablet_replica.cc:333] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:37.002502  5410 raft_consensus.cc:2243] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.003038  5410 raft_consensus.cc:2272] T 6d713a8451ee46c2be2578e9a0e3fcef P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.005573  5410 tablet_replica.cc:333] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:37.006170  5410 raft_consensus.cc:2243] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.006822  5410 raft_consensus.cc:2272] T 219dd503f7bd49aab20bc06b6c4779ff P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.009197  5410 tablet_replica.cc:333] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:37.009768  5410 raft_consensus.cc:2243] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.010582  5410 raft_consensus.cc:2272] T 3fbd032884b645ae846c6f3733dcc917 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.012845  5410 tablet_replica.cc:333] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:37.013533  5410 raft_consensus.cc:2243] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.014214  5410 raft_consensus.cc:2272] T 0ca3425970884557b2bd8c85a567d4bf P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:37.015308 18223 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 39 similar messages]
I20260709 13:22:37.017064  5410 tablet_replica.cc:333] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:37.017771  5410 raft_consensus.cc:2243] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.018410  5410 raft_consensus.cc:2272] T 5eaa4568017f4ed28ce3d64303829b20 P 283925a3c2d24fb387b016ef4f9c5d69 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:37.018769 18223 consensus_peers.cc:597] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 -> Peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Couldn't send request to peer 283925a3c2d24fb387b016ef4f9c5d69. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.020787  5410 tablet_replica.cc:333] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:37.021440  5410 raft_consensus.cc:2243] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.021899  5410 raft_consensus.cc:2272] T d4697fc0d4ce422c8129cb8d07900a5e P 283925a3c2d24fb387b016ef4f9c5d69 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.024727  5410 tablet_replica.cc:333] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69: stopping tablet replica
I20260709 13:22:37.025261  5410 raft_consensus.cc:2243] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.026131  5410 raft_consensus.cc:2272] T e32ac98de5674dacaef8332140cb0d2d P 283925a3c2d24fb387b016ef4f9c5d69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.072207  5410 tablet_server.cc:196] TabletServer@127.5.72.129:0 shutdown complete.
W20260709 13:22:37.113147 18223 consensus_peers.cc:597] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 -> Peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Couldn't send request to peer 283925a3c2d24fb387b016ef4f9c5d69. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:39331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.122965  5410 tablet_server.cc:179] TabletServer@127.5.72.130:0 shutting down...
W20260709 13:22:37.148828 18297 consensus_peers.cc:597] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e -> Peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Couldn't send request to peer 0dc67624917f441fbe92efac4c8d3934. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:37.158844 18223 consensus_peers.cc:597] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 -> Peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Couldn't send request to peer 283925a3c2d24fb387b016ef4f9c5d69. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:39331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.160995  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:37.161721  5410 tablet_replica.cc:333] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.162518  5410 raft_consensus.cc:2243] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.163481  5410 raft_consensus.cc:2272] T 489df491873f484589e87f29b84fc0e6 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.165869  5410 tablet_replica.cc:333] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.166538  5410 raft_consensus.cc:2243] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.167838  5410 raft_consensus.cc:2272] T 72f6d0c0e5d748c8aa14bf23103600d1 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.171018  5410 tablet_replica.cc:333] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.171924  5410 raft_consensus.cc:2243] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.172564  5410 raft_consensus.cc:2272] T 24f07396f12743a89602f33e458543c3 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:37.172582 18297 consensus_peers.cc:597] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e -> Peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Couldn't send request to peer 0dc67624917f441fbe92efac4c8d3934. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.175287  5410 tablet_replica.cc:333] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.176136  5410 raft_consensus.cc:2243] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.176846  5410 raft_consensus.cc:2272] T aed0eb62ebee4d8ea4e76318af5574d0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:37.178033 18297 consensus_peers.cc:597] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e -> Peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Couldn't send request to peer 283925a3c2d24fb387b016ef4f9c5d69. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:39331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.180233  5410 tablet_replica.cc:333] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.181229  5410 raft_consensus.cc:2243] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:22:37.181246 18223 consensus_peers.cc:597] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 -> Peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Couldn't send request to peer 283925a3c2d24fb387b016ef4f9c5d69. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:39331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.182364  5410 raft_consensus.cc:2272] T 07bc88e5b51646d7877a5faf05e53a17 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.184703  5410 tablet_replica.cc:333] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.185276  5410 raft_consensus.cc:2243] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.186118  5410 raft_consensus.cc:2272] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.188652  5410 tablet_replica.cc:333] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.189254  5410 raft_consensus.cc:2243] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.189846  5410 raft_consensus.cc:2272] T 964b1facbeb64496ab5ca09fb2baf747 P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:37.190887 18297 consensus_peers.cc:597] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e -> Peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Couldn't send request to peer 0dc67624917f441fbe92efac4c8d3934. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.192373  5410 tablet_replica.cc:333] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.193161  5410 raft_consensus.cc:2243] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.193666  5410 raft_consensus.cc:2272] T 68bbc1fb66db4f4791fe746af459e9fb P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.195921  5410 tablet_replica.cc:333] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.196484  5410 raft_consensus.cc:2243] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.196955  5410 raft_consensus.cc:2272] T 219dd503f7bd49aab20bc06b6c4779ff P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.199383  5410 tablet_replica.cc:333] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.199934  5410 raft_consensus.cc:2243] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.200768  5410 raft_consensus.cc:2272] T 6d713a8451ee46c2be2578e9a0e3fcef P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.203088  5410 tablet_replica.cc:333] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.203670  5410 raft_consensus.cc:2243] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.204097  5410 raft_consensus.cc:2272] T b7ee78df24c040389e1bec48dcdfe6e0 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.206724  5410 tablet_replica.cc:333] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.207320  5410 raft_consensus.cc:2243] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.207973  5410 raft_consensus.cc:2272] T 4b1808a58b464d5d88dc40b33c33b464 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.210237  5410 tablet_replica.cc:333] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.210825  5410 raft_consensus.cc:2243] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.211297  5410 raft_consensus.cc:2272] T 3fbd032884b645ae846c6f3733dcc917 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.213408  5410 tablet_replica.cc:333] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.213907  5410 raft_consensus.cc:2243] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.214350  5410 raft_consensus.cc:2272] T 5eaa4568017f4ed28ce3d64303829b20 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.216457  5410 tablet_replica.cc:333] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.216938  5410 raft_consensus.cc:2243] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.217272  5410 raft_consensus.cc:2272] T d4697fc0d4ce422c8129cb8d07900a5e P 0dc67624917f441fbe92efac4c8d3934 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.219928  5410 tablet_replica.cc:333] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.220399  5410 raft_consensus.cc:2243] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.221148  5410 raft_consensus.cc:2272] T 854a26ffa5a447e48a8c9778e299eda7 P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.223263  5410 tablet_replica.cc:333] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.223682  5410 raft_consensus.cc:2243] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.224491  5410 raft_consensus.cc:2272] T 0ca3425970884557b2bd8c85a567d4bf P 0dc67624917f441fbe92efac4c8d3934 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.226547  5410 tablet_replica.cc:333] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934: stopping tablet replica
I20260709 13:22:37.227046  5410 raft_consensus.cc:2243] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.227432  5410 raft_consensus.cc:2272] T e32ac98de5674dacaef8332140cb0d2d P 0dc67624917f441fbe92efac4c8d3934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.271574  5410 tablet_server.cc:196] TabletServer@127.5.72.130:0 shutdown complete.
W20260709 13:22:37.295496 18297 consensus_peers.cc:597] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e -> Peer 0dc67624917f441fbe92efac4c8d3934 (127.5.72.130:40155): Couldn't send request to peer 0dc67624917f441fbe92efac4c8d3934. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.130:40155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.312389  5410 tablet_server.cc:179] TabletServer@127.5.72.131:0 shutting down...
I20260709 13:22:37.340281  5410 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:37.340983  5410 tablet_replica.cc:333] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.341763  5410 raft_consensus.cc:2243] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.342415  5410 raft_consensus.cc:2272] T 72f6d0c0e5d748c8aa14bf23103600d1 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.344852  5410 tablet_replica.cc:333] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.345455  5410 raft_consensus.cc:2243] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.345983  5410 raft_consensus.cc:2272] T 489df491873f484589e87f29b84fc0e6 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.348179  5410 tablet_replica.cc:333] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.348760  5410 raft_consensus.cc:2243] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.349280  5410 raft_consensus.cc:2272] T 7dc3d7cd8b134d5fb6d9144e947d2598 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.351445  5410 tablet_replica.cc:333] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.352015  5410 raft_consensus.cc:2243] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.352537  5410 raft_consensus.cc:2272] T 07bc88e5b51646d7877a5faf05e53a17 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.354591  5410 tablet_replica.cc:333] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.355208  5410 raft_consensus.cc:2243] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.355675  5410 raft_consensus.cc:2272] T 24f07396f12743a89602f33e458543c3 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.358868  5410 tablet_replica.cc:333] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.359470  5410 raft_consensus.cc:2243] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.359985  5410 raft_consensus.cc:2272] T aed0eb62ebee4d8ea4e76318af5574d0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.362031  5410 tablet_replica.cc:333] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.362603  5410 raft_consensus.cc:2243] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.363106  5410 raft_consensus.cc:2272] T 964b1facbeb64496ab5ca09fb2baf747 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.365190  5410 tablet_replica.cc:333] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.365757  5410 raft_consensus.cc:2243] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.366211  5410 raft_consensus.cc:2272] T 68bbc1fb66db4f4791fe746af459e9fb P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.368595  5410 tablet_replica.cc:333] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.369271  5410 raft_consensus.cc:2243] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.369951  5410 raft_consensus.cc:2272] T 6d713a8451ee46c2be2578e9a0e3fcef P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.372839  5410 tablet_replica.cc:333] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
W20260709 13:22:37.373006 18297 consensus_peers.cc:597] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e -> Peer 283925a3c2d24fb387b016ef4f9c5d69 (127.5.72.129:39331): Couldn't send request to peer 283925a3c2d24fb387b016ef4f9c5d69. Status: Network error: Client connection negotiation failed: client connection to 127.5.72.129:39331: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:37.373603  5410 raft_consensus.cc:2243] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.374575  5410 raft_consensus.cc:2272] T 219dd503f7bd49aab20bc06b6c4779ff P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.376838  5410 tablet_replica.cc:333] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.377367  5410 raft_consensus.cc:2243] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.378206  5410 raft_consensus.cc:2272] T 5eaa4568017f4ed28ce3d64303829b20 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.380532  5410 tablet_replica.cc:333] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.381081  5410 raft_consensus.cc:2243] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.381873  5410 raft_consensus.cc:2272] T d4697fc0d4ce422c8129cb8d07900a5e P 5b41bb7f2b7847c19b86e54214bf0f4e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.384617  5410 tablet_replica.cc:333] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.385149  5410 raft_consensus.cc:2243] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.385581  5410 raft_consensus.cc:2272] T 3fbd032884b645ae846c6f3733dcc917 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.387840  5410 tablet_replica.cc:333] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.388304  5410 raft_consensus.cc:2243] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.388777  5410 raft_consensus.cc:2272] T 854a26ffa5a447e48a8c9778e299eda7 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.390897  5410 tablet_replica.cc:333] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.391373  5410 raft_consensus.cc:2243] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.391799  5410 raft_consensus.cc:2272] T 0ca3425970884557b2bd8c85a567d4bf P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.393898  5410 tablet_replica.cc:333] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.394374  5410 raft_consensus.cc:2243] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.394814  5410 raft_consensus.cc:2272] T e32ac98de5674dacaef8332140cb0d2d P 5b41bb7f2b7847c19b86e54214bf0f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.397068  5410 tablet_replica.cc:333] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.397506  5410 raft_consensus.cc:2243] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.398227  5410 raft_consensus.cc:2272] T b7ee78df24c040389e1bec48dcdfe6e0 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.400239  5410 tablet_replica.cc:333] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e: stopping tablet replica
I20260709 13:22:37.400636  5410 raft_consensus.cc:2243] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:37.401041  5410 raft_consensus.cc:2272] T 4b1808a58b464d5d88dc40b33c33b464 P 5b41bb7f2b7847c19b86e54214bf0f4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.441686  5410 tablet_server.cc:196] TabletServer@127.5.72.131:0 shutdown complete.
I20260709 13:22:37.477836  5410 master.cc:562] Master@127.5.72.190:44229 shutting down...
I20260709 13:22:37.498466  5410 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:37.499109  5410 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:37.499461  5410 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3bacdc7796454bc687dcc4220430f64a: stopping tablet replica
I20260709 13:22:37.519374  5410 master.cc:584] Master@127.5.72.190:44229 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11821 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (26330 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (411703 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:22:37.579910  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 445 messages since previous log ~15 seconds ago
I20260709 13:22:37.580163  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 445 messages since previous log ~15 seconds ago
I20260709 13:22:37.580348  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 443 messages since previous log ~15 seconds ago
I20260709 13:22:37.580575  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~34 seconds ago
I20260709 13:22:37.580770  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:22:37.581001  5410 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 19 messages since previous log ~272 seconds ago
I20260709 13:22:37.581204  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 6 messages since previous log ~271 seconds ago
I20260709 13:22:37.581406  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 1 messages since previous log ~162 seconds ago
I20260709 13:22:37.581694  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 9 messages since previous log ~0 seconds ago
I20260709 13:22:37.582010  5410 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 48 messages since previous log ~10 seconds ago
